About a year ago, something strange was happening with one of our production monitors. Its score kept coming back the same. The monitor measures procedure instruction adherence: whether an agent actually followed the steps prescribed by the methodology. For weeks, it returned the same low mark, run after run. A steady score reads as a steady system, and that's what we thought. The agents were falling short of the procedure, but at least they were falling short consistently. So, a bounded problem: we could locate, diagnose, and repair that.
It turns out the consistency wasn't really there. Our deterministic evaluator returned only three values, and the real variation in how agents followed, skipped, or reordered the procedure was being compressed into a scale too coarse to show it. The flat line was not simply a property of the instrument, it was a property of the relationship between the instrument and the variation in the work. And an instrument that can't express instability will report stability forever (despite very real instability).
That version of our evaluation records didn't give us a way to separate these two cases. The records carried a score, but almost nothing about the judgment that had produced it: which evaluator was running, which version, which criteria it had applied, which distinctions it could express, or which part of the work it described. Admittedly, none of those fields would have diagnosed the problem for us. They would, however, have made it possible for us to locate the source of that apparent stability so that we could ask whether the work had stopped changing or whether our instrument had simply stopped being able to describe the changes.
When a record carries only the result, the properties of the measuring instrument and the properties of the thing being measured are fused together. When you scale, the fusion happens silently, thousands of times a day, across every system watched.
There are systems where this fusion costs almost nothing. When the quantity is simple and the instrument has been standardized into the background, a bare number is enough: nobody asks a thermometer for its provenance, because temperature was settled long ago, telling us what the instrument is, how it behaves, and what a degree means. So the coordinates still exist, and were investigated in advance, for everyone. The complexity inherent in expert work probably won't lead to that kind of standardization. For complex systems, the criteria are plural, contested, and always moving. And now, we have these new instruments that aren't completely trustworthy. The same piece of work can be judged differently from different positions for legitimate reasons. Those are the domains we work in, and in them a number arriving alone is not a simplification. It's a claim with its evidence removed. So the score travels, but the judgment that gave it meaning does not.
Judgment is being stripped from the handoff
The temptation is to treat this as a tooling problem, and, honestly, our immediate fix looked like one: identify the evaluator, version it, and retire the three-point scale. But the reason the failure sat undetected for weeks was not only that a field was missing. We had built a record that could not identify the conditions of its own judgment, and then trusted it as evidence about someone else’s work.
I have argued before that the intelligence of expert organizations lives partly in the connections between people: in what gets passed upward, what gets corrected, what gets escalated, and what gets absorbed. Hierarchy isn't only a way of allocating authority, it's also a structure through which people positioned at different scopes of the work pass partial judgments to one another. No one sees the whole system directly. The organization becomes capable of judgment by preserving enough signal between those different views.
This is not a new understanding of cognition. Distributed-cognition research has long treated the representations moving among people and artifacts as part of the cognitive system itself, while human-computation research has shown that an annotation is not a context-free label: who judged, how they interpreted the task, why they disagreed, and what evidence they used may all be part of the signal. The agentic problem is what happens when those representations become production records, move continuously across scopes, and have to support both correction of the work and revision of the instruments observing it.
As more of the work becomes agentic, some of those handoffs themselves become machine-mediated. Agents perform the work, then evaluators inspect particular properties of it, then records carry the resulting observations into monitoring systems, review processes, and decisions made elsewhere. The record is not the whole channel, but it has become part of it. When it carries only a score, the receiving layer gets the conclusion without the basis for understanding what was judged, what the instrument could see, or how much confidence to place in the result. So the handoff is there, but much of the judgment has been stripped out of it.
Meanwhile, I'm watching the conversation about evaluating agentic systems largely collapse into a single word: outcome. Did the run get where it was supposed to go? It's an understandable step, because it sounds like accountability and produces a number that can be compared. But an outcome is one judgment made at one scope of the work. A system can reach an acceptable answer while skipping required procedures, mishandling dependencies, concealing uncertainty, or requiring repeated human correction. It can succeed at the output while failing in execution, coordination, handoff, or fitness.
Judgment should really follow the structure of the work. Each scope exposes a different object to judgment, to a differently positioned reviewer, with different evidence available. What should move between those scopes isn't a universal score. What should move is the structured trace of judgment being exercised: what was assessed, by whom or what, against which criteria, at what scope, on the basis of which evidence, and with what uncertainty or consequence.
I want to be clear: that trace does not contain the expert or reproduce everything the expert knows. But, it carries enough of the expert’s position in relation to the work for another participant to locate the judgment, interpret it, and respond.
Different scopes expose different properties of the work to differently positioned judges. Verification depends not on one score spanning the hierarchy, but on judgments remaining interpretable as they move between its layers.
A judgment needs coordinates
A record can't carry expertise, at least not in any complete sense. Expertise includes accumulated experience, tacit distinctions, and the capacity to recognize when the available categories no longer fit the case in front of you. What a record can carry is a trace of expertise being exercised on a particular piece of work, and in our own evaluation results, three coordinates of that trace were repeatedly missing in ways that made judgments nearly impossible to interpret once they left the systems that produced them: provenance, reference set, and scope.
The first, provenance, carries who or what made the judgment, by which method, and in which version. The flat line from the opening remained uninterpretable for weeks because the record could not distinguish a deterministic matcher from an LLM evaluator or a human reviewer, and those instruments do not merely differ in quality, they fail in different ways. A deterministic matcher may be too coarse or too rigid for the variation emerging in the work. An LLM evaluator may change when its underlying model or prompt changes. Human reviewers may apply a standard differently as their experience, instructions, or local conditions change. Once the record identifies the judge, a property of the instrument no longer has to masquerade as a property of the system, and we can at least ask the question that took us weeks to formulate: has the work changed, or has the judgment process changed?
The second is the reference set: the cases, examples, or prior population in relation to which the judgment was made. A result is never simply a property of the system; it is a property of the system encountering particular work under particular evaluative conditions, and the same evaluator may produce very different distributions over routine cases and unusual ones. Naming the reference set does not make two results comparable. It makes it possible to determine whether they can responsibly be compared.
The third is scope, because judgments travel farther than the work they describe. They leave the run that produced them and appear in dashboards, quality reports, and decisions made by people who never observed the underlying execution. Was the object of judgment a single tool call, a procedural step, an entire run, or a pattern across many runs? When scope is ambiguous, a local result gets stretched to support a broader conclusion: a step-level check becomes evidence of end-to-end quality, and the number stays the same while the claim made from it quietly expands.
This is where our work in OpenTelemetry fits. The GenAI semantic-conventions effort is developing a shared vocabulary for agentic tasks and evaluation results across tools, and our contribution addresses one narrow part of the problem: making the evaluation event retain its provenance, reference set, and scope. A telemetry standard can't capture expertise, decide what an organization should judge, or ensure that anyone acts on the result; it can keep part of the judgment from being discarded at the handoff. Nor is the point that the record will read identically wherever it lands. Different recipients may challenge the criteria or disagree with the result, and that disagreement is the next level of judgment doing its job, not a failure of standardization.
A judgment event should not hold the work still. It gives the people and systems around it a common way to say where they are.
The outcome can't report absences
A judgment event can only describe work that produced one. A case that never ran emits no result, a skipped step leaves no evaluation of that step, and a handoff that never happened produces no correction or escalation, so every event that does arrive may be well-provenanced and correctly scoped while the record stays collectively silent about the work that disappeared.
Coverage therefore has to come from somewhere outside the events themselves: an expectation about which cases were in scope, which procedures and dependencies mattered, and where review should have occurred. Comparing the work to that expectation is not mechanical, though, because a skipped step may be a failure, a reasonable adaptation, or evidence that the prescribed route no longer fits the work, and telling those apart takes someone positioned broadly enough to understand what the missing work would have contributed.
Expert organizations have always worked this way. For example, an audit reviewer does not infer completeness from the workpapers that happen to be present. The engagement defines the population, procedures, risks, and review responsibilities against which the file is judged, and "everything we examined was fine" is a different claim from "we examined everything required."
Agentic systems should get the same skepticism. A run can reach the expected output while omitting procedures, avoiding difficult cases, or failing to invoke a review that should have occurred, and an outcome evaluator will score everything it sees correctly while being able to certify only the surviving path. The system passed every check that happened, but that's not evidence that the necessary checks happened.
The work does not need to stand still for measurement
If we have an expected route, we can recognize omission. But if we don't want every run to have to follow the same path, or hold the route itself fixed. Expert work changes as people encounter new cases, revise procedures, and learn that something they had treated as variation is actually a different kind of problem. A verification system has to preserve that capacity to move.
A dominant benchmark does more than rank what already exists. It shapes which forms of value can register at all: work that fits its categories appears as progress, while work whose value lies along another dimension may remain invisible until that dimension becomes important (Lotfi et al. 2026). The answer is not necessarily a larger benchmark. No fixed measure can anticipate every distinction that will matter, so we need to create systems where new distinctions can enter the record, travel across scopes, and eventually become stable enough to measure.
To be fair, this constancy requirement was never about the work. It came from the limitations of our instruments: once several things moved at once, we could no longer tell whether the system had changed, the cases had changed, or the evaluator had drifted, so we held everything still to preserve attribution. A judgment system offers another way out. If judgments are preserved consistently enough (what was observed, from which position, against which evidence, and what happened in response), we do not need to decide every relevant comparison in advance. We can discover, later, which results belong together and which differences reveal that the work, or our understanding of it, has changed. The required stability might not be in the task or the metric but in the communication structure around them. If so, we can standardize the coordinates without standardizing the conclusion.
A cockpit is a better model for that kind of system than a scoreboard. It does not hold the aircraft, the weather, or the route still--instead its instruments provide partial views of a changing situation. No reading determines the right action on its own, and the instruments are not the crew’s only source of evidence. People situated in relation to the work can compare those readings with the case in front of them, notice conditions the instruments were not designed to capture, and importantly challenge those instruments when they no longer agree. Cockpit instruments keep differently-positioned participants oriented to the same moving system.
The instruments for agentic work should serve the same purpose. A procedural evaluator may show that a required method was not followed, while a human correction may reveal that the method no longer fits the case. A pattern across runs could show a local exception becoming a population-level change. These aren't competing attempts at the one true score. They are bearings taken at different altitudes and scopes, each partial but also capable of changing what the organization understands about the route.
The goal, then, is not to make the work stand still long enough to measure it. It is to preserve the organization's capacity to locate itself while the work moves. We don't have to hold everything constant. We need to hold the communication open.
The expert moves with the loops
The beauty is that agentic systems change what can be observed. Their work unfolds through tool calls, intermediate decisions, corrections, and responses that leave traces at a granularity earlier automation couldn't afford to capture. And, the same underlying technology can inspect those traces, extending judgment across more of the work than human review alone could sustain.
The instruments are not experts, though. They haven't spent years inside the review hierarchies that teach practitioners which distinctions matter, which deviations are defensible, and which apparently small failures change the meaning of the whole. This is why a model can detect a departure from a route without knowing whether the route or the departure is wrong.
The opportunity, then, is not more scores. It is a wider field of view for judgment, and a more durable path through the organization for what experts learn from the work. A local correction can become evidence of a recurring failure. A disagreement can reveal an unstable standard. A pattern across runs can show that the population, procedure, or purpose has shifted.
Reliability comes from building a system in which judgment can keep moving, so that the organization can keep learning where it is.
There is a further question waiting behind this one. Once we can locate the work, the judgment applied to it, the ground it covered, and what never happened, we can ask something harder: what was any of it worth? Calculating the value of the work is where I’m headed next.
References:
Aroyo, L., and Welty, C. "Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation." AI Magazine 36(1), 2015, 15–24. https://doi.org/10.1609/aimag.v36i1.2564
Hollan, J., Hutchins, E., and Kirsh, D. "Distributed Cognition: Toward a New Foundation for Human-Computer Interaction Research." ACM Transactions on Computer-Human Interaction 7(2), 2000, 174–196. https://doi.org/10.1145/353485.353487
Lotfi, S., Iranmanesh, A., Naghashyar, L., Shirali, A., Nateghi Haredasht, F., Koyejo, S., Torr, P., Lee, Y. S., Barez, F., Lehman, J., Norvig, P., and Narayanan, A. "Position: There Are Futures That Benchmark-Driven AI Cannot See." ICML 2026, oral. https://icml.cc/virtual/2026/oral/71151 (paper PDF on OpenReview: https://openreview.net/pdf/c0833e1fef52e998ef8d65944caaa3aae0eaa35c.pdf)
OpenTelemetry GenAI Semantic Conventions, gen_ai.evaluation.result event. https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-events.md
Srivats, M. "Add evaluator provenance attributes to gen_ai.evaluation.*," open-telemetry/semantic-conventions-genai, PR #359. https://github.com/open-telemetry/semantic-conventions-genai/pull/359
