AI agent observability at the service desk
When an AI agent handles a ticket, something happens inside that process that most teams cannot see. The ticket gets triaged, a reply goes out, a note appears, and nobody can answer the question: what exactly did the agent decide, based on what, with how much confidence, and at which point did it decide a human needed to take over?
That gap is not a minor inconvenience. It is the thing that makes AI at the service desk either trustworthy or a black box you are afraid to tune. Observability is the answer: a per-run record of what the agent did, what knowledge it used, what confidence it assigned to each step, and why it escalated or stayed in lane.
This post covers what observability means for a service desk AI agent, what data you actually need per run, how confidence signals work in practice, how shadow mode fits into the picture, and what you can do with the logs once you have them.
Why observability matters more for agents than for scripts
A scripted automation does what the script says. If the script says "if category equals hardware, assign to IT Ops," you can read the script and know exactly what will happen. You can audit it with a text editor.
An AI agent evaluates context and makes a judgment call. That judgment is shaped by the ticket text, the knowledge it retrieved, the confidence of its own prior steps, and the configured thresholds it is working within. None of that is visible from the ticket record alone. The ticket shows the output. Observability shows the reasoning.
This matters for three practical reasons:
First, debugging. When an agent does something unexpected, whether it categorized a ticket wrong, sent a reply the team did not agree with, or failed to escalate a ticket that should have been escalated, you need to trace back what it saw and what it decided. Without a decision log, you are guessing.
Second, improvement. If an agent consistently assigns high confidence to a category that turns out to be wrong, you can only see that pattern if you have per-run confidence scores to analyze. The output alone tells you there are errors. The confidence log tells you where the agent's model of the problem is off.
Third, trust. A team that can open a run record and see exactly what the agent did, step by step, will calibrate their trust in the agent more accurately than a team that sees only the output. Over-trust and under-trust are both problems. Observability is the calibration mechanism.
What a run record should contain
A useful run record for a service desk AI agent covers the full lifecycle of a single ticket event. At minimum it should have:
What triggered the run. The event type (new ticket, status change, new comment), the ticket identifier, the timestamp, and which agent configuration was active.
What the agent read. The ticket content it processed: subject, body, existing notes, attachments considered, the platform the ticket came from, and any prior enrichment already on the ticket from previous runs.
Knowledge retrieved. Which knowledge base articles or known error entries the agent queried for, what the query was, which results came back, and what confidence score the retrieval assigned to the top results. If the agent used a piece of knowledge in its reasoning, that knowledge item should be logged by reference.
Each decision step. For a typical triage run this might be: classify category, classify priority, check for existing knowledge match, decide whether to reply or escalate. Each step should log the input it received, the output it produced, and the confidence score for that output.
The action taken. What the agent actually did: sent a reply, posted a private note, assigned the ticket, escalated, or took no action because confidence was below threshold. If it took no action, the log should say why.
Escalation reason if applicable. When the agent hands off to a human, the reason should be explicit: confidence below threshold on category, low confidence on the knowledge match, sentiment signal above configured limit, or explicit human-only rule for the action type requested.
Run duration and outcome. Total time, whether the run completed normally or was interrupted, and any errors encountered.
This is a lot of data per run. The question is always how to store it accessibly. Storing the full run record in a structured format tied to the ticket identifier means you can query it when something goes wrong, analyze it in aggregate for patterns, and surface a summary to the handling agent when they look at a ticket.
Confidence scores: what they mean and what to do with them
Confidence is the agent's self-assessment of how certain it is about a decision. A confidence score of 0.9 on a category assignment means the agent is quite sure. A score of 0.4 means it has a best guess but is not confident. These numbers are only meaningful if you can calibrate them against outcomes: does 0.9 actually mean the category is right 90% of the time?
Calibration happens over time as you compare the agent's stated confidence against whether its decisions were correct. If the agent says 0.9 confidence on category and is right 85% of the time, the calibration is reasonable. If it says 0.9 and is right 60% of the time, the confidence scores are inflated and your thresholds need adjusting.
This is why the run log needs to record confidence at each step, not just the final action. The final action confidence is the score that triggers escalation. But intermediate confidence, for example low confidence on knowledge retrieval followed by high confidence on the reply, is a signal that the reply may be poorly grounded even though the agent was willing to send it.
Practical threshold management looks like this: set an initial threshold conservatively. Review the run logs after two weeks. Find the distribution of confidence scores and where correct versus incorrect decisions cluster. Adjust the threshold to a point where the incorrect decision rate falls to a level you accept. Then do this again after the next category of tickets enters the mix.
More on the mechanics of human oversight thresholds is in when AI should hand off to a human.
Shadow mode as the first phase of observability
Before you care about tuning thresholds on an active agent, you need a baseline: what would the agent have done on real tickets, and how often would it have been right?
Shadow mode solves this. In shadow mode the agent processes tickets and writes its decisions, proposed actions, and confidence scores as internal notes rather than taking action. The ticket is handled by humans as normal. The shadow note sits alongside the ticket for comparison.
After a period in shadow mode you have a dataset: for every ticket the agent saw, you know what it would have done, with what confidence, and what the human actually did. That comparison tells you where the agent agrees with your team (and therefore what you might safely automate), where it disagrees but the team agrees the agent's approach was fine (configuration issue), and where it disagrees in ways that would have caused real problems.
Shadow mode is also the right starting position for a new agent type or a new ticket category. If you add a new product to your service catalog and want the AI to handle tickets in that category, start in shadow mode on that category specifically. Promote to active once the shadow performance meets the threshold you set. The full approach is in rolling out AI in shadow mode.
Confidence per knowledge item, not just per action
A specific observability gap that causes problems in practice: the confidence score on the final action conflates the confidence of the reasoning with the confidence of the knowledge the reasoning was based on.
Consider: the agent retrieves a knowledge article about a password reset procedure. The retrieval confidence is 0.95 because the ticket clearly matches. The agent is then asked to write a reply using that article. It assigns high confidence to the reply. But if the knowledge article is outdated (the procedure changed last month), the reply will be wrong despite high confidence at every step.
Logging the confidence of the knowledge retrieval step separately from the confidence of the reply step lets you identify this pattern. If knowledge retrieval confidence is high but reply quality is low, you have a knowledge base accuracy problem. If retrieval confidence is low but the agent acted anyway, you have a threshold problem.
This is connected to why hallucinations at the service desk are often a knowledge problem rather than a model problem, discussed further in AI hallucinations at the service desk.
What you can do with observability data in practice
Once you have a run log per ticket, several practical operations become possible:
Error investigation. When a user escalates a complaint that an AI reply was wrong, you open the run record and trace exactly what knowledge the agent used, what confidence it had, and what it read. That is a five-minute investigation rather than a half-day one.
Threshold calibration. Export the distribution of confidence scores per action type, compare against outcome correctness, and set thresholds based on evidence rather than gut feel. Run this quarterly or after any significant change to your knowledge base or ticket mix.
Knowledge base gap detection. Where is the agent consistently retrieving low-confidence matches or returning no match at all? Those topics are your knowledge base gaps. Adding articles in those areas will improve agent performance before you touch any thresholds.
Agent performance comparison. If you are running multiple agent configurations, whether for different ticket categories, different platforms, or testing a configuration change, per-run confidence logs let you compare them on a common scale.
Audit for regulated environments. In some environments, particularly those handling personal data or regulated services, being able to show that an AI took a specific action, on what basis, and with what oversight mechanism in place is a compliance requirement. A per-run structured log satisfies this where a general "we use AI" statement does not. For more on how ITSM Autopilot handles data processing and audit controls, see the security page.
What escalation logs should capture
Escalation is the moment where observability has the most immediate value, because it is the moment where the system explicitly hands control to a human. The escalation log should capture:
- Which step in the run triggered the escalation
- The confidence score at that step and the configured threshold
- Whether the escalation was due to confidence, a hard rule (action type is human-only), or a sentiment signal
- The text the agent was about to act on when it stopped
- What internal note (if any) the agent left for the handling human
See the broader discussion of human-agent handoff patterns in when AI should hand off to a human.
Keeping observability data useful over time
Run logs accumulate fast in a service desk environment. The risks are: too much data to search usefully, logs so large they slow down the main application, and logs that are technically present but never actually consulted.
Practical choices that help:
Store run logs separately from the main ticket record but linked by ticket identifier. Querying the full log for investigation is different from displaying the ticket to an agent, and the storage and access patterns should reflect that.
Summarize for the handling agent. The full run log is for investigation and calibration. What a handling agent sees on the ticket should be a short summary: what the AI did, with what confidence, and the key knowledge reference it used. The detail is one click away.
Set a retention policy. Detailed run logs at 90 days with summary records retained longer is a reasonable starting point. Adjust based on how often you actually use logs older than 90 days for investigation versus calibration.
Flag low-confidence runs for review rather than storing them identically to high-confidence ones. If 20% of runs have confidence below 0.5 on a key step, those are the ones worth sampling for quality review, not the high-confidence runs that almost certainly produced the right result.
Frequently asked questions
What is AI agent observability at the service desk?
It is a per-run record of what an AI agent decided, on what basis, with what confidence, and what action it took or declined to take. It covers the full reasoning process for a single ticket event, not just the final output.
Why is confidence logging more useful than accuracy logging?
Accuracy logging tells you the agent was right or wrong after the fact. Confidence logging tells you, before the fact, how certain the agent was. Together they let you calibrate: does high confidence actually predict correctness for your ticket mix? That calibration is how you set thresholds that work in practice.
Does shadow mode give you observability data?
Yes. Shadow mode is the clearest form of observability because the agent writes everything it would have done as a visible record, and a human then does the actual work. Comparing the two is the purest form of calibration. Active agents with run logs provide observability on live decisions instead.
How detailed does the run log need to be?
Enough to answer "why did the agent do that?" for any individual run, and enough to run aggregate analysis on confidence distributions and knowledge retrieval patterns across many runs. The full reasoning trace is not always needed for every ticket, but it should be reconstructible from the log when needed.
When should I adjust confidence thresholds?
After reviewing the distribution of outcomes against confidence scores across at least a few hundred runs. Not immediately after a single bad outcome (too little data), and not never (thresholds set on day one may not fit the ticket mix you actually see at month three). Quarterly review is a reasonable default.
Real service by real people. Administrative work by machines. Observability is what makes the machine's work auditable.