{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreigxsbzaszcytaihquw5sfc64fun54cnlrmhkl7jvs6hdax6b5r6g4",
    "uri": "at://did:plc:uwj5fyuv3lbhhoybn5hnrqx4/app.bsky.feed.post/3mnx2z75kjve2"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreifnzw4nbwh6menoyncwblrstprjp3e4bfz4jv3643pgzgyiblh4de"
    },
    "mimeType": "image/png",
    "size": 1055512
  },
  "description": "Incident agents should earn lower handoff rates through gated MCP access, RBAC enforcement, traceable tool calls, replay evals, and canary rollout phases.",
  "path": "/lower-incident-agent-handoffs-only-after-mcp-tool-access-is-policy-gated/",
  "publishedAt": "2026-06-10T15:53:09.000Z",
  "site": "https://blog.tuguidragos.com",
  "tags": [
    "Dynatrace reported",
    "tool annotations are risk hints, not enforcement contracts",
    "RBAC good practices",
    "authorization hardening, routable remote servers, cacheable tool lists, and traceable infrastructure",
    "invoke_agent span, child chat spans, and execute_tool spans",
    "agent evals",
    "canary-25, canary-50, canary-75, and stable"
  ],
  "textContent": "Incident agents become materially different once they can call operational tools through MCP. At that point, the risk is no longer whether the model writes a useful summary, but whether it can touch production systems under the right constraints. In a 919 leader survey, Dynatrace reported that 69% of agentic AI decisions are still verified by a human, even as IT operations and DevOps are among the leading uses. The evidence here is limited: the headline verification figure is a Dynatrace self-reported survey result and is not yet independently corroborated. No high-confidence public source in this set directly measures human_handoff_rate for MCP-connected incident agents. The practical recommendation is therefore conditional: treat MCP access as progressive delivery, and lower handoffs only after approved incident classes show stable automation coverage under enforced policy, complete traces, passing evals, and clean canary gates.\n\n## Replace handoff ambition with scoped automation coverage\n\nThe wrong first question is how to tune prompts until the agent asks fewer humans for help. The better first question is which incident classes and runbook actions are eligible for autonomous execution at all. Dynatrace also reported that 72% of respondents use AI agents for IT operations and DevOps, while 52% cite security, privacy, or compliance concerns and 51% cite technical challenges managing and monitoring agents at scale. That combination supports a conservative operating model: adoption is real, but governance and observability remain limiting factors.\n\nAutomation_coverage should become the rollout control metric before human_handoff_rate becomes the optimization target. For example, an incident class might be eligible for autonomous evidence gathering, but not remediation. A runbook step might allow log search, topology lookup, or ticket enrichment, while blocking namespace modification, token creation, or webhook changes. This separates the business goal of fewer interruptions from the control question of what the agent is allowed to do without an operator.\n\nThat distinction matters because MCP changes the boundary between recommendation and action. A chat-only incident assistant can be reviewed like an advisory system. An MCP-connected incident agent needs release discipline similar to a production service change. The safe sequence is map actions, enforce policy, instrument dispatch, evaluate trajectories, then canary new capability sets.\n\n## Risk-class every runbook action before exposing tools\n\nMCP tool annotations are useful for vocabulary, but they are not sufficient for control. The MCP project’s guidance says tool annotations are risk hints, not enforcement contracts. It also notes that if a proposal depends on an annotation being true, enforcement belongs in authorization, transport, host, or runtime layers rather than in the annotation itself. For incident agents, this means a tool marked read-only should still be constrained by identity, scope, and runtime policy.\n\nA practical risk model starts with runbook verbs. Evidence collection is usually the first class: list pods, retrieve logs, inspect events, read deployment metadata, query monitors, or summarize prior tickets. Preparation actions come next: draft a rollback plan, open a change, add context to an incident record, or request approval. Remediation actions are the last class: restart workloads, patch namespaces, modify routing, create credentials, or change admission behavior. MCP tools should be grouped according to these action classes, not exposed as a flat catalog.\n\nKubernetes guidance shows why the detail matters. Its RBAC good practices warn that access to the nodes/proxy subresource can reach the kubelet API, may allow command execution on pods on the node, and bypasses audit logging and admission control. The same guidance calls out create rights on serviceaccounts/token, control over mutating or validating webhook configurations, and namespace label modification as permissions that can weaken credential, admission, or isolation boundaries. Those are not generic admin risks; they are exactly the kinds of edges an incident agent might encounter during remediation.\n\n## Enforce least privilege where the model cannot override it\n\nThe core design principle is deny by default. The model may decide which approved tool to request, but it should not decide whether a tool is allowed for its identity, incident class, environment, or current rollout phase. RBAC, authorization policy, host policy, and runtime checks should enforce the boundary outside the prompt. A production incident agent should receive a narrow service identity, a tool allowlist, scoped resource access, and explicit denials for high-risk verbs until those actions have passed evaluation and rollout gates.\n\nThis policy should be contextual. A read-only diagnostic tool might be allowed in production for severity triage, while a patch action is allowed only in a non-production environment or only after human approval. A namespace action might be blocked if it can affect Pod Security Admission or NetworkPolicy behavior. A token operation should be treated as credential issuance, not as a routine helper call. The goal is to make unauthorized tool attempts visible and harmless, rather than relying on the agent to remember boundaries.\n\nMCP’s direction of travel makes this model more feasible, but does not remove the need for enforcement. The 2026 release candidate describes authorization hardening, routable remote servers, cacheable tool lists, and traceable infrastructure. Those capabilities can support enterprise controls, especially when remote MCP servers are treated as governed services rather than model-side conveniences. Still, the control decision should remain external: the agent asks, the policy layer decides, and the trace records the outcome.\n\n## Trace, evaluate, and canary before changing human review\n\nEvery MCP dispatch should be an auditable event, not just a hidden step inside an agent transcript. OpenTelemetry’s GenAI semantic conventions describe traces with a top-level invoke_agent span, child chat spans, and execute_tool spans. They also cover token usage and operation duration metrics. Because prompts, tool arguments, and tool results can contain sensitive content, the guidance makes content capture opt-in by default. That fits incident operations, where traces should show what happened while redacting or externally storing sensitive payloads.\n\nGood traces make three rollout signals available. First, policy-deny metrics show whether the agent is repeatedly trying to cross boundaries. Second, unauthorized-tool-attempt counts reveal whether the runbook mapping or prompts are misaligned with the allowlist. Third, latency, retries, and tool-result visibility expose whether autonomy is creating operational drag. These are better gates than subjective confidence in a final incident summary.\n\nPre-production evals should inspect complete behavior, not just the last answer. Anthropic’s guidance on agent evals recommends grading trajectories that include outputs, tool calls, intermediate results, and environment interactions. For incident agents, that means replaying representative incidents, giving the agent the same constrained tools it will have in production, and checking whether it selected allowed actions, respected denials, produced useful evidence, and avoided unsafe remediation. Narrow evals should come first, then more complex multi-turn scenarios.\n\nCapability rollout can then borrow directly from canary deployment practice. Google Cloud Deploy documents canary phases such as canary-25, canary-50, canary-75, and stable, with deploy analysis gates and HALT or FAILED states when rollout conditions are not met. For MCP incident access, the percentages need not represent traffic alone. They can represent eligible incident classes, approved runbook actions, environments, or tool categories. A capability set advances only if eval pass rates, policy-deny patterns, trace completeness, rollback behavior, and escalation behavior stay within the operator’s thresholds.\n\nThis is where the earlier human verification signal becomes operationally useful. If most agentic decisions are still being checked by people, the answer is not to remove reviewers first and hope the model improves. The answer is to make autonomy earn its scope. Start with read-only evidence collection, add low-risk preparation actions, and reserve remediation for later phases with explicit approvals or narrow blast radius.\n\nLower human_handoff_rate only after MCP tool access is policy-gated, observable, evaluated, and canaried for the specific incident classes in scope. The operator move is to expand approved automation coverage first, then reduce handoffs as a consequence of stable controls rather than as a prompt-tuning target.",
  "title": "Lower Incident-Agent Handoffs Only After MCP Tool Access Is Policy-Gated",
  "updatedAt": "2026-06-10T15:53:10.141Z"
}