{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreig2piwxqujfcdmtk3fxmwiwxk7hrtqnfqhiygsupxkz6lgevl4jsa",
    "uri": "at://did:plc:qz6ohvpdsdvv5kniizyfz25y/app.bsky.feed.post/3mlclzmlzqm52"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreihxkm5qiu6ztnn3bykqidgnbrkurdmo56cyxzifcofiztlu4fnyly"
    },
    "mimeType": "image/jpeg",
    "size": 5684079
  },
  "path": "/article/4167999/when-ai-writes-code-it-joins-the-software-supply-chain.html",
  "publishedAt": "2026-05-07T10:00:00.000Z",
  "site": "https://www.cio.com",
  "tags": [
    "Artificial Intelligence, Development Tools, Generative AI, Software Development",
    "interact with development pipelines with limited human intervention",
    "24% properly evaluate AI-generated code for security and quality risks",
    "non-human identity",
    "cybersecurity disclosure requirements",
    "Want to join?"
  ],
  "textContent": "AI tools designed to assist developers are no longer staying in the background. They are starting to shape what actually gets built and deployed.\n\nThey open pull requests.\n\nThey modify dependencies.\n\nThey generate infrastructure templates.\n\nThey interact directly with repositories and CI/CD pipelines.\n\nAt some point, this stops being assistance.\n\nIt becomes participation.\n\nAnd participation changes the problem.\n\n## When assistance becomes participation\n\nThe shift from generative to agentic behavior is the inflection point.\n\nEarlier tools operated inside a tight loop. A developer prompted. The system suggested. The developer reviewed. Nothing moved without human intent.\n\nThat boundary is eroding.\n\nNewer systems propose changes, update libraries, remediate vulnerabilities and interact with development pipelines with limited human intervention. They don’t just accelerate developers. They begin to shape the artifacts that move through the software supply chain — code, dependencies, configurations and infrastructure definitions.\n\nThat makes them something different.\n\nNot tools.\n\nParticipants.\n\nAnd once something participates in the supply chain, it inherits the same question every other participant does:\n\nHow is it governed?\n\n## A simple scenario\n\nConsider a common pattern already emerging in many environments.\n\nAn AI system identifies a vulnerable dependency.\n\nIt opens a pull request updating the library.\n\nA workflow triggers automated tests.\n\nThe change is promoted into a staging environment.\n\nFour steps.\n\nNo human review.\n\nNo explicit governance checkpoint.\n\nEach step is individually valid. Nothing looks wrong in isolation.\n\nBut taken together, they create something fundamentally different: A system that can change enterprise software without human intent being re-established at any point. Research from Black Duck found that while 95% of organizations now use AI in their development process, only 24% properly evaluate AI-generated code for security and quality risks.\n\nThis is autonomous change propagation across the software supply chain.\n\n## The “human-in-the-loop” fallacy\n\nMany organizations rely on a “human-in-the-loop” (HITL) requirement as a safety mechanism for AI-generated code.\n\nAt low volumes, this works.\n\nAt scale, it breaks.\n\nWhen an AI system generates dozens of pull requests in a short window, review becomes a throughput problem, not a control. The cognitive load of validating machine-generated logic exceeds what a human can realistically govern.\n\nWhat remains is not oversight, but a checkpoint.\n\nAnd checkpoints without effective review are not controls.\n\n## The governance gap\n\nMost governance models assume a stable truth: Humans are the primary actors.\n\nControls tied identity to individuals, approvals to intent and audit trails to accountability.\n\nEven automation systems are treated as extensions of human intent — predictable, bounded and deterministic.\n\nAI systems break that model.\n\nThey can generate new logic, act on it and propagate changes across systems. Yet in most environments, they are still governed as if they were static tools.\n\nThat mismatch is the gap.\n\n## Machine identity is no longer what it was\n\nOne way to see this clearly is through identity.\n\nEvery interaction an AI system has — repository access, pipeline execution, API calls — requires credentials. In practice, these systems operate as machine identities.\n\nBut they are not traditional machine identities.\n\nA service account executes predefined logic. Its behavior is known in advance. Its risk is bounded by what it was configured to do.\n\nAn AI-driven system is different. It generates the logic it then executes.\n\nIt can propose new code paths, interact with new systems and trigger actions that were not explicitly predefined at the time access was granted.\n\nThat is a category change.\n\nNot just a new identity type, but a new attack surface: _Identities that can generate the behavior they are authorized to execute._\n\nThe World Economic Forum has identified this class of non-human identity as one of the fastest-growing and least-governed security risks in enterprise AI adoption.\n\n## Measuring exposure before solving it\n\nMost organizations already track access-related metrics. Those metrics were designed for human-driven systems.\n\nThey are no longer sufficient.\n\nIf AI systems are participating in the software supply chain, organizations need to measure where and how that participation introduces risk.\n\nA few signals matter immediately:\n\n  * AI-generated artifact footprint: What portion of code, dependencies or infrastructure definitions in production originates from AI-assisted processes?\n\n\n  * Authority scope of AI systems: What systems can these identities access — and what actions can they take across repositories and pipelines?\n\n\n  * Autonomous change rate: How often are changes introduced and propagated without explicit human review?\n\n\n  * Cross-system interaction surface: How many systems does a single AI workflow touch as part of normal operation?\n\n\n  * Auditability of AI-driven actions: Can changes be traced cleanly to a system, workflow and triggering context?\n\n\n\n\n\nThese are not abstract concerns. They are measurable.\n\nAnd until they are measured, they are not governed.\n\n## The regulatory imperative\n\nThis is not just a technical shift. It is a governance and liability shift.\n\nAs regulatory expectations evolve — from AI accountability frameworks to cybersecurity disclosure requirements — organizations are increasingly responsible for explaining and controlling automated decisions inside their environments.\n\nIf an AI-driven change introduces a vulnerability or leads to a material incident, “the system generated it” will not be an acceptable answer.\n\nAccountability will still sit with the enterprise.\n\nThat raises the bar: Governance must extend to how autonomous systems act, not just how they are accessed.\n\n## The architecture gap\n\n_AI systems operate horizontally across systems, while governance remains vertical_\n\nPuneet Bhatnagar\n\nThe issue is not that any one control is missing.\n\nIt is that AI systems operate across the seams of systems designed to govern within their own boundaries.\n\nRepositories enforce code controls.\n\nPipelines enforce deployment controls.\n\nIdentity systems enforce access controls.\n\nSecurity tools enforce policy checks.\n\nEach works as designed.\n\nBut AI systems move across all of them.\n\nThey read from one system, generate changes, trigger another and influence a third. Authority is exercised across systems, while governance remains within them.\n\nThat is the architectural gap.\n\n## A different governance model\n\nMost organizations will respond to this shift by trying to extend existing access controls. That instinct is understandable — and insufficient.\n\nThe problem is no longer just who or what can access a system. It is how control is maintained when authority can generate new actions dynamically.\n\nThis requires a different model of governance.\n\nOne that treats software systems as actors whose behavior must be bounded, observed and continuously evaluated across workflows — not just permitted or denied at a point of access. Governance becomes less about static permissions and more about controlling the shape and impact of actions across systems.\n\nThat is the shift.\n\n## Conclusion\n\nThe conversation around AI in software development often focuses on productivity.\n\nBut as AI systems begin to participate in producing and modifying enterprise software, the more important question becomes governance.\n\nAI is not just accelerating the software development lifecycle. It is becoming part of the software supply chain itself.\n\nAnd that changes the problem.\n\nThe challenge for CIOs is no longer just managing developers, tools or pipelines. It is understanding and governing the authority that software systems exercise across them.\n\nBecause in a world where software can act on behalf of the enterprise, governance is no longer just about access.\n\nIt is about authority — what systems are allowed to do, and how that authority is controlled and measured over time.\n\n**This article is published as part of the Foundry Expert Contributor Network.**\n**Want to join?**",
  "title": "When AI writes code, it joins the software supply chain"
}