# AI Liability in Defense Contracting: Who Is Responsible When Military AI Fails **By Dan Cumberland** · Published May 19, 2026 · Categories: AI Strategy > The principle is simple. AI systems are tools. Tools don't have legal standing. When a tool produces harm — whether intentional, negligent, or unforeseeable —... ## Why Responsibility Can't Shift to AI **Responsibility for AI system outcomes stays with the humans and organizations across the AI supply chain — developers, deployers, operators, and supervisors\. AI systems lack legal personhood and cannot bear liability\.** The principle is simple\. AI systems are tools\. Tools don't have legal standing\. When a tool produces harm — whether intentional, negligent, or unforeseeable — the law looks for the person or organization that designed it, integrated it, ran it, or oversaw it\. That's not a quirk of common law\. It's a deliberate framework\. According to the National Telecommunications and Information Administration[2](/blog/digital-engineering-for-defense-summit#ref-2), accountability for AI systems anchors to the people and organizations who develop and deploy them, not the systems themselves\. Courts are already enforcing this\. Legal analysis from Taylor Wessing[3](/blog/digital-engineering-for-defense-summit#ref-3) documents that "AI did it" defenses fail in negligence cases\. Judges instead examine whether the developer implemented best practices, whether the deployer integrated the system responsibly, and whether the operator exercised reasonable oversight\. If those answers don't hold up, liability attaches\. It doesn't disappear into the model weights\. The international consensus is just as clear\. Fifty\-eight nations have endorsed the U\.S\. State Department's Political Declaration on Responsible Military Use of AI and Autonomy[4](/blog/digital-engineering-for-defense-summit#ref-4), which states that responsibility for the use of force rests with human beings, not machines, and that states must build safeguards so a human being remains at the end of every algorithmic chain\. That language wasn't written for philosophers\. It was written for the people who buy, build, and deploy military AI systems\. Here's what makes this harder than it sounds: responsibility doesn't sit at one node\. RAND research on AI liability[5](/blog/digital-engineering-for-defense-summit#ref-5) establishes that responsibility is distributed across the supply chain\. Developers carry responsibility for system design\. Deployers carry responsibility for integration and configuration\. Operators carry responsibility for oversight\. Supervisors carry responsibility for governance\. None of them can shed liability by pointing at the others\. Both are true\. All of it matters\. For defense contractors, the practical implication is straightforward\. If you sell an AI\-enabled capability, you are accountable for what it does\. If you deploy someone else's AI inside your system, you are accountable for what it does\. If you supervise an operator who used AI to make a decision, you are accountable for what it does\. There is no node in the chain that the responsibility flows past\. ## The Responsibility Gap Problem **A responsibility gap is a governance failure, not a technology failure\. It occurs when the distributed structure of an AI supply chain — developer plus vendor plus deployer plus operator plus supervisor — allows every stakeholder to disclaim liability, leaving no one clearly accountable\.** Peer\-reviewed research from Springer Nature[6](/blog/digital-engineering-for-defense-summit#ref-6) identifies four distinct gap types: - **Culpability gap:** Who should be blamed when something goes wrong? - **Moral gap:** Who acted unethically? - **Public accountability gap:** Who is publicly answerable for the decision? - **Active responsibility gap:** Who actually has the authority to act and prevent recurrence? Each gap emerges in a different way, and they can stack\. A contractor might have a clear answer to "who is publicly answerable" \(the CEO\) but no answer to "who actually had the authority to override the AI's recommendation" \(the active responsibility gap\)\. In that case, the public answer is symbolic\. No one was empowered to stop the harm\. The mechanics are predictable\. The developer says the vendor configured the system\. The vendor says the deployer integrated it\. The deployer says the operator approved the output\. The operator says the AI told them to do it\. And underneath all of that, no one had a defined role with the authority to break the chain\. The gap isn't a mystery\. It's the absence of design\. The scale of this problem is significant\. Current 2026 industry data[7](/blog/digital-engineering-for-defense-summit#ref-7) shows that 41% of organizations globally are using agentic AI systems, but only 27% have governance frameworks mature enough to monitor and manage them effectively\. That's a fourteen\-point gap between deployment and oversight\. In defense contexts, where one bad output can carry classified information, ITAR exposure, or operational consequences, that gap is the difference between a compliant program and a False Claims Act case\. This is why "responsibility gap" is the right name\. It's not an AI problem you can patch with a better model\. It's a governance problem you have to design out before deployment — the kind of work covered in our [AI governance strategy](/blog/ai-governance-strategy) playbook\. ## What Defense Contractors Face Now **Under NDAA 2026, defense contractors must implement CMMC for AI — the Cybersecurity Maturity Model Certification framework applied to AI/ML systems\. Failure to comply creates False Claims Act liability for any contractor that certifies compliance without genuinely operationalizing accountability controls\.** This isn't a guideline\. It's binding statute, with material liability for non\-compliance\. Legal analysis from Crowell & Moring[8](/blog/digital-engineering-for-defense-summit#ref-8) confirms that CMMC for AI mandates a cybersecurity framework specifically for AI and ML technologies, and that contractors who certify compliance while deploying non\-compliant systems face False Claims Act suits[9](/blog/digital-engineering-for-defense-summit#ref-9)\. The False Claims Act doesn't ask if you meant to deceive\. It asks if you certified\. The timeline is aggressive\. Implementation guidance published by the National Law Review[10](/blog/digital-engineering-for-defense-summit#ref-10) indicates contractors have 30 days to designate a compliance lead and roughly six months for full implementation[11](/blog/digital-engineering-for-defense-summit#ref-11)\. If you're reading this and you don't yet know who owns CMMC for AI inside your organization, that's the first call to make this week\. On top of CMMC, the Department of Defense requires every contractor to operationalize five Responsible AI Tenets[12](/blog/digital-engineering-for-defense-summit#ref-12)\. Each one maps directly to accountability: ```html-table
TenetWhat It Requires
ResponsibleSystems are designed for their intended purpose; misuse cases are anticipated and constrained
EquitableSystems are tested for and free of unintended algorithmic bias
TraceableEvery input, decision, and output is logged and auditable
ReliableSystems perform consistently across the conditions they will encounter
GovernableHumans retain the authority and means to override or shut down system behavior
``` These tenets are not optional documentation\. Each one has to show up in a contract deliverable, an audit log, a test plan, or a governance procedure\. Saying you "value" them isn't compliance\. Operationalizing them is\. There's a second layer that often gets missed\. DoDI 5000\.97[13](/blog/digital-engineering-for-defense-summit#ref-13) makes digital engineering — digital twins, model\-based systems engineering, AI/ML integration, and digital threads — the default approach for defense system development\. When digital engineering is the operating method, AI/ML is no longer a feature you bolt on\. It is woven into how requirements, design, test, and sustainment happen\. That means accountability has to be designed into the digital engineering process itself, not retrofitted onto the deployment\. For contractors, this combines into a single picture\. CMMC for AI sets the cybersecurity floor\. The five Responsible AI Tenets set the governance behavior\. DoDI 5000\.97 sets the engineering method\. And the False Claims Act sets the consequence for certifying compliance you haven't actually achieved\. The risk is not abstract\. It is procurement\-level, it is auditor\-level, and it is litigation\-level\. The contractors who treat this as a compliance checkbox will spend the next two years bouncing between auditors, lawyers, and program offices\. The ones who treat it as a design requirement will spend the next two years winning work\. ## How to Build Accountability Into Systems **Responsibility gaps are solvable\. They aren't technology problems\. They're governance problems — and governance is something you design\.** The framework is operational, not theoretical: clear role definitions, documentation and traceability, human\-in\-the\-loop authority, pre\-deployment testing, and personnel training on AI limitations\. Each component closes a specific kind of gap\. Together, they make the five Responsible AI Tenets executable\. Here's how each one works\. ### Clear Roles and Decision Authority Every AI\-enabled workflow needs a named owner for every decision the AI touches\. Not "the program\." Not "the team\." A person, with the authority to approve, override, or escalate\. If you can't draw an org chart for who decides what across the AI lifecycle, you have an active responsibility gap waiting to happen\. Map the chain end to end before the system goes live\. Then test the chain by asking: who has authority to stop this if it produces a bad output at 2 a\.m\. on a Saturday? If the answer is unclear, the design isn't done\. ### Documentation and Traceability Every input, every model decision, every human approval, every override, every configuration change — logged, time\-stamped, and reviewable\. This is what the **Traceable** tenet actually means\. It is also what your auditor, your IG, and \(in a worst case\) your DOJ counsel will need\. Documentation isn't paperwork\. It's the artifact that proves accountability existed before the harm did\. ### Human\-in\-the\-Loop That Actually Works **Meaningful human control means humans have both the authority and the practical ability to understand, oversee, and override AI recommendations\.** Springer Nature research[6](/blog/digital-engineering-for-defense-summit#ref-6) makes the distinction explicit: theoretical authority isn't enough\. An operator who has the right to override the AI but doesn't have time, training, or interface access to do it isn't actually exercising control\. They are rubber\-stamping\. Operationalize meaningful control by giving operators the time budget to review, the system access to override, the training to recognize bad outputs, and a culture that rewards interventions instead of punishing them\. ### Pre\-Deployment Testing and Red\-Teaming You find failure modes by looking for them\. Pre\-deployment red\-teaming — adversarial testing against the specific environments your system will operate in — surfaces the failures that don't show up in lab conditions\. ICRC analysis of military AI governance[14](/blog/digital-engineering-for-defense-summit#ref-14) makes this point bluntly: organizations that skip adversarial testing find their failure modes during operations, where the cost is much higher\. The investment in red\-teaming is the investment in not learning the hard way\. ### Training on AI Limitations This is where automation bias becomes the hidden killer\. Brookings Institution analysis[15](/blog/digital-engineering-for-defense-summit#ref-15) defines automation bias as the human tendency to over\-rely on automated systems and recommendations — even when those recommendations are wrong\. Training alone won't eliminate it\. What works is governance design that combines training with mandatory human review gates and an organizational culture that values critical examination over speed\. ICRC research[14](/blog/digital-engineering-for-defense-summit#ref-14) confirms that even highly trained personnel are vulnerable to automation bias when system design rewards deference over judgment\. The connection back to digital engineering is direct\. When AI/ML is woven into the system from inception \(the DoDI 5000\.97 approach\), accountability mechanisms get designed in alongside the technical architecture\. When AI is bolted on later, accountability has to be retrofitted, and retrofitted governance is always weaker than designed\-in governance\. Early design is the cheapest possible way to comply\. The contractors who do this well aren't doing it because they have to\. They're doing it because designed\-in accountability produces more reliable systems, lower liability exposure, and faster compliance with every framework that gets layered on top\. Governance maturity is a competitive advantage, not a tax\. The 27% number cited earlier isn't a benchmark to meet\. It's a floor to clear\. If mapping the right governance structure to your AI\-enabled programs feels like a full\-time job on its own, that's exactly the kind of problem a [strategic AI implementation partner](/services/ai-implementation) can compress into weeks rather than quarters\. The principle is the same one that drives our work with founder\-led professional services firms: the goal isn't to install AI\. The goal is to build the human accountability structure that makes AI safe to install\. ## The Digital Engineering Summit Connection The **Digital Engineering for Defense Summit** \(June 24\-25, 2026\) brings contractors, DoD leadership, and academic stakeholders together around the same set of questions: how do we modernize system development with digital twins, AI/ML, and digital threads, and how do we do it in a way that holds up legally, operationally, and strategically?[1](/blog/digital-engineering-for-defense-summit#ref-1) Accountability is central to that conversation, even when it isn't on the session title slide\. Here's why\. Digital engineering, by design, increases the role AI/ML plays in defense system development\. More automation in modeling\. More AI\-driven simulation\. More machine\-generated artifacts moving across the digital thread\. Every one of those expansions creates a new place where a responsibility gap can open — and a new place where contractors who designed governance in early get to set the standard the rest of the field will follow\. The contractors who lead the digital engineering transition are the ones treating governance as a system requirement on equal footing with performance, reliability, and security\. That's the strategic positioning play\. Early adopters of designed\-in accountability reduce liability risk, improve system reliability, and accelerate compliance with every framework downstream\. Late adopters spend the next decade retrofitting\. The summit is where the leading contractors are figuring out which side of that line they want to be on\. This isn't a question of *whether* to integrate AI/ML into defense system development\. That decision was made\. It's a question of how to do it responsibly enough that the system holds up under audit, in court, and in operations\. Both are true\. All of it matters\. ## The Path Forward Here's the bottom line\. Responsibility for AI system outcomes stays with humans and organizations[2](/blog/digital-engineering-for-defense-summit#ref-2)\. You cannot transfer it to the AI itself\. Courts, regulators, and the False Claims Act all enforce that principle\. CMMC for AI compliance is not optional, and the timeline is now\. Accountability designed into systems from inception is both a regulatory requirement and a competitive advantage — the same logic that drives our [AI decision framework](/blog/ai-decision-framework-founders) for leaders weighing where to invest first\. The mindset shift is simple but consequential: accountability isn't a compliance checkbox\. It's the system property that makes the rest of your digital engineering work hold up\. Organizations that act now win the digital engineering for defense transformation\. The ones that delay will spend the next two years answering audit questions instead of winning new program awards\. The practical next step: assess your governance maturity against the five DoD Responsible AI Tenets[12](/blog/digital-engineering-for-defense-summit#ref-12)\. If you're sitting below the 27% line on real governance maturity, you're carrying outsized liability risk\. If you're above it, you're already positioning ahead of contractors still treating accountability as an afterthought\. Either way, the work starts the same: designate the lead, map the chain, log the decisions, train the operators, test the failures, and keep a human being at the end of every algorithmic chain\. That's the defense the courts will accept\. That's the standard the DoD is enforcing\. And that's the work the next decade of [defense AI implementation](https://dancumberlandlabs.com) will be measured against\. ## FAQ ### If my company didn't build the AI system, are we still liable for what it does? Yes\. RAND research cited in the article establishes that responsibility is distributed across the entire supply chain — developers, deployers, operators, and supervisors — and no party can shed liability by pointing at another\. If you deploy someone else's AI inside your system, you are accountable for what it does\. ### What is the actual deadline for CMMC for AI compliance under NDAA 2026? Implementation guidance from the National Law Review indicates contractors have 30 days to designate a compliance lead and approximately six months for full implementation\. Contractors who certify compliance without genuinely operationalizing accountability controls face False Claims Act liability — and the False Claims Act does not require intent to deceive, only that you certified\. ### What is a "responsibility gap" and why does it matter for defense contractors? A responsibility gap occurs when the distributed structure of an AI supply chain allows every stakeholder to disclaim liability, leaving no one clearly accountable\. Peer\-reviewed research identifies four distinct gap types: culpability, moral, public accountability, and active responsibility — and they can stack, meaning a contractor may have a clear public answer for who is accountable while no one actually has authority to stop a harmful output\. ### What does "meaningful human control" actually require in practice? Meaningful human control means operators have both the authority and the practical ability to understand, oversee, and override AI recommendations — not just the theoretical right to do so\. Springer Nature research cited in the article draws an explicit distinction: an operator who lacks the time, training, or interface access to override the AI is rubber\-stamping, not exercising control\. ## References 1. ASD Events, "Digital Engineering for Defense Summit 2026" \(2026\) — [https://www\.asdevents\.com/event\.asp?id=26062](https://www.asdevents.com/event.asp?id=26062) 2. National Telecommunications and Information Administration, "AI Accountability Policy Report: Using Accountability Inputs" \(2023\) — [https://www\.ntia\.gov/issues/artificial\-intelligence/ai\-accountability\-policy\-report/using\-accountability\-inputs/liability\-rules\-and\-standards](https://www.ntia.gov/issues/artificial-intelligence/ai-accountability-policy-report/using-accountability-inputs/liability-rules-and-standards) 3. Taylor Wessing LLP, "AI Liability — Who is Accountable When Artificial Intelligence Malfunctions?" \(2025\) — [https://www\.taylorwessing\.com/en/insights\-and\-events/insights/2025/01/ai\-liability\-who\-is\-accountable\-when\-artificial\-intelligence\-malfunctions](https://www.taylorwessing.com/en/insights-and-events/insights/2025/01/ai-liability-who-is-accountable-when-artificial-intelligence-malfunctions) 4. U\.S\. Department of State, "Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy" \(2023\) — [https://www\.state\.gov/political\-declaration\-on\-responsible\-military\-use\-of\-artificial\-intelligence\-and\-autonomy\-2/](https://www.state.gov/political-declaration-on-responsible-military-use-of-artificial-intelligence-and-autonomy-2/) 5. RAND Corporation, "Liability for Harms from AI Systems" \(2023\) — [https://www\.rand\.org/pubs/research\_reports/RRA3243\-4\.html](https://www.rand.org/pubs/research_reports/RRA3243-4.html) 6. Morley et al\., "Four Responsibility Gaps with Artificial Intelligence: Why they Matter and How to Address them," Springer Nature, *Philosophy & Technology* \(2021\) — [https://link\.springer\.com/article/10\.1007/s13347\-021\-00450\-x](https://link.springer.com/article/10.1007/s13347-021-00450-x) 7. Expert LinkedIn, "The Agentic AI Accountability Gap: When Your AI Assistant Becomes Your Liability" \(2026\) — [https://expertlinked\.in/posts/2026\-02\-07\-agentic\-ai\-accountability\-gap/](https://expertlinked.in/posts/2026-02-07-agentic-ai-accountability-gap/) 8. Crowell & Moring LLP, "CMMC for AI? Defense Policy Law Imposes AI Security Framework and Requirements on Contractors" \(2026\) — [https://www\.crowell\.com/en/insights/client\-alerts/cmmc\-for\-ai\-defense\-policy\-law\-imposes\-ai\-security\-framework\-and\-requirements\-on\-contractors](https://www.crowell.com/en/insights/client-alerts/cmmc-for-ai-defense-policy-law-imposes-ai-security-framework-and-requirements-on-contractors) 9. Crowell & Moring LLP, "CMMC for AI? Defense Policy Law Imposes AI Security Framework and Requirements on Contractors" \(2026\) — [https://www\.crowell\.com/en/insights/client\-alerts/cmmc\-for\-ai\-defense\-policy\-law\-imposes\-ai\-security\-framework\-and\-requirements\-on\-contractors](https://www.crowell.com/en/insights/client-alerts/cmmc-for-ai-defense-policy-law-imposes-ai-security-framework-and-requirements-on-contractors) 10. National Law Review, "DoD AI Compliance: Key Requirements and Strategic Implementation" \(2026\) — [https://natlawreview\.com/article/dod\-ai\-compliance\-guidance\-government\-contractors](https://natlawreview.com/article/dod-ai-compliance-guidance-government-contractors) 11. National Law Review, "DoD AI Compliance: Key Requirements and Strategic Implementation" \(2026\) — [https://natlawreview\.com/article/dod\-ai\-compliance\-guidance\-government\-contractors](https://natlawreview.com/article/dod-ai-compliance-guidance-government-contractors) 12. U\.S\. Department of Defense, "Implementing Responsible Artificial Intelligence in the Department of Defense" \(2021\) — [https://media\.defense\.gov/2021/May/27/2002730593/\-1/\-1/0/IMPLEMENTING\-RESPONSIBLE\-ARTIFICIAL\-INTELLIGENCE\-IN\-THE\-DEPARTMENT\-OF\-DEFENSE\.PDF](https://media.defense.gov/2021/May/27/2002730593/-1/-1/0/IMPLEMENTING-RESPONSIBLE-ARTIFICIAL-INTELLIGENCE-IN-THE-DEPARTMENT-OF-DEFENSE.PDF) 13. U\.S\. Department of Defense, "DoDI 5000\.97 Digital Engineering" \(2023\) — [https://www\.esd\.whs\.mil/Portals/54/Documents/DD/issuances/dodi/500097p\.PDF](https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/500097p.PDF) 14. International Committee of the Red Cross, "The \(im\)possibility of Responsible Military AI Governance" \(2024\) — [https://blogs\.icrc\.org/law\-and\-policy/2024/12/12/the\-im\-possibility\-of\-responsible\-military\-ai\-governance/](https://blogs.icrc.org/law-and-policy/2024/12/12/the-im-possibility-of-responsible-military-ai-governance/) 15. Brookings Institution, "It's Time to Start Thinking About Governance of Autonomous Weapons" \(2019\) — [https://www\.brookings\.edu/blog/techtank/2019/05/10/its\-time\-to\-start\-thinking\-about\-governance\-of\-autonomous\-weapons/](https://www.brookings.edu/blog/techtank/2019/05/10/its-time-to-start-thinking-about-governance-of-autonomous-weapons/) --- Source: https://dancumberlandlabs.com/blog/digital-engineering-for-defense-summit/