This piece is necessary to level set, but I want to be clear that this is by no means a comprehensive review. The intent here is to surface the key issues as a means of setting the stage for future pieces, while also pointing curious readers to sources that might satisfy their curiosity, should they so choose. If you’re an old head in this space, relax. If you’re new to this, welcome to “a riddle, wrapped in a mystery, inside an enigma.”
These contortions we’re about to go through are also only relevant to nations and actors that truly care about their obligations under international law. Certainly, many potential adversaries are not so bothered by these ethical challenges. But we, the West, are, as we should be.
Finally, this piece borrows heavily from a Primer I previously wrote, as I think it does a good job of establishing some of the core principles inherent in this debate and will set the stage for follow-on pieces.
BLUF:
Adding autonomous decision-making to kinetic weapons systems without appropriate legal and doctrinal safeguards threatens to create the so-called “accountability gap”. Nations are legally and morally obligated to close this gap before deploying lethal autonomous weapons systems (LAWS). “Meaningful human control (MHC)”, or variations thereof, have been proposed as a solution, but they are no panacea, and significant work remains if we are to deploy LAWS in accordance with Western values, law, and policy.
I. CONTEXT
The macro context:
The great promise of robotics is to physically remove humans from the dull, dirty, and dangerous jobs (the “3Ds”). And while removing humans can lead to safer and more efficient outcomes, this displacement also removes the less tangible human characteristics that form the foundation of almost all interactions and endeavours: trust and accountability. Trust is a thing that can only be held by humans (more on this another time). It underpins any task requiring more than one person. Its violation, it could be argued, is what has given rise to our legal system and other means of accountability.
Robots cannot hold trust. We may come to trust them through anthropomorphization and automation bias, but that, as we shall see, brings its own risks. We also cannot hold robots accountable, creating the accountability gap that is the crux of this entire debate.
The weapons context:
The allure of LAWS is conflict is completely rational and understandable. Systems which “based on conclusions derived from gathered information and preprogrammed constraints, [are] capable of independently selecting and engaging targets” (see Crootof, The Killer Robots are Here: Legal and Policy Implications) are arguably better able to accomplish the 3Ds without risking human life. However, these systems also fundamentally challenge some of the foundations of LOAC since humans are now delegating decisions that were previously made by combatants in a given battlespace.
Moreover, every nation is obligated under the Law of Armed Conflict (LOAC) to balance military necessity (operational goals) with humanity (protection of non-combatants, etc.). A failure to implement necessary accountability frameworks is a failure to respect basic tenets of LOAC. We must never sacrifice our moral ambition at the altar of operational effectiveness, and particularly not because we fear our adversaries might do the same. To do so would be to race to the bottom.
II. THE ACCOUNTABILITY GAP
Accountability within LOAC functions best when there is a “body to be kicked, and a soul to be damned”. Both nations and the international community have created mechanisms to deal with users of force who breach LOAC, such as military justice systems and the International Criminal Court. For example, attacks that fail to discriminate between protected persons/things and valid military objectives, or those that are disproportionate, in which the “collateral damage” outweighs the anticipated military advantage to be gained, are uses of force for which an individual is typically liable domestically, or in the absence of enforcement, internationally.
But what happens when the decision maker is an autonomous system? Enter the “accountability gap”.
Frank Kendall, the Secretary of the Air Force under Biden, was interviewed in 2024 and raised many of the salient questions.
“It’s who do you hold accountable ... I think we’ve got to think through. Is it the person who used the weapon? Is it the designer? Is it the tester? Is it somebody in the chain of command? I think there needs to be a discussion about the mechanism by which people are held responsible for whatever weapons do when they do something that’s not allowed.”
The core challenge is that our entire military system – whether doctrine, operations, administration, or military justice – presupposes a human exercising agency at the use of force juncture, based on trust from comrades and the chain of command, enforced through accountability mechanisms. Kendall’s list of possible places to locate accountability mirrors the mental gymnastics we’ve all been going through for more than 10 years, and the tricky part is that none of the suggested alternatives truly work. And that is because none of these alternatives would possess the requisite mental intent – mens rea – that is necessary for accountability within our current military systems.
This is a prime example of words mattering. We must take a moment to understand what we mean by “autonomous” in LAWS. We’re not discussing “automated”, or even “highly automated”. We’re discussing a true stand-in for a human, which means that, just like humans, we never truly know what the decision maker is going to decide. But thanks to the professionalism of our modern militaries, we have a pretty good idea of how human soldiers are most likely to act, and what to do when they are disobedient. We’ve become quite good at training, and constraining, humans when it comes to using force in the battlespace. There is ample operational training, as well as instruction in LOAC, which itself positively obligates commanders to ensure that their subordinates are aware of their obligations under the Geneva Conventions.
Why the gap is so hard to close
Then there is the law. Humans are, generally speaking, both individually and generally deterred from breaching the law due to a fear of punishment. But when they do, in a criminal context, which is how breaches of LOAC are treated, we must find both the act –actus reus – and the mens rea. Kendall’s list, though not exhaustive, suggests individuals who, for all practical intents and purposes, are not sufficiently connected to the act and the requisite intent to be found liable for a breach of LOAC.
To hold people so disconnected from the intent and event in question would go against the very foundation of criminal law. Moreover, it would likely also fail to deter future breaches, since the humans involved almost certainly did not intend for the systems they deployed to breach LOAC. Just about any result is undesirable. First, systems may end up being used with veritable impunity, where there is no answering for breaches. Second, if we still rightly feel to need to locate accountability, it may be that the last humans in the chain deploying the systems are the ones taking the blame, becoming little more than liability sponges. Finally, it could also create a chilling effect wherein end-users choose not to use systems at all for fear or becoming the above-noted liability sponges.
So then how do we close the gap?
III. MEANINGFUL HUMAN CONTROL
The best “solution” we’ve collectively arrived at is “meaningful human control”. This seemingly simple phrase reflects, at its core, the belief that humans should remain in control of use of force decisions within the battlespace. And the idea is difficult to argue against! Unfortunately, the last 10+ years of international debate have shown that the exact content of this concept is difficult to clearly articulate, let alone operationalize.
MHC, as a core concept, has been restated by various nations. Canada’s extant defence policy, Strong, Secure, Engaged, commits the Canadian Armed Forces to maintaining appropriate human involvement [emphasis added] in the use of military capabilities that can exert lethal force.” The United States’ DoD Directive 3000.09 similarly holds that “[a]utonomous and semi-autonomous weapon systems will be designed to allow commanders and operators to exercise appropriate levels of human judgment [emphasis added] over the use of force.”
One the one hand, it’s heartening to see nations adopting the spirit of MHC, but it is not yet particularly clear how these restatements substantively differ from your run of the mill MHC. As former USAF Secretary Kendall noted, “that leaves a lot of gray space in terms of how certain are you, what’s the degree of certainty you have that that’s a threat before you commit a weapon, and what degree of competency you want to have that you’re not going to impose collateral damage and kill civilians unnecessarily.”
And these concerns must also be balanced against the ever-increasing speed of OODA loops and kill chains as more decision-making is delegated to autonomous systems. “Kendall himself has worried aloud for years that a human operator might make decisions too slowly to survive against a fully computer-controlled threat.
‘If the human being is in the loop, you will lose. You can have human supervision, you can watch over what the AI is doing. If you try to intervene, you’re going to lose.’”
These are valid operational concerns, signalling that the United States is trying to strike the balance of control vs. speed. How the former is maintained while the latter is controlled is an open question.
Canada, on the other hand, has taken a more conservative approach, stating in its 2024 DND/CAF AI Strategy that “applications involving lethal force must always retain the human in the loop.” Stated otherwise, Canada is telling the world that, at least for now, meaningful control requires active human participation, and it is willing to sacrifice a degree of speed to ensure its control of LAWS allows for the necessary accountability.
IV. RESISTING THE RACE TO THE BOTTOM
The balance that must necessarily be struck then is between that of maximum speed / operational effectiveness and a degree of control that could rightly allow for the human involved to be held accountable, thereby allowing for the legally compliant use of LAWS. Many, including Kendall, have voiced concerns that our adversaries will not feel similarly bound by international law. “They will field systems which are clearly about their operational effectiveness, without regard to collateral damage or inappropriate engagements. And the more stressing the operational situation is, the more inclined they’ll be to relax their constraints.”
While it is true that our adversaries may very well feel no compunction about breaching LOAC, this should never be grounds for us to do the same. This same argument applies to all extant aspect of LOAC. It is the same reason that, despite ostensible operational advantages, western forces do not engage in perfidy, execute prisoners, or actively target civilians and protected structures. More than ever, the global west is fighting to preserve the rules-based international order, of which LOAC forms an important part. Fielding LAWs that are non-compliant simply because we fear our adversaries will do so would bring into question the very things for which we fight.
In seeking to strike the appropriate balance for MHC, lawyers, operators, and NGOs, and policy makers must all also remember that targeting operations are process-driven, not outcome-driven. Collateral damage estimates are just that – estimates, based on the best information available to a given decision-maker, whether human or non, at the time of the attack. Judging the outcome of attacks solely on collateral damage does not reflect the law and is unhelpful to the wider debate. This means that as we design, test, and deploy autonomous systems, we must ensure that the proper processes are in place throughout – processes that allow for human control to be truly meaningful, even in a battlespace compressed in time and space. This will result in systems that are legally compliant, reflective of our moral ambitions, and will therefore provide the necessary accountability.
V. TOUGH QUESTIONS NO SINGLE PARTY CAN ANSWER
Finally, operationalizing MHC within lethal autonomous decision-making will demand a truly multidisciplinary approach spanning the public/private divide during development and perhaps even more so during testing, requiring difficult ethical decisions to be made along the way. For example, some fundamental questions remain unanswered:
How far up the causal chain can we locate individual accountability?
Do we even have weapons review processes able to validate LAWS?
Must a LAWS be at least as good as an average human? If better, by how much?
No one single party can answer these questions. The answers will require input from many stakeholders across government, academia, and civil society, all of whom are rightfully entitled to have their views heard and considered.




Really enjoyed the primer, super helpful to a mind-bogglingly complex topic. Wondering your thoughts on how IHL can adapt to address the necessary demarcation challenges between combatants and non-combatants, and help ensure civilians remain protected: thinking of civilians reporting Russian advances in Ukraine, for example.
Also, interested to hear more thoughts around 'on the loop.' You've pointed out the difficulty of answering these tough questions, but wondering whether MHC matters less when it comes to ISR (specifically, target acquisition)? Morally, I'd say MHC is still essential when identifying targets, but not as essential as the actual deployment of lethality? Sort of a chicken before the egg issue: is it less of a contravention if an operator is simply on the loop earlier up the kill chain? At what level in the kill chain does MHC become absolutely necessary? Just food for thought!
Looking forward to the next article.