AI weapons systems that are designed to learn and adapt could become increasingly difficult for humans to predict, explain and control. This creates an accountability gap that existing oversight mechanisms may not be equipped to close.
The concern begins with how these systems learn. Machine learning allows AI systems to identify patterns in large complex datasets and use them to make predictions or classifications. This learning can happen in two ways. Most AI systems are static – they learn from fixed training data. Continual learning systems, by contrast, can keep updating using new data after deployment. This ability to adapt to changing operational conditions can make their behaviour harder to anticipate.
AI systems’ ability to process vast amounts of information at machine speed is already changing military operations. During Operation Epic Fury against Iran in early 2026, US Central Command claimed that American forces had struck more than 1,000 targets in the first 24 hours. AI-assisted systems helped achieve this operational tempo.
As AI processes information and classifies threats at a scale humans cannot match, more targeting functions can be delegated to machines. When AI systems are designed to adapt or update from new operational data, their behaviour can change as they encounter conditions in real-world combat that were not present during initial training. This adds another layer of unpredictability to weapons already raising concerns about explainability and bias.
This unpredictability creates a problem around how these weapons are tested and certified. Developers can expose a static system to known scenarios, measure its performance and identify its weaknesses before deployment. Commanders can then assess whether it is reliable enough for operational use. But for an adaptive system, certification may lose its certainty once the weapon encounters new and adversarial conditions that were not present during testing.
The battlefield makes this problem particularly difficult because operational data can be deliberately manipulated. An adversary can use camouflage, deception, electronic warfare and other techniques to distort what an AI system observes and create weaknesses in how it adapts.
Beyond the technical aspects, the risks lie in who answers for the technology’s actions. Developments in Ukraine show why this question is becoming urgent. On 10 June 2026, a report surfaced claiming that Ukrainian forces had used 10 AI-controlled ‘Terminator’ drones to identify and attack targets without a human in the loop.
According to Ukrainian drone developer Alexander Kokhanovskyy, the drones were cut off from their communications link. They then relied on onboard AI to identify and engage targets, reportedly killing Russian soldiers. The incident highlights how machines are increasingly being given greater authority over parts of the kill chain.
A more recent incident involving a Russian drone provides another indication of this shift. In July 2026, an AI drone that killed three civilians in Zaporizhzhia, Ukraine, was found with an onboard Nvidia chip and cameras but apparently no antenna for communication with a human operator. These facts led Ukrainian air defence commanders to conclude that the drone was guided fully autonomously.
These are not incidents of military AI already adapting to battlefield data. Their significance is that they show humans are already relinquishing parts of the targeting process to machines. The next challenge is what happens when those machines are also capable of adapting to the environments where they operate.
Consider an adaptive weapon that misidentifies a civilian vehicle as a military target after adapting to new operational data on the battlefield. Responsibility could become difficult to locate among the commander who authorised its use, the operator who activated it, the developer who designed it, the engineers who trained its model and the authority that certified it.
The machine may have made the immediate decision, but it cannot bear legal or moral responsibility for the consequences. This is the accountability gap.
It also exposes a weakness in conventional certification. Certification establishes that a system meets certain requirements under specified conditions at a particular point in time. For an adaptive system, however, that approval may become less meaningful as the system continues to learn in the operational phase.
Closing this gap requires moving beyond the idea that a human in the loop is sufficient. Human control must extend across the weapon’s lifecycle, from development to deployment, with clear limits on where and when adaptive systems can operate, continuous monitoring, and reliable mechanisms to intervene, deactivate or reassess them when their behaviour changes. Clear accountability is essential to prevent unintended harm and dehumanisation in combat.
Military AI does not need to become conscious to outpace human control. It only needs to become sufficiently autonomous, adaptive and opaque such that humans cannot confidently predict how it will behave when conditions change.
Leave a comment