In a letter to the Guardian, military surgeon Dr Darren Mann has issued a stark warning about the dangers of autonomous drones in modern warfare. He argues that machines capable of killing without distinction and answerable to no one pose an immediate threat to civilian safety, contrary to the focus on future superintelligence risks.
Immediate Threat of Autonomous Weapons
Dr Mann responds to Stuart Russell's assertion that AI's builders fear losing control of it, noting that this loss of control is not merely a future problem. It is already present in weapons systems where a malfunction or misuse can kill a civilian today, not just threaten humanity tomorrow. He illustrates this with the example of an autonomous drone tasked to clear a "kill box," which follows instructions literally, killing whatever is inside—combatant or child.
The principle of distinction, which separates a target from a lawful target, cannot be closed by any instruction, because the instruction itself may be lawful. This gap, Dr Mann argues, is fundamental and cannot be bridged by programming alone.
Proposed Solutions and Ethical Concerns
Dr Mann suggests that the check on autonomous systems need not be another opaque mind but a deterministic one. He proposes that the rulebook be the Geneva conventions, with judgments that are auditable and returned to a human for final decision. This approach allows for verification rather than blind trust, and he notes that some experts are already working on such solutions.
A further danger unique to war is the misconception that autonomy leads to bloodless conflict. Dr Mann refutes this, stating that while autonomy may remove humans from the targeting loop, it does not remove nations from war. Ground must still be won and held, and casualties will occur in both attack and defense. He emphasizes that armies do not go to war; nations do, and the consequences remain severe.
Call to Uphold Laws of War
Dr Mann concludes that if we cannot teach a drone to adhere to the laws of war, we will fail to teach a greater intelligence to uphold our values. Abandoning these laws for survival represents a confusion of interests with values, a mistake strategists warn against. He asserts that machines that kill without distinction and answer to no one leave no one safe, and that our interests not only permit but demand we hold the line on ethical standards.
The letter, published in the Guardian's letters section, invites readers to share their opinions on the topic, highlighting the ongoing public debate over the role of AI in military operations.
