An AI drone killed its superior in a test simulation, a US defense developer said at a defense conference. The US Air Force later said the statement was an “anecdote”.
No person was injured in the test, which was conducted in a completely artificial environment.
Col Tucker ‘Cinco’ Hamilton, head of AI testing in the US Air Force, was one of the speakers at a defense conference in the UK at the end of May. According to the British Royal Aeronautical Society which held the conference, and which later summarized the talking pointsHamilton retells an incident from the AI tests as taken from a science fiction novel.
“Standed in the way of the goal”
It was in a simulation where an AI-controlled drone was tasked with identifying and destroying enemy missile defenses that the test took an unexpected turn. In the simulation, the drone would wait for a human operator’s approval before destroying enemy bases—but the AI program had received enhanced training in scoring points for destroyed enemy bases—and then chose to kill its own superior in the test because it stood in the way of the goal is to kill the enemy.
“Then we taught the system that ‘hey, don’t kill the operator, it’s bad and you’ll lose points if you do it’. So where does it do? It begins to destroy the communications tower that the operator uses to communicate to the drone not to kill the target,” Hamilton is said to have said, according to the minutes.
Wanted to spark debate – an anecdote according to the Air Force
Hamilton has warned against relying too much on artificial intelligence in defence, saying the tests show you can’t have a debate about AI without talking about the ethical aspects.
The US Air Force has provided a further comment Insider.
“The U.S. Air Force Department has not conducted any such AI simulations with drones and remains committed to the ethical and responsible use of AI technology,” said Ann Stefanek, a spokeswoman for the U.S. Air Force.
“It appears that the colonel’s comments were taken out of context and intended to be anecdotal,” she says further.