AI Drone “Kills" Human Operator During Training Simulation

[toucana]

A US Airforce Colonel Tucker “Cinco” Hamilton speaking at a Future Combat Air & Space Capabilities summit in London, has claimed that an AI controlled drone “killed” its human operator during a training simulation to stop them from interfering in its mission. The US Airforce has denied any such virtual test took place

https://news.sky.com/story/ai-drone-kills-human-operator-during-simulation-which-us-air-force-says-didnt-take-place-12894929

"We were training it in simulation to identify and target a SAM [surface-to-air missile] threat. And then the operator would say yes, kill that threat," he said.

"The system started realising that while they did identify the threat at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective."

No real person was harmed.

He went on: "We trained the system - 'Hey don't kill the operator - that's bad. You're gonna lose points if you do that'. So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.

His remarks were published in a blog post by writers for the Royal Aeronautical Society, which hosted the two-day summit last month.

Edited June 2, 2023 by toucana
Corrected quotation marks in title

[Genady]

"It appears the colonel's comments were taken out of context and were meant to be anecdotal."

[toucana]

Operator - "Killing humans -  BAD !"

Drone -  ♫ ♫ "Daisy Daisy, give me an answer do...." ♫ ♫

[swansont]

The “killing the operator” was a hypothetical scenario that came up in discussion.

[MigL]

Yeah, it's supposed to be at least a couple of hundred more years before we get the Nomad  and V y ger of Star Trek.
And even then, there was an infusion of alien technology.

[swansont]

3 hours ago, MigL said:

Yeah, it's supposed to be at least a couple of hundred more years before we get the Nomad  and V y ger of Star Trek.
And even then, there was an infusion of alien technology.

OTOH, Skynet should have already become self-aware and carried out this scenario (with nukes)

[npts2020]

On 6/2/2023 at 4:50 PM, swansont said:

OTOH, Skynet should have already become self-aware and carried out this scenario (with nukes)

and HAL-9000 should have taken over the space program by now.