“We killed a man?”

“Well, technically you killed a man.”

“I killed a man?”

“Not even a man. You killed a customer.”

“How the hell did I kill a customer?”

“Shh, don’t shout it for everyone to hear.”

Tim looked around the office. The sea of cubicles stretched from one beige wall to another. Here they were, some of the smartest developers in the world, building the most advanced automated vehicles on the roads, and they didn’t even get free coffee. No-one cared enough to put the milk back in the fridge, let alone listen in to a private conversation. He sat back down and faced Dan across the aisle.

“How did I kill him?” Tim asked.

“Remember that ethical policy framework update that came through last year? For the new models,” Dan said.

“The stuff from Legal about making sure we didn’t hit any pedestrians?” Tim scratched at his stubble. “Vaguely. We just tweaked the decision tree and shipped it.”

“Exactly.”

“I still don’t see how I killed a man.”

“Well it seems someone in Legal was over-zealous with the spec requirements.”

“Dan…”

“Remember you wrote the module for collision avoidance.”

“Yeah…”

“And there was the logic for deciding what to do if avoidance was impossible.”

“Like when the car has to decide between hitting a pedestrian in the road or swerving off of a bridge. Oh…”

Tim felt sick.

“Yeah,” Dan said.

“How?”

“Kid and her mother ran out in front of a 4th gen and the car put the risk of injuring the driver over the two pedestrians or the oncoming traffic. Swung into a fitch barrier and put him through the windscreen.”

“What are Legal doing?”

“Well, they’re saying it was driver fault. He wasn’t wearing a seatbelt at the time of the incident. And overall it’s kind of a win, because there was only one casualty instead of three.”

“Comforting.”

“But we’ve got a bigger problem. Marketing need to see all the code you wrote.”

“Marketing?” Tim said.

“It seems they’re worried no one’s going to want to buy a car that might drive them into a crash barrier at 100 miles per hour.”

“But it might.”

“But then they can’t sell any cars. No one will buy one.”

“So what do they want us to do? Tell the car to prioritise the life of the driver over any pedestrians?”

“Maybe.”

Tim shook his head.

“How many?”

“How many what?” Dan asked.

“How many pedestrians is the life of the driver worth? One? Two? A family? A crowd?”

“I don’t know. Maybe we can code around this, find a better option.”

“This code exists because it’s the last option the car has. Not enough time to stop, nowhere else to turn. It’s them or it.”

“Maybe Legal have an answer.”

Tim turned back to his desk, awaiting the myriad meeting requests to come.

“Unless they can change the laws of physics, people are going to have to accept the fact: their car might kill them for the greater good.”