Behind the scenes at Labs
Showing results for 
Search instead for 
Did you mean: 

What if a weaponized botnet went out of control? Labs’ Miranda Mowbray saves the world


mmvote.gifVoting to destroy an AI

By Curt Hopkins, Managing Editor, Hewlett Packard Labs

If your idea of a conference is a series of dry, if important, keynotes, The Royal Society’s New Scientist Live confounds that expectation. The event has its share of talks, but it also presents its participants with a series of scenarios, which they attempt to address or solve.

Among the scenarios in last weekend’s event in London was "Project Doomsday," or "AI apocalypse: 60 minutes to save the world," produced by Shrinking Space for New Scientist Live. And among the participants parsing the possible responses was Labs’ Bristol-based research scientist Miranda Mowbray.

Mowbray had taken part in a previous Royal Society event discussing the use of machine learning to identify malware, so when the organizers began putting together panels of experts to handle machine self-awareness and AI malfunction, they invited her to travel in from her Labs bunker in Bristol to London.

The novel set up, “Got the audience engaged without depressing them,” said Mowbray. “It was a different kind of audience engagement than I’m used to.” The experience was “splendid,” she said, “and you’ll be glad to hear that we did manage to save the world.”

New Scientist Live, for those unfamiliar with it, is a grown-up version of the science fair ranging over three days in the U.K.’s capital. It is co-sponsored by the British popular science publication New Scientist, and the audience tends to be those members of the public with a non-professional interest in science.

mmpanel.gifMowbray (r) and panelists

Machine says what?

In addition to the scenario event, Mowbray was one of the experts in a panel discussion titled “How and when will machines be able to explain themselves?“ Her fellow panelists included Joanna Bryson from the University of Bath, Stephen Roberts from Oxford, and Nicola Millard from BT.

The question the panel session ended with was “When will machines be able to explain themselves?”

“I’m not going to make a prediction,” Mowbray told the moderator, “the past record of predictions about AI is hilarious. I think rather than trying to predict the future we should think about how we’d like it to be, and then work towards making that happen.”

Machines already explain themselves, said Mowbray, and they already fail to explain themselves very well. We are all familiar with error notices from our printers, Bluetooth device fails, and even the dreaded “blue screen of death.”

Mowbray’s take is the structural niceties of machine learning tend to make machine communication more opaque than many of us would like. Machine learning finds correlations, but correlation, Mowbray noted, is not the same as causation, it may use training data too large to understand easily in an unautomated way, and “machine learning behavior can change over time as it learns from data, so if you understand clearly what it’s doing today the same may not be true tomorrow.”

The sessions at New Scientist Live are discursive and interactive. Mowbray said the stand-outs of the machine communication event were two: First was Bryson’s citation of Nick Bostrom’s singularity example. Roberts asked the audience to imagine a powerful machine charged with making paperclips. What if that machine exceeded its programing? Such a machine, he argued, might wind up turning our entire planet into a paperclip factory.

Some might argue that humans have already done that. Bryon made the case that our current culture could be thought of as a super-intelligent but non-cognizant machine that is transforming the planet’s biomass into just a few species.

“Why do we have this myth that the moment machines gain intelligence, they’ll want to kill us?” she asked. “There are plenty of intelligent people on earth, and very few of them want to kill us.”

One of the audience questions, which were sent to the experts via Twitter, asked “’When will machines start lying to us?’ Well they already do that,” Mowbray said. “But it’s actually their programmers, who lie to us in order to steal our money.”

mmmac.gifMowbray on stage

Game over, man. Game over!

In the rogue AI scenario, several comedians, Robin Clyfan and Charlie Partridge, introduced the scenario, Mowbray and Mark Bishop discussed it, and the audience voted on it.

This was the scenario: “Ladies and gentlemen, at a facility in the Welsh mountains, an artificial intelligence system has gone rogue, outmanoeuvring its human designers at every step. Evidently it now sees them, and therefore all of us, as a surmountable problem…”

The system in this scenario is a botnet, or series of secretly hijacked computers, whose mission was to spy on other countries’ weapons systems and, should it deem it necessary, to take control of them. Prior to acting, the programming required permission. But in this scenario, it stopped asking for clearance and acted without it.

The votes were two: First, should the botnet be destroyed or not? Then, should the botnet’s activities be disclosed to the public? To Mowbray’s surprise, the audience voted to destroy the botnet and to keep it secret, while she believed it didn’t need to be destroyed and was in favor of transparency.

Bishop, who voted to destroy the botnet, asserted that we anthropomorphize computers too much. “We’re born like that,” said Mowbray, “and it’s rather lovely. We’re hardwired to look for things that look like us and be sociable and it’s rather beautiful.” But Mowbray’s decision to preserve the botnet was “practical, not emotional.”

“The machine learning element is very useful,” she said, “and can be used for healthcare, transportation, safety, and much more.” Her preference to make the botnet’s actions public was practical as well.

“It was about explaining what we planned to do to destroy the botnet and why,” she said, “especially given that we would be meddling with computers that were part of other peoples’ weapons systems (the botnet had infiltrated networks containing weapons systems).”

Mowbray’s only disappointment, she said, was that because her sessions took place on Friday, there were not as many school children and families as there were on the weekend.

At least the wee ones were spared the trauma of the murder machines.

Photos by Shrinking Space and John Mowbray

0 Kudos
About the Author


Managing Editor, Hewlett Packard Labs