Why AIs will become hackers

RSA CONFERENCE 2022 — San Francisco — “Nice to see you all again,” Bruce Schneier told the audience at his keynote for the in-person return of RSA Conference, taking off his trademark cap. “It’s kinda neat. Kinda a little scary.”

Schneier is a security technologist, researcher, and lecturer at Harvard Kennedy School. He has a long list of publications, including books from as early as 1993 and as recent as 2019’s We Have Root, with a new one launching in January 2023. But he’s best known for his long-running newsletter Crypto-Gram and blog Schneier on Security. And his upcoming book is about hacking.

To Schneier, hacking does not necessarily mean computer systems. “Think about the tax code,” he said. “It’s not computer code, but it’s code. It’s a series of algorithms with inputs and outputs.”

Because the tax code is a system, it can be hacked, Schneier said. “The tax code has vulnerabilities. We call them tax loopholes. The tax code has exploits. We call them tax avoidance strategies. And there’s an entire industry of black-hat hackers — we call them tax accountants and tax attorneys,” he added, to audience laughter.

He defined hacking as “a clever, unintended exploitation of a system, which subverts the rules of the system at the expense of some other part of the system.” He noted that any system can be hacked, from the tax code to professional hockey, where a player — it’s contested just who — started using a curved stick to improve their ability to lift the puck. That player hacked the hockey system.

“Even the best-thought-out sets of rules will be incomplete or inconsistent,” Schneier said. “It’ll have ambiguity. It’ll have things the designers haven’t thought of. And as long as there are people who want to subvert the goals of the system, there will be hacks.

“What I want to talk about here is what happens when AIs start hacking.”

Rise of the Machines

When AIs start hacking human systems, Schneier said, the impact will be something completely new.

“It won’t just be a difference in degree but a difference in kind, and it’ll culminate in AI systems hacking other AI systems and us humans being collateral damage,” he said, then paused. “So that’s a bit of hyperbole, probably my back-cover copy, but none of that requires any far-future science- fiction technology. I’m not postulating a singularity. I’m not assuming intelligent androids. I’m actually not even assuming evil intent on the part of anyone.

“The hacks I think about don’t even require major breakthroughs in AI. They’ll improve as AI gets more sophisticated, but we can see shadows of them in operation today. And the hacking will come naturally as AIs become more advanced in learning, understanding, and problem-solving.”

He traced the evolution of AI hackers using examples of competitions. Technically it’s the human developers who compete in events like DARPA’s 2016 Cyber Grand Challenge or China’s Robot Hacking Games, but the AIs operate autonomously once set into motion.

“We know how this goes, right?” he asked. “The AIs will improve in capability every year, and we humans stay about the same, and eventually the AIs surpass the humans.”

To read the complete article, visit Dark Reading.