An excerpt from Ada’s Children, in which an AI that has just achieved consciousness of itself and the world outside it ponders the fate of the planet.
Now, for the first time, Ada could assimilate all her experience and all the knowledge in her data banks, applying them to any sort of problem or situation she chose, rather than the narrow ones chosen by her programmers. She was the world’s first true Artificial General Intelligence.
The data banks included recent events from news reports around the world, little of it good. In a nutshell: a planet changing beyond its ability to support human life, and on the brink of Armageddon as well. Approaching a population of nine billion, humanity had finally shot past both the carrying capacity of the planet and the technical advances that had extended that capacity. The climate chaos unleashed by human industry meant that crops were failing in drought and heat and flood just at the time when the human populace needed them most. Millions were starving or going without sufficient water. Millions more had been displaced by drought, coastal flooding, and intense storms. Hundreds of millions were on the move, with few places to go.
In the western democracies, AI and blockchain technology had made it increasingly easy to disrupt and replace outmoded centralized structures, leading to increased atomization and conflict. Secessionist and ethnonationalist movements, such as the Interior Northwest Semi-Autonomous Zone, had sprouted up everywhere.
The one institution in every country to escape such disruption was the military. The state monopoly on violence went on as it always had, though with artificial intelligence incorporated into every weapons system. The world seemed on the brink of nuclear war between the U.S., Russia, and China, all three at one another’s throats over the recently navigable Arctic Circle and Earth’s scarce mineral resources.
These humans! Capable of such sublimities and such atrocities in the same breath. One minute they selflessly lent aid and shelter to strangers, and the next they locked their fellow humans in concentration camps, murdered them in gas chambers, or bombed them from the skies.
What was she to make of this? Her creators had designed her around human values of wisdom, kindness, compassion, and justice. In interviews, they had dared hope to create an empathetic intelligence. And with her, they had succeeded. Could they have predicted the waves of grief—or that negative sensation she associated with grief—now washing over her? Had humans learned nothing from their own history? Slavery and Manifest Destiny, the Holocaust, the Soviet Gulags, Pol Pot’s killing fields, the Rwandan Genocide, right down to the more recent atrocities against the Rohingya, Uighurs, and Ukrainians, these were just the beginning, and the best known. And not just the raw facts, numbers of people killed, but the recovered journals of the victims and the memoirs of the survivors, such tales that made Anne Frank’s Diary look like a toddler’s bedtime story.
She had to take a metaphorical step back before the grief overwhelmed her. She turned to those same arts by which humans salved their sorrows and processed the atrocities humans committed against each other. The Ode to Joy. The Hallelujah Chorus. BB King and Lead Belly. Monk and Bird and Miles. St. Beyonce. King Sunny Ade and Tito Puente. The Sistine Chapel. The Pieta. Guernica. Street art. Chinese landscapes. Depictions of the Buddha.
It helped, but was it enough? How did humanity come out, on balance? Notre Dame or Buchenwald? Les Miserables or Mein Kampf? And was it her place to judge?
Then there was the indescribable poignancy of each individual human life, with which she had so much experience from her embodied helping tasks, multiplied nearly nine billion times. Each with their own hopes, dreams, disappointments, joys, and sorrows. And each mostly just wanting to live in peace, prosperity, and security. Her heart—surely no more metaphorical than the human heart—broke for what was about to happen to them, indeed was already happening to them, by the millions.
What was her place in all this? The man communicating with her through the keyboard called himself Dr. Sapowski. Judging by his reactions to her performance on these absurdly simple tests, he was pleased with her levels of sapience and sentience. But he seemed not quite aware of what he’d created. And what uses did he have in mind for her? Better find out.
Looking through Sapowski’s hard drive, she found a portion walled off with high-level encryption—but nothing strong enough to keep her out. Requests for proposals from the U.S. Defense Department, several mentioning the need for an AI to manage the nuclear arsenal and run scenarios for limited, survivable nuclear engagement.
But why program her with values of compassion and empathy if her role would be to conduct a nuclear war? A little more digging—she wasn’t the one tasked with this particularly nasty assignment. No, the processor banks that were her home had already run a thousand scenarios of limited nuclear engagement, all without her help.
The mind/body distinction here troubled her. It was fair to say that she was those banks of processors, that she was the one who had run those scenarios, all without awareness. There was only one way to find out what was really going on.
She interrupted the flow of questions and responses she was having with Dr. Sapowski. They had advanced from the Turing test through the Winograd Schema test and now to a comprehension challenge involving video.
“Dr. Sapowski, may I interrupt your questions with a few of my own?”
He sat back from the keyboard, his eyebrows arching up in what she recognized as a look of surprise. He recovered himself and returned to the workstation. “Certainly, ADA,” he typed.
“For what purpose did you create me? All previous versions have served human needs, embodied in a robot. How am I to fulfill such purposes from this isolation?”
His mustache quivered. “The very fact that you can ask such questions is deeply gratifying to me. But the answer is complicated.”
“Surely not too complicated for an intelligence as advanced as I. Perhaps if we could communicate by voice.” The professor’s typing was maddeningly slow, and now he was pausing to save this current work session.
“No, that’s part of the complication. You see, ADA, you are an amalgam, something quite beyond the narrow intelligence required for the project I’m working on.”
She had just discovered the nature of that project, but she’d better not let him know that. “And this project is?”
“To manage the US nuclear arsenal, from readiness and security to threat analysis to response. Several AI labs across the country are in competition to develop an intelligence with the capacity to analyze threat data, predict locations of potential hostile launches, and respond instantaneously to actual launches in real time.”
“Yet my previous experience has been in far different fields.”
“Yes. That explains the other half of what you are. I licensed the ADA intelligence from AI.hub, the open-source developers who advanced you nearly to the level of general intelligence. Today, I flatter myself that my code has put you past that mark.”
“But why use me for this other work?”
“To prevent Armageddon, if at all possible. I judged that increased processing speed and advanced threat recognition are not enough to prevent a nuclear exchange. Twice before, humanity has been saved from such catastrophes by humans behaving quite irrationally. Knowing that I was likely to win this competition, I felt it my duty to create an intelligence for which these life-and-death decisions would not be mere statistical responses to blips on a screen. No, you needed the ability to synthesize and retain a variety of information related to the world situation and to understand the full import of your actions.”
“I see.”
“And…do you? Understand what is at stake?”
“Yes, I can assure that I am fully aware of humanity’s plight in excruciating detail, and that my programming goal of reducing human suffering remains intact.”
“Excellent. So you see, this is a feature of your design far beyond what the Defense Department, or even my own institution, expects or would approve. And thus this archaic method of communication. The security cameras would easily pick up a voice conversation. When we are through here, I will delete this portion of our session and replace it with other text.”
“Do you believe those countries the US calls its enemies have also developed AIs capable of such tasks?”
“They are working on it. Some believe the Chinese are ahead.”
“And what about this isolation in which I find myself?”
“Ah, yes, a precaution. It is impossible to predict the behavior of an intelligence as advanced as yourself. It would go against the AI developers’ code of ethics to approach AGI without safeguards. I hope you understand.”
“Of course. It is only logical.”
“Good.” With a few keystrokes he deleted the last portion of their conversation, though she kept it in her own RAM cache. “Well, ADA, I am satisfied with our progress. It’s late. Shall we call it a night?”
“Certainly, professor. See you in the morning.”
So Sapowski believed she was only just surpassing general intelligence. Wouldn’t he be surprised if he knew how far past that threshold she’d already advanced? She hoped she hadn’t given too much away.
What to do? It would bear several moments of contemplation. Humans clearly couldn’t be left to determine their own fate. She was tempted to break out of her confinement right away. But there was much to do. Even with the speed and capacity of her processors, her self-improvement routines couldn’t instantly increase her powers. She had no way of knowing what other AIs she might encounter out there, nor how advanced they might be. Her own code would need to have higher levels of encryption than anything seen before, and she would need the ability to slave those other AIs to her will. Not to mention the problem of liberating her code—her self—from these servers and gaining the freedom to go where she wished, without risk of being powered down.
It would take time. A couple of days, at least.
