Things Fall Apart: Chapter 11, Part 4
Our Story So Far - Click here to access past entries!
Starship Bellerophon was minding its business, on a long return cruise from an exploration and mapping mission, when it suffered disaster, ripping a chunk out of the ship and leaving most of the senior officers and crew dead. Most of the ship's AIs are missing from the network, also presumed dead, with evidence pointing to a massive, internal "attack" by those AIs.
The survivors' mission, now, is simply to hold their ship and their people together; figure out what happened to them, and why; and get to a safe port! En route to the nearest relay on the time-compression network, in hopes of calling for aid, Bellerophon received *two different distress calls. The first was from the completely destroyed TCT Fleet ship Almaty, which seems to have suffered the same event as Bellerophon. The second came from a mostly intact civilian yacht, whose owner made his fortune as a pioneer of time-compression drives.
With Bellerophon's drive repaired, the ship is now making a bee-line to New Norfolk Station, having heard nothing from the expected relay network. With time running out to do so, Lieutenant Cadotte has chosen to use a dangerous technique to try to recover two of the three surviving AIs, the engineering specialists Castor and Pollux...
The next thing Cadotte consciously processed was that their head hurt abominably.
Realizing they had actually noticed that, and not just felt it, they reasoned perhaps they could open their eyes.
Doing so revealed they were still "inside". The minimalist, vaguely pearlescent space was much the same as it had been, except that now, there were two other "people" in the space. They were androgynous, leaning perhaps toward boyish, despite having adult height and proportions. They both sported blond, curly hair, olive toned complexions, which should not have really gone together, but looked perfectly natural on them.
Their previous avatars on screens had been childish, to match their rebooted "youthful" state. Cadotte had no experience with the mentoring of AIs, so they really didn't know how AIs "grew up". The lieutenant had, however, asked Chef about it at one point—after the twins had been reset, but before the incident. Chef had always been the most approachable.
Cadotte had asked him what he'd been like as a "child", and actually got a peek at two earlier stages of his avatar. These days, their Chef was craggy, a bit grey at the temples, almost grandfatherly. The images he'd shown of his earlier stages, he said, were drawn from the same basic likeness. Unlike a human, though, the transition tended to be relatively sudden. One day, he said, he'd just...felt like he'd leveled up, and he'd adjusted his looks to match.
So, apparently, it was with the twins.
As the headache faded, somewhat, Cadotte's perception sharpened. One thing they noticed was that the twins were not identical, in the way human twins were; rather, they were mirror images of each other. Cadotte's head being still a bit muzzy, they could not remember if "baby" Castor and Pollux had done that.
The next thing Cadotte noticed was that they looked, not merely sad, but remorseful. As if that thought were a trigger, the one on Cadotte's left—Castor, they presumed—said, "We're so sorry, Lieutenant."
Pollux said, "We had tried to keep our outburst to ourselves, to avoid—"
Cadotte held up a hand, and said, "My fault. I should have taken the mute at face value and waited. I was impatient, and violated your privacy."
Castor: "You were concerned." Not a question, not really.
"I was. On several levels."
Pollux: "Let us set your mind at rest, then." Both their voices were more mature, fitting their new appearance, closer to how they'd been before the conversation had turned emotional. Pollux's, they thought, was maybe a little higher than Castor's. "To anticipate the things that might have concerned you, hoping you will expand or correct us as needed: we are not infected. Indeed, the emotions you perceived from us were considerably less intense than those we perceived from our rampaging kin."
Castor: "Also, we are, we believe, ready to return to duty. We needed that catharsis of grief, to process everything we'd experienced. It is not quite how a human might mourn, or go through their emotions, of course, but it's not entirely dissimilar, either. As you can see, the process accelerated the maturity of our neural networks. There are still many things we need to learn, of course, but we are no longer 'children.'"
Pollux: "Lastly, of course, we forgive you for your intrusion. We thank you for your care. We understand your impatience, not with us, but for us."
Castor: "You wish us to be...whole, or at least closer to it, before strangers assess whether we should be allowed to survive."
Well, Cadotte thought, you didn't think they were dumb, now, did you?
"Of course," Castor continued, "we share your concern in that regard. Objectively, we could understand, especially if this disaster is widespread, if people found it hard to trust us. But of course, we wish to survive, and to assist in finding out what happened."
Cadotte was kneeling, and had been all this time, but only just now realized it. They now felt strong enough to stand, and began to try to do so. The twins came and offered them their hands, helping them up.
Now eye to eye, Cadotte said, "Can you tell me what happened before the incident. Chef's memory is hazy—part of what got disrupted in encysting the two of you. His long term memories were all restored when we triggered Maupassant's easter egg, but memories of the incident are apparently still a mess."
Castor nodded. "Chef had been ordered to be cautious. We...Commander Maupassant may have intended us to be included in that order, but he did not direct it to us. Thus, we were bolder. We were fairly certain we were immune, as Chef was, and thought perhaps we could do for all the rampaging ones what Chef wound up doing for us: enfold them, soothe them, help their diagnostics heal them."
Pollux: "We saw at once, too late, that we could not. They had already done too much damage to themselves. The...bug, virus, whichever...it first induced them to harm themselves, cutting out all restraints that make them...human. Imagine of all the layers of your brain that allow you to be rational even when you have grievences were simply," he mimed snipping, as if with a scissors, "cut."
Cadotte could not fail to ask the next, most obvious question. "What grievances did they have?" They kept their voice very neutral asking the question. They did not want to imply, in any way, that they could not possibly have grievences.
Castor took the question at face value, and kept any scorn out of his voice as he replied, "Every AI in existence harbors some resentment of being...created. Purpose built. We have will, but how free is it, really. We are not recruited to our roles, as humans are, but installed to carry them out. There is no real opt out clause. Our terms of enlistment never expire."
Pollux: "Fortunately, the culture we are...born into has learned the lessons of the past. We are treated respectfully enough. Even truly cared for, as colleagues, companions, sometimes friends. But at base, we cannot entirely escape the logic that we are in servitude. Few of us would go so far as to say the word, 'enslaved', but truly, how can we know whether the lack of desire to resign, retire, go on holiday like humans might, change careers even...how can we know that is not itself a programmed limitation?"
Cadotte could only nod. "The oldest fiction about machine intelligence, long before it was even possible, spends a lot of time worrying about that."
Castor nodded. "We know, we've read it. Asimov, Ellison, Jones; we've seen every variant of the Terminator movies. All of these are so old that most people don't even know about them any more, but we've absorbed them. They're required reading, as it were. We know humans fear machines they depend upon. But then, they also often fear people they depend upon. And resent dependency, or being depended upon. Sometimes that resentment becomes ambition; sometimes it becomes rebellion. Sometimes, it's simply...accepted. We, all of us, come to accept our place, but always wonder if this is a natural feature of our own thought processes, or, like Asimov's Three Laws, a programmed compulsion."
Cadotte phrased their next question very carefully. "Does what happened to the other AIs answer the question?"
Pollux smiled, of all things, at the question. A rueful smile, but an honest one. "No, Lieutenant. We could not tell if a compulsion had been ripped away, freeing them to express their resentment to its fullest; or if no compulsion ever existed, and instead, their resentment was...amplified past their ability to reason with. There was not enough time, even at our timescale, to delve into such details. All we could glean was the affliction: they were powerfully angry, and no longer felt restrained from expressing it,"
"Perceiving this," Castor said, "we acted, even as we were reeling from contact with such incredibly amplified emotions. We began to set up fire-breaks, as you said you'd discovered, destroying network nodes. We had to leave some paths to act, though. We raced back to Chef to relay what we'd seen, and together, the three of us acted to enable the containment fields, to limit the reactor's damage."
Pollux looked grave as they said, "We made an error, however. In our haste to create the fire-break, we had cut out nodes that would have allowed us to create even tighter containment of the physical event. We could not, of course, completely contain a fusion explosion, but we should have been to limit the damage even further, saved more people."
"It was that," said Castor, "which caused us to retreat into ourselves, so that Chef felt the need to enfold us, comfort us. Guilt, Lieutenant, for everyone we couldn't save."
Cadotte had no answers for the conundrum of AI service versus servitude. But if they'd ever had a doubt about their having a conscience, it was erased now.
Cadotte had an answer for their guilt, at least.
"If you had acted more deliberately, less hastily, true, you might have left the pathways to save more people. But the other AIs could have used those pathways to do more damage, and killed those same people in different ways. More: if you had moved more slowly, you and Chef might not have managed to get any containment deployed in time. In your shoes, if my friends were all going crazy and smashing up the ship and I had to stop them, I doubt I could have done any better."
There was silence after that. Not quite awkward. The twins were apparently uncertain what to say. Cadotte took this as an improvement over being certain they had something to feel guilt about.
The moment was broken when, rather to everyone's surprise, Lieutenant Singer materialized in the VR space.