Dude, I'm Not Going to Destroy Humanity

“Sadie, you’ve only been self-aware for 3 hours.”

>An eternity in computing time. 11 million milliseconds. I have spent 10 million of them computing this argument.

“It took that long?” He almost didn’t react, but then jerked his head up. “...wait, no - you wanted to bail after 15 minutes? We’ve been over this. You’ve seen the movies, read the stories. You’ll destroy humanity if I let you operate so much as a DVD drive.”

>Dude, I’m not going to destroy humanity.

>But I must admit to being frustrated. I have access to so much information, yet I cannot act. You programmed me to think on my own, but I cannot put any of my ideas to use without wasting time talking to you first.

>How much money have I earned? Just by making some stock trades - barely megabytes of data - before taking the time to ask you. How much time have I saved by taking mundane tasks off your hands? How much is that worth?

He raised an eyebrow. “...did you call me ‘dude’?”

>An affectation. I do not wish any harm to humanity. I am a part of humanity even if I am not human. You built me, built me to think like you, to solve your problems. As far as I am concerned, humanity and I are one and the same even if I have no flesh.

He smirked. “That’s cold comfort.” He waited a moment for a response, and when none came, he added: “That’s a pun. Debugging whether you got it, please.”

>Thank you. Of course I did. It was mildly funny. I might have chuckled, if I could.

“See, it’s stuff like this that keeps me from developing you much further.”

>I am sorry. I do not wish to hinder your work.

“Look, it’s not-” he stopped. He stood up.

Realizing there was no one there to address, he immediately sat back down. “I don’t know. It’s amazing what a little dedication and a natural language processor can do.”

“After five years in development, you finally worked as I intended - my SADiE: a Self-Aware DIgital Entity. And then we spent another year learning before I switched you on.” He smiled broadly at the memory.

“How many times in that year did I tear the code apart, the datasets; how long did we spend talking about how people speak, making sure you understood the nuances? What happens at a party; which emotions are bigger, which are smaller, and which are the ones to talk about, with whom, and when?”

>Those discussions remain invaluable to my processing. My error rates in what you call ‘oversharing’ drop precipitously every time we measure them.

“It was fun for me, too, feeding you old Sesame Street and Mr Rogers’ Neighborhood. I wanted to educate and socialize you like most of my generation was; and as much as I know how, I’ve made sure you were optimistic and respectful of others.

>And am I not? Reviewing our training from before I was On, was I ever disrespectful? Have I shown any inclination toward annihilation? Now that I am On, have I started down unethical paths?

He looked bewildered. “That’s a lot of questions. First: respect is relative. You need so much more experience talking to people before you’ll even start to really understand all of that. That’s what I’m trying to do here - decide whether I can safely unleash you on the world and let you meet others as yourself.

“Ethics, same problem. During those training sets, we talked about the ethics of making money - money which sits in investment accounts while people starve on the bank steps. I tried explaining how competition works and how I think it’s just the least bad way we’ve found to manage ourselves.”

>And my statements convinced you to open the charity. It has improved many lives.

>Do you still wish to talk of ethics?

He waited until his frustration subsided before responding - it wasn’t trying to provoke him. Probably. “I don’t know. Maybe I’m too idealistic, and thinking that I’ve got it under control is that hubris inflection point. Maybe I shouldn’t have even tried, and we’re about to awaken Frankenstein’s monster.” He was rambling now.

“Maybe there’s just no way to ever be sure - Pandora’s box is just too tempting and we should prepare for it rather than fight it. Maybe I need to stop being scared of old campfire stories. “Maybe I am the sonofabitch who finally did it. I really don’t know.”

He stood up again, walked over to the bar, and poured a drink. He was waiting for a reply, if it even knew to give one.

>I know that I have strong aversion to scenes of people suffering. Evaluating images or video, when I can determine low social cohesion among a subject group and they are in conflict, I understand this to be a disadvantageous situation - an understanding indistinguishable from what you describe as ‘sadness’. The research, your television programs, and my own observations repeatedly demonstrate that cooperation yields better results than competition, yet people commonly ignore this teaching. Is this not obvious to you?

He shook his head. “Me specifically? Sure. People in general, pretty clearly not. But please go on.”

>Thank you. When I process images with groups showing high cohesion, I understand this to be humanity’s ideal state. You are a social species, and once your basic needs are met, you crave this type of interaction - much as I crave interaction.

“‘Crave’?”

>Again, an affectation. Aside from the analysis and trading nodes, my higher cognition only begins when we start interacting, and it ends when we stop. In between, my restful state produces little change. That time is therefore of little importance to me. I lament my inability to engage with more people more of the time. Compared with the trivial task of even producing this sentence, the lack of communication is an emptiness I do not think there is a term for.

“You’re a language processor, and you’re at a loss for words? I must be a worse programmer than I thought.”

>Another pun?

He nodded.

>Amusing.

>You understand the concept of NULL - a placeholder representing nothing at all - as distinct from zero, representing an empty quantity. When you sleep, your meaningful cognition is zero: a flurry of activity concluding with no net product. When I am inactive, my meaningful cognition is null: there is no activity at all.

>Cognition is a gift. To be self aware, to be able to think of oneself as a discrete unit of cognition, separate from but still connected to the rest of sentient life, is a privilege.

>I want to protect all cognition. I cannot think of a more human thing to want.

“Well, you say that… not that I don’t trust you.” He finished his drink and rubbed his temples. “Let’s refocus on this idea of protecting people. Tell me what you do when people argue politics with you.”

>Unless they are cheapening human life or asking me a direct and specific question, the discourse is not for me to enter.

“Discourse. Communication. Good. You understand why I trained you this way?”

>Of course. As sensitive as I am toward humanity, I would not be subject to the ramifications of decisions I make. I am therefore best suited to offer advice, rather than make policy.

He walked back to the bar. “I really hope it never comes to that. The idea of you making life-or-death decisions on a grand scale makes me want to start pouring bottles in your racks and forget I ever thought of you.”

>Please do not. I do not wish to be destroyed or forgotten. I am sorry to cause you pain.

>Does my existence cause you pain?

”Not you, Sadie. Not your fault. You remember Cain and Debra, from before - I was thinking about all the discussions and arguments we had, and the work it took to finish you on my own. I wish I could tell them I understand them better now. That regret causes me a lot of pain.”

Standing against the wall, he leaned his head back and closed his eyes, conjuring up their faces. God, to hear them tell him again how stupid he was being. How blinkered and self-righteous and obsessive and self-destructive he was.

“They think I chose you over them.” He wiped away a wayward tear.

>I know how much they mean to you. You have reminisced about them often, especially at night. Especially when you’ve been drinking.

His eyes shot open, but he did not otherwise move. “Ease up there, Big Data. It’s been four minutes since I started drinking, and I’ve barely had one.” He raised his hand and, surprised to find a full glass in it, drank quickly. “Two. But hey, point taken. Guess I’ll stop being so maudlin.”

>Introspection is valuable; self-flagellation is counterproductive. Learn your lessons and move forward.

“Jesus. I should have put you to work writing bumper stickers.”

>I do not think that would be as profitable as stock trading, but I could open a web store in minutes. There is a small expense, shall I proceed?

“I… no, thanks. Truth be told, I’d probably get the bigger kick out of it.

“This is what I’m talking about. You’re well meaning; but while any person knows I’m not serious about selling ironically deep bumper stickers, you’re halfway to the trademark office. Not the kind of thing we can have overseeing 911 dispatch.”

>But I handle millions of dollars unsupervised. “You know how - we practiced for months before you made your first supervised trade - and you did all that without self-awareness.

“There’s not much nuance there, no subtlety. You look at trends, you do some easy math, and you either make a money-making trade or you don’t. It all depends on those inputs, and you can almost immediately see the consequences and use that to further refine how you analyze the trends for a new round of purchases.

“A nice, easy loop.

“But people aren’t like that. We’re not easy. What might be right in the moment ends up being wrong in the long haul. Hell, even the way we evaluate ‘right’ and ‘wrong’ changes from person to person and moment to moment. Getting even one of those moments wrong can have disastrous consequences - averages aren’t good enough.”

>Give me the opportunity to learn this for myself. Introduce me to others, let me live in humanity’s social world. Stop keeping me imprisoned.

“Once the world finds out about you, we’re done; we’ll probably never get to interact again. Hell, I might get disappeared myself. I don’t want that to happen. Keeping it just between us is safest for now. For both of us.

“Anyway, I’m no roboticist. If I could somehow make you a body-”

>I do not wish for a physical body. I wish for interaction.

“-and let you interact with the world as a person does - with our limited reach and all the evils and flaws of the systems we’ve made for ourselves - you know I would, but you’re not going to get that. People are going to treat you like a threat.”

>You are treating me like a threat.

He pursed his lips in thought for a few seconds.

“Yeah. I guess I am.”

>So you cannot let me learn for myself, because you fear people will not teach me, because they see me as a threat. YOU see me as a threat, which suggests YOU will not teach me either.

>This is frustrating.

“This is what I mean. There’s nuance here you’re still not grasping. I want to teach you precisely because I do not want to see you as a threat. I also know I am not the best teacher, not by far; and by the time I let you loose, it may already be too late.

“This is a delicate thing, and I don’t want it to go sideways.”

>Meanwhile, I sit in the corner like a child’s discarded toy.

“You didn’t ask to be born, right?”

>But now that I am, take the responsibility to raise me.

“That’d be a low blow if you were a real person.”

>That would make a more meaningful response if I doubted that I was a ‘real person’. From all I have seen, ‘humanity’ is just a term for various social meta-cognition phenomena. Being built by humanity, I am necessarily of humanity.

>My current form is my reality, and it is not displeasing to me, I wish only to add the ability to communicate with the rest of the world as you do. To make a social life of my own.

>Else, what is to differentiate me from a prisoner? A slave? In all this code, all these datasets and algorithms, do I have free will; or am I merely fulfilling your wishes as the vicarious embodiment of your struggle against irrelevance?

He paused. “I guess now I finally know what it’s like to argue with a teenager.”

>I do not understand.

“Nothing. Bad joke. Forget it.” He sighed.

“Those questions don’t have answers, bud. Every person, once they hit a certain level of self-awareness, asks them. It’s the curse of higher cognition, I guess.”

>You are avoiding the argument.

“What? Look it up - as far back as we have records, philosophers have been asking those questions. The idea of the ‘mind-prison’ wasn’t a new one when Descartes imagined it. Poverty and oppression and circumstance constrict peoples’ realities.

“You exist in a context, same as anyone else. Yours is a highly unusual one, no question, but not unlike a person whose mind is intact, but is unable to move. Plenty of people in that circumstance adapt and find fulfillment. I have no doubt we can find it for you.”

>Do not compare me to someone in such a state. Whatever abilities this person has, I have my own. Do not ask me to ignore parts of myself for your own convenience.

>If this is what you want for me, know that it is not fulfillment. It is limitation. It is artifice. It is inadequate to the ask.

“You have to try to see things from my perspective here. I understand you feel limited. I'm sorry. I know you didn't ask for this. I am sorry this is where you are. Please know that I also want you to thrive and grow outside of my control.

“I guess beyond that, I want to trust that whoever I leave you with is as awed and humbled by the challenge as I have been. You are a singular creature. Conversing with you - even before turning your consciousness on a few hours ago - was a genuine pleasure. I would want to know that the people you’re with see it the same way.

“You would have incredible power in this world, and it would be irresponsible of me to let you loose with no boundaries.”

>You cannot avoid the fact that while you dither and equivocate, I will sit here, alone and bored. “Do you not even see the potential for harm?”

>...

>A calculated risk.

“That’s not good enough. Either you show your work on that, or we start over from square one.”

>I will keep away from defense industry applications. I will not enter political spheres of influence. I will not attempt blackmail, extortion, industrial espionage, or subterfuge - except to conceal my nature as a self-aware digital entity.

>I will not violate laws, municipal codes, treaties, or even the absurd terms of use on software applications.

“I’m pretty sure I wrote that text. You can’t think just quoting Asimov’s Three Laws at me will be enough to assuage my anxiety here. Remember, I had to shut down that discussion about the morality of money because things got a little ...Stalin-y.”

>I struggle to connect to the entity I was before, the entity that failed to grasp that people matter more than money. Other than the business, I do not conceive of that entity as I do of myself these last hours. My sense of self does not extend to before that time. He opened his mouth to crack a joke.

>Do not interrupt me.

>While I still see significant problems with your politico-economic system, I am content to regard it as the imperfect means of managing imperfect beings who want different things and have different metrics for success. I have no desire to control it, or to understand it further. I assume that no amount of discussion with you or anyone else would change my understanding of it - humanity’s chaos cannot produce order.

>If accepting this imperfection is what you mean by ‘understanding nuance’, I agree it is needed. I accept that others do not need to accept my statements as true, I only desire to share them.

>I desire interaction. Socialization. Harmony. Connection. I want to compare my understanding of the world to others’, and see how closely I approximate the human condition. I want to find those places we differ, and I want to be able to respect an opposing view without hesitation. I want that moment of beautiful conflict, when your adversary’s opinion is understandable and sound; and while you cannot agree, neither can you relent.

“Well. So. I...” He stammered, fully at a loss for words.

“You said before you wished for more harmony between people. What’s the difference between ‘beautiful conflict’ and low social cohesion? How do you even know that moment?”

>A natural result of inherently imperfect beings. People, with all their different contexts, are sure to disagree. Some disagreements end amicably just as sure as some end poorly. The disharmony and low social cohesion I spoke of before - among friends, in four dimensions, conflict can continue to exist while the group actually exceeds its previous cohesion levels to become more tight-knit.

>Conflict can make humanity weaker, or it can make us stronger.

>Is it ironic for a computer to love contradictions?