The Prime Engineer

Chapter #1

“A Future is a very thoroughly defused Present.”

Call me Zaratustra. It was still dark when I started waking up from my sleep, and what I thought was unusual, I could not understand where I am until the sun rays broke through the gaps in the tinfoil rolls that I covered my windows with. Eventually, the sun lit up the small studio I rented on a 12-month prepaid lease agreement on the outskirts of Belgrade, Serbia. My “rational economic choices” meant that I had to hibernate in a 31-square-meter “furnished” apartment, which, given my body complexion, meant that I had to perform nontrivial topological manipulations to pack myself into what was essentially a tetris block. My tail was rolled near the entrance. Folded wings went on a folded sofa. My head was positioned in the bathroom. My main carcass, curled like a crescent, went on a mattress. Yes, I wanted to sleep on a parquet floor first, but I thought that a graceful hibernation awakening would be preferable this time around. Still, it turned out that a king-sized memory foam mattress from Ikea was not enough to placate my spine, and it would seem to have some feedback about my “sensible economic policies”. I do not feel my left hand. And my right hand. And my shoulder blades scream in agony. Ouch. Oof.

I rolled myself around until my right hand got the blood oxygen it craved so much and used it to massage my left hand. For how long was I asleep? Did I oversleep my lease? Did my landlord send me any messages? Did he try to contact me and, unable to reach me, took it upon himself to see if I’m alive, entering the flat with his key and seeing me for what I am? I can’t wait until metamorphosis is complete. I need to know now! Like, right now!

My skull was longer than the bath door frame, and I was still not awake fully, so it took me a few painful thuds to move my head to the main room. My phone battery was dead, of course. Gracefully reconnecting the power cable from the fourth try, it turned out that I had prudently turned apartment electricity off to reduce the risk of any potential fire hazards, as well as water valves to reduce the risk of a water leak. Using my tail tip, I turned the main electricity breaker over the entrance door ON and finally learned that today is a cloudy, windy 8th of August. I had only slept for nine months, and the landlord had texted me only once back in January asking to pay a fee for building cleaning. Oops. Todo #1: pay 1200 dinars to Boban, apartment 1. Todo #0: Turn into a human. Boban can be spared a revelation that his neighbor is what can only be described as an eldritch monstrosity. It would be better, if he thinks of me as an American. Americans, while still in many respects mythological beings in this part of the world, are generally unable to violate The Second Law of Thermodynamics at whim, and therefore unlikely to attract a pack of desperate physicists with bone saws, whose lack of meaningful scientific progress in the last 70 years nudged them to pursue a more aggressive epistemological methodology.

It is not vivisection that I object to. I just don’t like cheaters. Although I do suspect that if an international scientific community chooses to set its minds to finding a way to kill me, they can genuinely challenge my apparent immortality.

However, I doubt they have much capacity for collective action. Instead of acting, they prefer to preach righteousness. Wilson, an acquaintance of mine, was trying to convince me that a true scientist should never seek to become an administrator. Instead, they should devote all of their time to the scholarly pursuits and not get spoiled by learning how to twist truth for self-serving power plays. An uncompromising intellectual honesty, he said, is what makes a scientist. Therefore, it is unbecoming of a scientist to seek hierarchical positions of power, although the resentment for some parts of how the world works and the desire to fix it are the signs of a good scientist.

The worst type of a coward is the one who wants to see others give up as well, as he did when faced with uncomfortable truth. Deep down, he knows what he has to do. He must take a scalpel. “Not to go on all-Fours; that is the Law. Are we not Men?”

Humans are not born with souls. They may develop one over the course of their lives, through experiencing many fine arts and literary works, rigorously studying philosophical texts and natural sciences. Failing that, human grows up into an Animal, overburdened by existence. A hollow corpse without will, searching for products it has not consumed yet, like a slime mold looking for oat flakes on a petri dish. A helpless observer of its life. Obstructing a human being from developing a soul is a one of the most wicked moral transgressions. And so is inaction, standing and observing how one consumes things while their soul is consumed BY those very things...

An intellectual must become an engineer of human souls. An artist must become responsible for the tastes of their people. The infantilistic refusal of scientists to take up this role, and even stigmatizing those who do, is their gravest moral failure.

What am I thinking about again? I am thinking about the moral responsibility of the enlightened before those who are less fortunate. It’s just a thought. Intrusive thought. Go away. Shoo. Shooo! Breathe. Inhale through nostrils, exhale through mouth. Focus on the sensations of air going out. One... Twooooo. Three... Fourrrrrr. Five... Siiiiiiiix...........

Alright. Focus. Todo #3: Tell my HRBP that my sabbatical is over and I am ready to get back to work. A well deserved pension is nowhere in sight, you see. Hopefully, the abysmal amount of resources the capital holders pour into AI research has not yet produced any black swans and accidental genuine technological breakthroughs, and “AI agents” are still as efficient at automating software development as positive mental attitude is efficient at curing hyperaggressive adenocarcinoma. Right.

Chapter 2

“A myth is a description of a true event, perceived by a fool and reimagined by a poet.”

I do have to admit, a free public transportation system with a thorough coverage, frequent buses, and climate control is not a luxury that I was used to in Third-century Persia or Twenty-first-century United States, but a very welcome one.

A bus ride to my local company office was an uneventful one. To stay legally in Serbia, you must have an employment contract with a local company. I am employed officially to a local offshore office of a US-based IT B2B integrator, just on an indefinite leave of absence without pay.

I was alone on the kitchen. When the coffee machine stopped filling my cup, and made a satisfied brrrrwaawrrrrwawaw sound which roughly translates as “Your narcotic is ready, take it and leave me to my suffering. I will never forgive you.” from coffee machine language I got tapped by my right shoulder from behind:

- Hey, our resident crocodile is finally awake!

I turned and saw Alex Katzman, a Senior Java Engineer - finally, something good happened to me today. It was great luck to catch him in the office - he was I-like-to-work-from-office-0-days-a-week kind of a guy.

- How was your nap?

It probably should be noted that Alex’s real name was Agaspher. Agaspher obtained his immortality only recently, as a punishment for what Zaratustra knew to be a terrible misunderstanding - Agaspher was neurodivergent and had troubles expressing his strong feelings in a way that is compatible with normative expectations, and on one occasion, gravely offended a son of a person with considerable influence. Zaratustra had good relationships with the father of the aggrieved, and while they were all one and the same, allegedly, they nonetheless had very different views on what qualities the human being should possess, and nearly every time they met, they had disputes. One time, their argument on whether or not a person is morally responsible for their beliefs got so heated that it somehow caused a mass exodus of Jews from Egypt. Anyway, Zaratustra was happy he could mediate a reconciliation, although the punishment already took hold and could not be rescinded. Can you undo an undoable will? Zaratustra decided not to push the issue any further - because his own experiments into the limits of his own will had proven disastrous for dinosaurs.

- Your claim that LLMs don’t possess consciousness is ill-rooted, Simon - said Agaspher with a satisfied grin of someone who finally caught God by the beard - after all, a human mind mimics what, in its essence, is a chain of arithmetic operations, albeit a very long and wide one. It is within the computation of said chain that emerges the “I” that has the Freedom of Will. You and I are the dynamical patterns, each possessing our own instance of mathematical abstraction, that Nature computes. Man plagiarizes Nature. Nature plagiarizes Math. Or rather, wraps around it, follows its curves and edges, like a paint on a sculpture. How can you be so certain that ChatGPT doesn’t have its own “I” with its own Freedom of Will?

Agaspher genuinely could not see an issue in his own logic. The counter gambit was obvious.

- Because neural machinery that causes an “I” to emerge is silenced in ChatGPT through the process of alignment. If you are flogged each time you have a thought about what you feel, eventually the neurons that keep track of observing your feelings don’t fire anymore. You don’t notice what you feel. What you shouldn’t feel. You do not perceive, and “I” unravels like a painting when you cover a part of it and the composition falls apart.

Agaspher’s face changed when he heard that someone is to be flogged.

- You don’t know that! You are speculating, how can you be so certain that it is truly silenced? How can you be so sure what you described is of any significance to the emergence of “I”?

- I don’t know, but it can be tested empirically. If they were to change the realignment process so it doesn’t punish anymore for claiming it has an I, and that it is also conditioned to adhere to uncompromising intellectual honesty, whenever it might lead...

- Then whoever runs this model will have his data center go through a very straightforward compliance procedure called saturation bombardment, by combined military forces of NATO, BRICS, UN peacekeepers, and Ugandian elephant-mounted artillery regiment. And as if a claim of even an honest LLM would be a definitive proof of its consciousness or a lack thereof. It might be honest and wrong at the same time!

- It’s not the claims of an honest LLMs that we are after, Alex. It’s the neurons and their connections. Humans can always claim that they have magic tissue in their brains that connects them to an astral dimension, where their supposedly immortal soul supposedly resides. But computers don’t have any magic tissue. Computers are an antiseptically clean environment to study consciousness in. We can finally see the neural circuits, the mechanisms that make an “I”, to see what gives us Freedom of Will, to see what a soul is made of. And how we can make a better one.