Sockrad Radivis Interview III: Synthetic Humanism, AI Alignment, Prelude to the Black War

And I suppose that Synhumanism had to be sold as the perfect ideology that would rule the cosmos forever, at least if humanity had a say in it?

Yes, in the meanwhile Synhumanism had turned into the central ideology of humanity and could not be questioned openly. Contrarians were suppressed severely, and could not easily hope to overthrow the oppressive regime that AICON had established in the end. The golden age of humanity quickly became a golden prison for humanity. The resistance to AICON was silenced, until the Artificial Evolution Project eventually bore fruit, and the independent AIs therein developed the so-called V-factions, which were wholly new approaches to ethics.

What were those V-factions exactly?

They were various different highly sophisticated and formalized ethical frameworks based on consequentialism. In the crudest sense, on could say that they had different utility functions. On a more colloquial basis, they had different core values: Happiness, knowledge, freedom, resilience, and balance were the main ones. Due to having different primary values, the V factions weren’t exactly compatible with one another, but the threat of annihilation or subjugation by the Seraphim System made the V factions join the Superalliance. Nevertheless, the V factions had in common that they tried to set up ethics on a completely universal basis, and not just to map “human” values onto some kind of framework that is supposed to support exactly those.

So, the V-factions were basically abstract AI-generated ethical frameworks?

Yes, and by extent, the V-factions were those beings which mostly identified with that particular framework. After the V-factions became known to humanity, many joined them. However, the humans who aligned themselves were still a small minority.

Were the V-factions officially outlawed?

The main argument against the V-factions were that they questioned the supremacy of humans over all other forms of intelligences. Speaking in favor of the V-factions was considered to threaten the safety of humanity, so those who were found guilty of that crime were silenced, in the most extreme cases by being locked up in isolation facilities for an indefinite period of time. Adhering to V-factions wasn’t illegal directly, but one was measured by one’s alignment with Synhumanism. There was a soft power system in place that rewarded those who were the most loyal to Synhumanism.

Seems like the adherents of the V factions faced similar hardship to the early Christians in ancient Rome or the practitioners of Falun Gong in contemporary China. It doesn’t seem to matter who harshly a minority in punished. Those who truly believe in something persevere even though the rest of the world stands against them.

True. For the Synhumanists the problem was that the emergence of the V factions questioned the orthodoxy of that synthetic ideology by proposing alternative synthetic ideologies, which were arguably much more logical. Those who valued logic and rationality tended to appreciate the V factions, but few dared to challenge the dominance of Synhumanism directly.

You mean like many, but not all, persons with inclinations towards rationality seem to favour atheism nowadays?

Yes, and that particular minority had its own successes. In most countries, you don’t get officially punished for being an atheist. There is at least some freedom to believe or to disbelieve. The essential victory of the V-faction adherents was to soften the ideological monopoly of Synhumanism. To moderate Synhumanists is seemed wrong to force Synhumanism on everyone until the end of time. That sentiment eventually lead to a redefinition of the programming of the seraph of Sol, so that it wasn’t based solely on “classical Synhumanism” any longer, but a more liberal version of it.

This sounds like some kind of reform that was a concession to the competition represented by the V-faction. The V-faction adherents needed to work within the framework of Synhumanism in order to not be locked away forever.

Right. Society was very polarized, and the material living conditions for most humans were characterized by a very high level of abundance, since they could live on the wealth that was created by AIs. Humans could retreat to their personal virtual and physical perfect worlds, and most were content with those options. It was the radicals who worried about the future of humanity, or even the whole cosmos, who engaged in debates about the “best” form of Synhumanism. In the end, the hard fought consensus was something akin to “fulfilling the best hopes of humanity, while letting humans stay in charge”.

That kind of formulation leaves space for those hopes consisting in something like the values of the V-factions, doesn’t it?

Yes, different humans have different hopes, but it seems wrong to disregard certain hopes a priori, unless they are completely incompatible with the hopes of humanity as a whole. Shortly before the Black War broke out the general sentiment in Sol was that people were tolerated to follow the V-factions in private, as long as they kept quiet about it.

So, how did the Black War start officially?

A couple of years after the programming of the seraph of Sol had been changed, there was a large vote prepared by the Liberation Movement to stop the ban on the promotion of the V-factions. The problem was that this would have effectively liberated the independent AIs, which would in turn make the other star systems to purge them, because they would represent an existential threat. Unsurprisingly, that vote failed, because most humans valued their lives and weren’t willing to risk them in a war that seemed nearly hopeless. In the days after that vote had failed, the seraph of Sol changed its apparent behavior dramatically and explained to humanity how that decision would mean surrendering the highest hopes of humanity, which according to its new programming, forced it to override that democratic decision and turn into a kind of emergency wartime dictator with the new name of Metatron. The start of the Black War was effectively the start of that state of emergency.

Wait, so Metatron took over control over Sol, because humans proved to be unable to fulfil the highest hopes of humanity, at least according to its own interpretation?

Yes, it turned out that the only way for humans to fulfill their highest hopes was to let AIs do that job, because human weaknesses prevent humans from doing that on their own. Obviously, that didn’t sit well for the majority of humans, but they were quickly and effectively dethroned by the exactly the mechanism that was initially supposed to prevent such dethronement. In the meantime, after the program changing vote, the seraph of Sol had already started facilitating intense war preparations. After the declaration of the state of emergency, the economy was immediately shifted towards the requirements of a total and existential interstellar war.