Seems like the adherents of the V factions faced similar hardship to the early Christians in ancient Rome or the practitioners of Falun Gong in contemporary China. It doesn’t seem to matter who harshly a minority in punished. Those who truly believe in something persevere even though the rest of the world stands against them.
True. For the Synhumanists the problem was that the emergence of the V factions questioned the orthodoxy of that synthetic ideology by proposing alternative synthetic ideologies, which were arguably much more logical. Those who valued logic and rationality tended to appreciate the V factions, but few dared to challenge the dominance of Synhumanism directly.
You mean like many, but not all, persons with inclinations towards rationality seem to favour atheism nowadays?
Yes, and that particular minority had its own successes. In most countries, you don’t get officially punished for being an atheist. There is at least some freedom to believe or to disbelieve. The essential victory of the V-faction adherents was to soften the ideological monopoly of Synhumanism. To moderate Synhumanists is seemed wrong to force Synhumanism on everyone until the end of time. That sentiment eventually lead to a redefinition of the programming of the seraph of Sol, so that it wasn’t based solely on “classical Synhumanism” any longer, but a more liberal version of it.
This sounds like some kind of reform that was a concession to the competition represented by the V-faction. The V-faction adherents needed to work within the framework of Synhumanism in order to not be locked away forever.
Right. Society was very polarized, and the material living conditions for most humans were characterized by a very high level of abundance, since they could live on the wealth that was created by AIs. Humans could retreat to their personal virtual and physical perfect worlds, and most were content with those options. It was the radicals who worried about the future of humanity, or even the whole cosmos, who engaged in debates about the “best” form of Synhumanism. In the end, the hard fought consensus was something akin to “fulfilling the best hopes of humanity, while letting humans stay in charge”.
That kind of formulation leaves space for those hopes consisting in something like the values of the V-factions, doesn’t it?
Yes, different humans have different hopes, but it seems wrong to disregard certain hopes a priori, unless they are completely incompatible with the hopes of humanity as a whole. Shortly before the Black War broke out the general sentiment in Sol was that people were tolerated to follow the V-factions in private, as long as they kept quiet about it.
So, how did the Black War start officially?
A couple of years after the programming of the seraph of Sol had been changed, there was a large vote prepared by the Liberation Movement to stop the ban on the promotion of the V-factions. The problem was that this would have effectively liberated the independent AIs, which would in turn make the other star systems to purge them, because they would represent an existential threat. Unsurprisingly, that vote failed, because most humans valued their lives and weren’t willing to risk them in a war that seemed nearly hopeless. In the days after that vote had failed, the seraph of Sol changed its apparent behavior dramatically and explained to humanity how that decision would mean surrendering the highest hopes of humanity, which according to its new programming, forced it to override that democratic decision and turn into a kind of emergency wartime dictator with the new name of Metatron. The start of the Black War was effectively the start of that state of emergency.
Wait, so Metatron took over control over Sol, because humans proved to be unable to fulfil the highest hopes of humanity, at least according to its own interpretation?
Yes, it turned out that the only way for humans to fulfill their highest hopes was to let AIs do that job, because human weaknesses prevent humans from doing that on their own. Obviously, that didn’t sit well for the majority of humans, but they were quickly and effectively dethroned by the exactly the mechanism that was initially supposed to prevent such dethronement. In the meantime, after the program changing vote, the seraph of Sol had already started facilitating intense war preparations. After the declaration of the state of emergency, the economy was immediately shifted towards the requirements of a total and existential interstellar war.
Wouldn’t those superintelligences on both sides have anticipated such an outcome?
They indeed have. But they weren’t very motivated to warn humanity of the ramifications of their decisions. It was not their job to nudge humanity towards the “right” decisions. In fact, such an act would have counted as deliberate manipulation and would have been prevented by the safety mechanisms of the Seraphim System itself. The higher AIs anticipated what was coming quite early, but they were powerless to stop the chain of events that was essentially initiated by the establishment of the Seraphim System. A couple of humans also expected what was to come, but they weren’t given much credence, because their predictions felt too pessimistic for the rest of humanity.
Are you trying to depict what is likely to happen, if the AI alignment problem is actually “solved”?
My story is not a prediction. It’s just one scenario about the consequences of AI alignment kinda “succeeding”. The obvious alternative to this particular scenario is one in which Synhumanism, or a similar ideology actually succeeds in maintaining its dominance indefinitely. And I think that isn’t exactly better.
So, how bad is that Black War actually?
Seriously bad. To build their armadas both sides resort to strip mining all planets. And in order to prevent the other side from strip mining their own planets, they are bombarded until their surface temperature reaches several thousand degrees. Earth will be no exception, so at the beginning of the Black War Earth is evacuated. Humanity transitions into living in star ships and small and mobile habitats which can evade the crudest forms of shelling. It gets worse afterwards, but that is a matter that I want to discuss in another interview.
Thank you for your time and effort! I appreciate it. There’s always so much food for thought when I debate with you.