Welcome to the twelfth interview with the science-fiction writer Radivis. It has been quite a ride so far, but there is still so much left to discuss.
Sockrad Radivis Interview XII: Coherence, Universal Value System, Sophocrats, Optimizers, Prediction Simulations, Economics of the far Future
Yes, for example the V faction called “Coherence”, which I’ve already mentioned a couple of times already. In this interview I will focus on that late V faction. As a reminder, the Balance created a synthetic value by adding up the values of the other V factions multiplied by certain weighting factors. I already mentioned that there were alternative sub-factions of the Balance which pursued a more complicated approach compared to simply adding up certain values. The Coherence studied the structure of the relations between different values in detail. It attempted to come up with a singular value comparable to the “recellence” of the Exaltation, but one which fairly encompassed all “pure” values of the other V factions.
What would a singular value like that look like?
It is a value that depends a lot on the context, for example whether you are in a training simulation or an “ordinary” world, and on your degree of knowledge about the world and ethics. The fancy thing about that particular value is that it considers any kind of possible context in a highly structured way. The evaluation procedure for evaluating value and the context of value uses a unified framework, in which it effectively ceases to make sense to distinguish value from its context at all. It’s like asking what 2+2 equals to, but you get the question why you want to know that exactly, and the actual result is a combination of the purely mathematical result with the processing of your motivation to desire to know it.
Why does the context matter at all?
Let’s take the context of a frowney simulation, for example. That is a simulation in which suffering only appears to be real, because actors imitate the behavior of suffering beings without actually experiencing that suffering. If you are in such a simulation, trying to improve its happiness by reducing suffering in that world would be highly misguided. At the same time, you might not know for certain in what kind of simulation you live, so the quality of your decisions will depend on your knowledge about that simulation and the degree of certainty of that knowledge. Your knowledge also determines the number of options you are even able to consider, and the areas in which you can consciously strive for excellence. Ignoring these contextual factors is unwise, so there is a need to incorporate those into your ethical framework.
How have the other V factions dealt with the problem of context?
By integrating over all possibly relevant contexts, weighted by the probability of that context. That’s actually a highly sophisticated operation, and the basic foundational achievement of the V factions. Nevertheless, it’s not complex enough, since it presumes that there are no complex inter-dependencies between value and context. These inter-dependencies turn out to exist, and a framework analogous to the general theory of relativity, which models the interactions of space and time, was developed to integrate those inter-dependencies. This necessitated the development of a grand theory that encompassed concepts like space, time, probability, mind, information, value, and context in an absolutely universal way. That theory was simply called the “Universal Value System ”.
What is the relation between that Universal Value System and the UVS theorem?
The Universal Value System was developed way before the theorem was completed. What the theorem basically does is to state the the UVS is essentially the only fully correct way to do ethics. Any other conceivable attempt at doing ethics must be flawed. Of course, that is an insanely strong statement, and incredibly hard to prove, which explains why the proof of the UVS theorem took so long.
And in the meantime between the development of the Universal Value System and the proof of the UVS theorem the Coherence had the status of a regular V faction, because nobody could be sure that the UVS is the right way to do ethics?
Correct! There was a tumultuous time in which the other V factions also made progress in the underlying theories, incorporating parts of the Universal Value System into them, thereby reducing their ideological difference to the Coherence. That time was called the Era of Convergence , which found its culmination in the proof of the UVS theorem. The modified value systems of the V factions were commonly referred to as “advanced value systems ”.
Was incorporating parts of the UVS into their own philosophies seen as a good idea universally by all V factions, or were there opposing voices?
It was certainly a great paradigm shift, but over time it was accepted broadly that a correct value system should have strong structural similarities to the UVS. The resistance against this change died down after a couple of decades.
Were there great changes in the way that the V factions operated during the Era of Convergence?
Yes, the advanced value system theories were more complex than the previous V theories. This caused an increasing stratification within the V factions between the very advanced AIs which used the advanced value system theories and the more intermediate AIs, which used the older V theories. The first group were eventually called sophocrats . Furthermore, the ideological differences between the V factions became less pronounced, which caused a large increase in their collaboration.
What role did the sophocrats play?
They were the new rulers, mediators, advisers, and consultants. Their judgement was generally given more trust than that of anymind else. Typically, they were highly valued and respected.
Wouldn’t most people want to become sophocrats then? If so, what stopped them from becoming sophocrats?
Becoming a sophocrat required a substantial amount of computational power. Turning everyone into a sophocrat was just as economically challenging as providing everyone with a large star-ship - theoretically doable with a small populations, but impossible with large populations. Since the system of cultural silvanism favored rather large population, those were quite typical.