You seem to be implying that ethics is an extremely hard, but ultimately solvable problem. Is that so?
That might be the most daring premise of Canonical Coherence, and it’s actually the foundational premise of that world. Everything else is basically exploring the ramifications of truly advanced AI to the max. I am not so sure about how reasonable that assumption of a provable and unique Universal Value System Theorem is for our world, but I propose that it’s reasonable to hope for that actually existing.
Why do you think it’s reasonable to hope for something that might not even exist?
Because it would solve so many problems, if it actually existed. We can decide to eventually give up on that hope, but we shouldn’t do so prematurely. If the most advanced ASI that is able to exist within this universe won’t get closer to finding and proving such a Universal Value System Theorem for millennia, we might reconsider our hopes and strategies, but it would be a mistake to do that sooner. After all, if we manage to find that theorem, it would enable lasting cosmic peace.
I see. So, I guess you wanted to display the consequences of such a theorem actually being found, right?
Yes, Canonical Coherence is an exploratory work that tries to figure out how life really might be, if a discovery of such magnitude was actually made. And that does make it special indeed.