[Although our current situation is mostly stable, it is very tense. The Aurelians rely on AICON preventing any dangerous, or even unreasonably disruptive superintelligence to appear and break lose, but most of them realize that they can’t stop the technologically fuelled evolution of intelligence forever. Their most realistic hope is to slow it down so much that it becomes totally non-disruptive and everything that is valuable about being “human” is preserved indefinitely.
The Upgraders, on the other hand, see AICON as despotic inhibitor of further progress. They fear that the era of rapid progress will halt very soon, once AICON starts to actually impose severe limitations on self-improvement of individuals. So far, AICON is closely monitoring many anthropoids with severe modifications that are classified as experimental and “potentially threatening”. The reason AICON can be lenient and allow anthropoids to modify themselves rather freely is its own position of clear superiority. Actually, the superiority of AICON is quite terrifying, and it’s the most effective system for suppressing any internal opposition that has even been designed.
Before the development of the peacenets, people were worried about “unfriendly AI” going “FOOM”, meaning that it would improve its own intelligence explosively. Unfriendly AI would then go ahead to do things which are “unfriendly” to humans. This motivated many people to attempt to invent a “friendly AI”. So far, the efforts in this direction have turned out to be rather futile. Instead, the peacenets were accepted as solution to the “control problem”, so that even if powerful AIs turned out to be unfriendly, we would still have the power to control them and prevent them from doing any real harm. Today, the ideas of a “friendly AI” are mostly seen as old-fashioned science-fiction nonsense. They have been replaced by the cold and efficient protection provided by the peacenets.]