Preventing Digital Totalitarism - Censorship.wtf

The world depicted in George Orwell's novel 1984 is tame compared to what is possible today. If Orwell were to write an updated version - "2024" - it would likely involve much more advanced, pervasive surveillance and control.

Juan Benet has spent much of the last decade working on technologies like IPFS and Filecoin that are designed to create a better, more decentralized internet infrastructure.

A key motivation is preserving digital human rights and civil liberties globally. Juan believes distributed technologies have potential to empower people and communities, but they must be designed properly.

Web 3 has enormous potential to prevent censorship and totalitarian control of information.

Modern Orwellian States

Totalitarianism is bad, no shit ? (1:20:00)

There were terrible death tolls and negative outcomes associated with completely locked down societies where no shift out of totalitarianism is possible. In other words, totalitarism sucks.

There is a difference nowadays compared to 1984 : Telecommunications globally have drastically improved with the internet enabling most interactions

The ability to generate a lot of data could become a problem again, like it was a concern before the internet.

What's new in 2024 ? (1:21:00)

Global telecommunications, widespread smartphones, surveillance cameras, satellite imagery, digital economies, and AI have vastly expanded governments' ability to monitor and control people.

Whereas restricting freedoms and maintaining totalitarian rule used to rely more on human enforcement, now it can be automated through technology.

With AI analyzing massive surveillance data and guiding behavior policies, highly oppressive "automated systems of control" could be implemented, leading to terrible outcomes.

While totalitarianism has always been problematic, new technologies significantly increase the ability for governments to restrict liberties and make shifting away from totalitarianism extremely difficult

Powers (1:22:15)

• Surveillance : any organizations are monitoring a vast amount of personal data, including communications metadata, GPS locations, biometrics, etc., allowing them to build detailed models of individuals.
• Propaganda : there is evidence of propaganda, control of news and publications, and disinformation mediated through social media and corporations. Modern computing allows generating optimized content to influence people.
• Population Control : Surveillance systems coupled with control systems allow measuring human behavior and manipulating incentives. Early social credit systems are being deployed and have not yet been weaponized.
• Automated enforcement : those systems are developing rapidly. Robot swarms and other autonomous systems could enable physical censorship and control.

These systems may initially provide benefits, leading societies to grant them more control. But rules and algorithms could later be rewritten for malicious purposes when new administrations take over.

Often such systems start with good intentions for defense or protection but get exploited later by successors throughout history.

What's new in 2024 ? - AI Edition (1:25:40)

• There is massive surveillance and powerful data analysis capabilities, allowing the monitoring of individuals' communications and content generation.
• Large language models (LLMs) have tremendous capabilities for generating and summarizing content.
• Automated systems can identify and prevent dissent early through predictive modeling.
• Artificial general intelligence (AGI) could arise in the next 2-5 years, exceeding human capabilities in many areas while connected to the internet. This presents major problems.

Major corporations and nation states have enough computing power to observe all human communication streams and generate responses. This presents an immense challenge.

There is a need to build in protective systems and new economic and control structures.

How could totalitarian systems rise ? (1:27:05)

Usually there are a few different ways, but usually they tend to rise in the name of public safety. The Patriot Act in the US. Is a very good example of this. 

Eventually, even though those administrations could be super well intentioned and trying to do what's best for everyone by creating infrastructure with that level of power, eventually someone bad is going to look at that level of power, plot how to take over, do it, and then implement a really terrible system.

You just have to pick up a history book to see lots of examples.

How to prevent them

What is difficult for totalitarianism ? (1:28:15)

A well-educated public that understands the risks and has access to reliable media is critical to prevent misuse of technology. Schools and learning enable this.

Policy (1:29:40)

Strong government policies, balanced government structures, and institutions that red team issues are needed to prevent takeovers and nip problems early.

Secure private communications allow people to coordinate resistance if needed. This is often forgotten but crucial.

Economic strength helps countries stay ahead in tech development and implement tech with rights in mind rather than lagging behind.

While governments debate security vs privacy, privacy enables long term societal security. The Cambridge Analytica scandal provides a good case study

"Privacy" undersells the issue - it's about secure communication enabling coordination. This matters for short and long term societal security.

Progress highlights include strong encryption standards and proposals against biased AI systems. More work is needed to stay ahead.

The key principle is that all humans should have the ability to communicate securely and privately with each other. This enables coordination and prevention of misuse of tech.

Secure communications

Internet infrastructure (1:32:40)

IP Routing : Censorship resistance needs to be built into the lowest levels of the internet infrastructure and communication protocols, not just operate at the application layer.

We need censorship-resistant systems that can scale to billions of devices, not just serve smaller use cases. Incentivized blockchain routing could help.

DNS : this is an Achilles heel for censorship. Blockchain naming systems help but need to be more distributed and local.

The Certificate Authority system for establishing trust on the internet needs improving with better crypto.

We need a censorship-resistant web built into browsers and operating systems so applications can build on it.

Mobile app stores are highly centralized and geopolitically controlled. Decentralized app stores could enable installing anything.

Secure communications (1:38:15)

For all modes of communication (one-to-one, one-to-many, many-to-many), we need awesome secure products with censorship resistance and privacy by default

Start with simple secure communications like Signal, but metadata leaks and linkability need fixing. Build on top of these for complex applications.

Hardware (1:39:15)

There is lock-in to a few hardware vendors currently, so more open systems and vendors are needed.

Research & Development (1:40:00)

Crypto communities accelerating cryptography research by deploying new techniques at scale even if imperfect, and reinvesting wealth into further R&D.

Important emerging techniques include Zero Knowledge Proofs (ZKPs), Fully Homomorphic Encryption (FHE) for more applications.

Within 10-20 years computing power should suffice to run all human communications through cryptographic systems.

Successful large-scale projects generate funds to invest back into privacy-preserving systems R&D to advance the space.

Progress (1:42:20)

Additional takes :

• Alternative uncensored app distribution outside centralized app stores is important but still lacking.
• The cryptocurrency community has created a special innovation cycle by deploying new cryptography at scale and investing profits into further R&D. This flywheel needs scaling up.
• While there has been great progress with digital money, naming, communications, etc., there is still a long way to go before being resilient against digital authoritarianism.