Generative AI is dominating headlines this year, and for good reason. The technology is rapidly being adopted across industries and increasingly visible in everyday life. In the disinformation world, there’s been a lot of focus on the potential for generative AI to rapidly produce false and harmful content at an unprecedented scale. While these concerns are valid, they’re really just the tip of the iceberg. The under-discussed danger of generative AI is that it puts users in a one-to-one connection with machines that are not necessarily reliable purveyors of accuracy and removes human feedback from the process. This feature of generative AI has the potential to further upend our already polluted information environment and fundamentally alter society.

Just like radio and television before it, the internet caused huge disruption to our information ecosystem. These disruptions come in phases with new innovations initially being met with scepticism before spreading into the mainstream and eventually becoming wholly integrated into society. 

The internet, however, is a more dynamic medium than either radio or television, one that has been the catalyst for countless subsidiary innovations. Generative AI, for example, is a byproduct of the internet, but is also an innovation that has the potential to completely reshape how we access information and even experience the internet itself. Understanding how this will happen requires us to look at each phase of the internet’s disruption of the information ecosystem. 

Phase One: Don’t trust what you see on the internet 

In the early days of the internet, online information was often treated with heavy scepticism - anyone could set up a web page and publish virtually anything they wanted on it. Before the large-scale shift from traditional to digital media really took hold, the internet was not widely considered a source of reliable information. 

Millennials, typically considered the first generation “raised on the internet,” are deeply familiar with this idea. Parents and teachers taught this generation not to trust any strangers online, to never cite user-generated content repositories like Wikipedia or Yahoo! Answers, and to rely on books and print media instead of digital information. 

The rise of search engines, social media and rapid digitalisation of media changed the status quo. Search engines were able to index massive repositories of human knowledge, traditional news outlets came online and people started to see personal friends - not strangers - post content online. 

Phase Two: Trust, but verify 

Starting in the late 2000s and stretching into the late 2010s, the introduction of smartphones, laptops and tablets transformed access to the internet. As the internet became always accessible at our fingertips and more integrated into our daily lives, the flat standard of “don’t trust what you see online” became untenable.  

While these new tech products helped make the online information ecosystem more diverse and credible by introducing millions of new users and inspiring traditional media to go digital, it also made it harder to disengage from the internet as whole economies moved online. "Trust but verify” became the go-to mantra for the digital ecosystem. Key institutions that rely heavily on public trust, like the Center for Disease Control, joined social media and justified it as a “supplementary” source of information to be verified offline. The US Department of Education, recognising the sheer size and potential of gathering and indexing knowledge online, started issuing guidance on how to find and assess information online. The old “don’t talk to strangers online” rule fell by the wayside as dating apps surged in popularity.

At the same time that users began to put more faith into digital content, cracks started to emerge. Social media algorithms increasingly prioritised content that users would be most likely to engage with over content from real-world friends and family or trustworthy news sources. This created highly polarised “echo chambers” full of disinformation and harmful content which, in turn, began to cause real-life political, social and economic havoc. This forced social media companies to reckon with angry lawmakers, brands and users who started to demand that they take action to reduce the harms of an increasingly chaotic digital information environment.

Phase Three: Trust in isolation

We are now in the third phase of this digital disruption. The internet is omnipresent in nearly every aspect of our lives, including economic activity, healthcare, education and more. Similar to other forms of media before it, the novelty and unfettered optimism of the internet’s potential has eroded by this phase. Instead, the internet is starting to become highly segmented, whereby users are algorithmically pushed into more isolated corners of the web. This segmentation often leads users to get their information from fewer, less traditional sources that typically adhere to their pre-existing biases and from social media users that exist in the same echo chamber. The rapid rise of AI-generated content may very well fall under the third phase of the internet revolution, starting in the late 2010s to the present.

In this phase, “trust” is more a matter of what content adheres to personal biases rather than objective facts, concrete evidence or journalistic standards. The information ecosystem has become so polluted and polarised that faith in mainstream media, major institutions and scientific consensus has significantly declined. While most people across the political spectrum believe that digital disinformation is a problem, they also believe that they would personally not fall victim to it - despite studies proving otherwise. 

The tsunami of AI-generated content is entering into this environment, and most folks are talking about how it has the potential to inundate users with even more disinformation at a global scale. That is a valid concern, but disinformation on the internet has always been a profitable business and can already catch hold quicker than the truth. The real danger of generative AI is that it could mark a huge departure from our current information ecosystem by removing a critical component - the human feedback loop. 

Generative AI is rapidly being integrated into many facets of our daily lives, such as search engine results, customer service and education. It exists in formats that are often opaque and inaccurate, and unlike social media or open web content, there is no place for human users to contest or contextualise potentially false or misleading information. Removing this human feedback loop by having no place for things like comment sections or public forum posts represents a sea change in the trajectory of our online landscape. 

Essentially, human feedback is the only thing that allows users to pop the information bubble that houses disinformation. While much attention has been devoted to social media platforms’ aforementioned “echo chambers,” social media still offers multiple routes for users to push back. X, formerly known as Twitter, recently released a tool called “Community Notes” which allows users to provide context for a post that may be misleading or inaccurate, which other users are then allowed to vote on. This tool, while not without its flaws, provides crowd-sourced information that immediately contextualises harmful and misleading content, thus greatly reducing the risk of its spread and potential to cause offline harm.  

All of the generative AI products on offer today either sharply reduce or entirely remove the human feedback loop, to the detriment of the entire information ecosystem. AI chatbots that utilise Generative Pre-trained Transformers (GPTs) - essentially AI models that use sophisticated algorithms to generate relatively sophisticated text -  are the perfect example of where accessing information without human input can cause tremendous harm. When users interact with GPT-powered AI chatbots they do so in isolation by feeding questions into the chatbot, which then spits out a response based on broad datasets that are not readily accessible to the user. Unlike reading an article on a site or listening to a lecture, there is no built-in opportunity for other users to potentially push back on the AI-generated information. This absence makes it far more likely that many users will simply take the information at face value.

If the AI-generated content is accurate, then there’s no problem, but due to the fundamental design of LLMs, a certain amount of “hallucinations” - generated responses that are at odds with reality - are inevitable. That means there is currently no way we can completely trust AI-generated content - but in the “trust in isolation” phase of the digital revolution, many people will trust it regardless.

We’ve already seen the dangers of our modern tendency to trust in the case of a New York lawyer citing completely made up legal precedents they got from ChatGPT in a real-life court case, and in researchers uncovering fictitious articles from reputable news outlets in generative AI search results. Google Search has come under fire recently for answering search queries with LLM-generated falsehoods, such as telling users to use glue to stick cheese on their pizza or insisting that a dog has played in the NBA. Without some kind of institutional check, this trend will continue and become more dangerous the more we integrate generative AI into daily life. 

The people, data and processes that create GPTs and other generative AI products are often opaque and democratically unaccountable because they are owned and operated by private companies with a motive to maximise profit. Since that sharply impacts the type of information that generative AI will produce, it is important - but currently impossible - to practise “trust but verify” with generative AI at scale. 

Many organisations, including GDI, are focused on retaining and scaling the human feedback loop amidst the generative AI boom. While companies building generative AI products naturally rely on their internal employees as the main conduits for human feedback, partnering with civil society organisations can help integrate more robust and specialised data and expertise into the loop. Generative AI has great potential to revolutionise things like education, economic development and more for the better, but only in conjunction with expert human review as a necessary guardrail. 

To function effectively as this guardrail, the organisations building them need to prioritise minimising bias, setting strong ethical guidelines and including a wide range of both internal and external experts to help build and train generative AI models. The outputs of generative AI are a direct reflection of the ideas, cultural backgrounds and ethics of the people who build and train the algorithms that power them. Ensuring the most robust, accurate and non-prejudiced outputs possible requires generative AI models to include an equally robust, accurate and non-prejudiced human feedback loop. Otherwise, we risk fracturing an already delicate digital information ecosystem - perhaps beyond repair.