Tech Roundup: AI "Slop" And The Risk Of Misalignment

"Slop" may be a term you have heard a little more recently in the AI space, and it seems to refer to a new type of content: quick, easy, cheap.

We have looked at AI video generation before and weighed up the pros and cons, and have discussed a potential a new wave of media: AI generated; niche, weird, and yet derivative - and there could be a lot of it.

Is this a problem?

Some are arguing we have reached a turning point for the internet. Up until recently, all of the online media we've consumed has been created by a human for the most part. The internet has been a massive log of humanities thoughts, feelings, and findings and it has been the perfect training set for the likes of ChatGPT, Gemini, Perplexity and Claude.

The result has been impressive chatbots that are knowledgable and sound like us. They have been trained on human-generated data and know how to make us happy. Their point of reference is a giant filing cabinet stuffed with the things that we, the human race, have collectively written, filmed, and photographed over the years since the internet was born.

But that filing cabinet now has a new contributor; AI itself. The next post you read could be written by a robot.

So how does this affect things? Is the internet polluted? And how will we train the next generation of AI models now that our dataset is filled with human content and robot content. These two types of content are not the same. We know that AI content is not always accurate. What would happen if these inaccuracies are fed into the knowledge set of the next group of models? Will these inaccuracies become magnified?

An example

Human written fact: "Lawyer Paul gives his first consultation free of charge."

AI Summary: "Lawyer Paul offers fee-free advice."

In the above example, what the AI has said about Lawyer Paul is true, he does offer fee-free advice, but a crucial fact that is missing is that this only applies to the first meeting.

There are many reasons why the AI may not be compelled to mention this important detail. Perhaps the terms and conditions of this fee-free advice doesn't fit in with its personality settings, and it has been told to stay snappy and positive when recommending services. Remember - it is often guessing what we want to hear, based on its instructions.

This inaccuracy could then become magnified if the watered down fact about Lawyer Paul (Lawyer Paul offers fee-free advice) were to be posted online and used as a source for another bot. Perhaps this next bot also has a slightly different vernacular, and by the time it has interpreted the statement, it reports that Lawyer Paul works for free. Too good to be true?

In the above example, we have painted a scenario in which misinformation has been cultivated by chaining AI summaries. Lawyer Paul does not work free of charge, but thanks to a game of Chinese Whispers, it eventually becomes stated that he does. This is called misalignment and it is a potential problem. If you would like to get into the nitty-gritty of it there is a paper written on this phenomena. This paper looks at how a an AI behaves when trained on inaccurate and insecure content and explains some of the misalignment terms in more detail.

How to combat this?

This is something everyone using AI and creating AI solutions should be wary of. If you are creating a chatbot or service for your business you must do your upmost to make sure it is aligned with the facts of your company and your services. Make sure it consistently refers to rich, detailed human-written content with a RAG system or similar when deciding what to say. It is tempting to use AI-summarised content in your RAG library but it could lead to issues like the Lawyer Paul example.

As for the internet? Get some human content out there: songs, videos, ideas. AI is fantastic for getting us information quickly, but lets keep the source information accurate. You can rest assured that you have just read a human-written blog.