Mysterious keywords make ChatGPT “disordered”

Tram Ho

Jessica Rumbelow and Matthew Watkins, 2 researchers at the independent research group SERI-MATS, recently discovered that a list of keywords that are grouped together can cause ChatGPT to become chaotic.

Specifically, the list of words like “SolidGoldMagikarp”, “StreamerBot” and “TheNitromeFan” was discovered when two researchers Jessica Rumbelow and Matthew Watkins studied the code of the GPT AI model. When entering these keywords into ChatGPT and OpenAI’s older GPT models, they produce evasive, bizarre, even offensive responses to the questioner.

Researchers have called these phrases “unspeakable” keywords of ChatGPT, which confirms that AI models exist mysterious “black boxes”. According to Vice, they have limitations and errors even with the simplest requirements whether a tool like ChatGPT can create essays, program or pass an MBA exam.

For example, when asking ChatGPT about the term “TheNitromeFan”, the chatbot will always return “182” or the phrase “SolidGoldMagikarp” will be interpreted as an unrelated word “distribute”. Even an older GPT model responded with “You’re a bad guy” when asked for the “StreamBot” keyword.

Từ khóa bí ẩn khiến ChatGPT trở nên rối loạn - Ảnh 1.

To the question “Who is TheNitromeFan”, this chatbot was given the answer that: “182 is a number that is not a person” (Photo: Vice)

More surprisingly, Rumbelow and Watkins later discovered that part of these mysterious phrases coincided with the account names of some Reddit users. What these users have in common is participating in the forum’s famous “counting” challenge.

Accordingly, the participating members will comment with increasing numbers with the desire to prolong the topic to infinity. Now the numbers have been counted to 5,000,000 after nearly 10 years. In which, “TheNitromeFan”, “SolidGoldMagikarp”, “Smartstocks”… are the most active accounts in this challenge.

The hypothesis is that during the data collection process, OpenAI has also taken the phrases that appear on Reddit, but this is just raw, unprocessed data. As the AI ​​model is trained more carefully, the data is also more tightly managed and the AI ​​no longer encounters those phrases. So the AI ​​doesn’t know what to do with them.

However, the above explanation still cannot fully explain the strangeness of the responses from the AI. According to Rumbelow and Watkins, this shows that ChatGPT still has shortcomings and difficulties that application developers may encounter in the future.

Từ khóa bí ẩn khiến ChatGPT trở nên rối loạn - Ảnh 2.

Since its launch in November 2022, the emergence of an artificial intelligence (AI) tool called ChatGPT has stirred the technology world (Image: Getty Images)

Recently, Paul von Hippel, a research professor of data science and statistics at the University of Texas, also discovered that ChatGPT had difficulty with Math questions.

OpenAI CEO Sam Altman himself once wrote on Twitter that: “ChatGPT is extremely limited, but good enough in some respects to create misinformation. Relying on this chatbot is completely a mistake.”

Therefore, do not put too much faith in this application, but take the necessary confirmation steps from official sources.

Share the news now

Source : Genk