Since launching ChatGPT, OpenAI has continued to work on new AI projects that build on the success and popularity of its AI chatbot. Now, the appearance of a new mystery chatbot gives the public a sneak peek at its latest project — and it’s impressive.
Last week, a new chatbot named “gpt2-chatbot” appeared on the Chatbot Arena, an open-sourced platform used to compare the performance of the latest large language models (LLMs).
Also: Google was right to be worried: OpenAI reportedly wants to enter the search market
The appearance of this chatbot caused quite the buzz because of its impressive capabilities, which have outperformed many of the most popular LLMs on the market, such as Gemini, Claude, and even GPT-4. However, to the disappointment of many, gpt2-chatbot was removed from the Chatbot Arena shortly after appearing on the platform.
Starting last night, if you visit the Chatbot Arena, you will encounter what seem to be two variants of the original chatbot, called respectively, ‘im-a-good-gpt2-chatbot’ and ‘im-also-a-good-gpt2-chatbot’.
Despite the two models sporting the “GPT” title exclusive to the OpenAI’s family of Generative Pre-trained Transformer (GPT) LLMs, the company has not officially acknowledged that it’s working on the model. However, OpenAI CEO Sam Altman wrote a cryptic post on X that just stated the name of the LLM, “im-a-good-gpt2-chatbot,” which seems to confirm the connection between OpenAI and the model, as seen below.
im-a-good-gpt2-chatbot
— Sam Altman (@sama) May 5, 2024
Even though the models are available in the Chatbot Arena, accessing them is tricky. If you look at the list of LLMs supported by the Chatbot Arena, you will not see the models listed there. As a result, you will not be able to test them in the Chatbot Arena (side-by-side) comparison since they’re not included in the dropdown.
Instead, if you want to access them, you must keep initiating an Arena (battle) comparison — which randomly selects two LLMs to compete against each other — until one of the two new models comes up. It took me five rounds to finally have it appear, as seen below. However, if you’re determined to test it for yourself, the wait might be worth it.
Once the model is revealed and you see that it’s either “im-a-good-gpt2-chatbot” or “im-also-a-good-gpt2-chatbot,” you can keep chatting with it by adding new questions to test its capabilities for yourself. You can continue chatting until you decide to start a new round or refresh.
Also: These four new Copilot for Microsoft 365 features make prompt writing like a pro even easier
People are taking to X to share the new anonymous model’s impressive capabilities, which include creating a Flappy Bird clone with one prompt, creating a code interpreter that uses Claude Opus, and even reasoning through basic physics questions.
These improvements have led people to speculate that the model is OpenAI’s GPT-5 or GPT-4.5, released under a penname so that OpenAI can benchmark its performance accurately. When a user asked the “im-a-good-gpt2-chatbot” what LLM model it was using, the chatbot did say, “I am based on the GPT-4 architecture, specifically the GPT-4.5 variant.”
However, there is no assurance that this isn’t the result of a hallucination; until OpenAI confirms anything, it is best to err on the side of caution when using the chatbot. However, if you are even the slightest bit curious, I encourage you to give it a try since it is free.