Inflection, the AI startup behind Pi, has released Inflection-2.5, an upgraded in-house model that they say is competitive with leading models like GPT-4 and Gemini. This new model combines raw capability with Inflection's signature personality and unique empathetic fine-tuning, making it a significant step forward in personal AI technology. Pi is a unique take on chatbots—Inflection calls it a personal AI designed to be not only empathetic and helpful but also remarkably intelligent.
The company says that Inflection-2.5 was developed with remarkable efficiency, approaching GPT-4’s performance while using only 40% of the compute for training. The model has made significant strides in areas of IQ like coding and mathematics, translating into concrete improvements on key industry benchmarks. Pi also now incorporates world-class real-time web search capabilities, ensuring users receive high-quality breaking news and up-to-date information.
The new model has already been rolled out to users, with positive feedback. User sentiment, engagement, and retention have significantly improved, accelerating organic user growth. Inflection's one million daily active users have exchanged over four billion messages with Pi, with an average conversation lasting 33 minutes. One in ten conversations lasts over an hour each day, and 60% of people who talk to Pi in a given week return the following week.
With Inflection-2.5’s powerful capabilities, users are engaging with Pi on a wider range of topics than ever before. From discussing current events and getting local restaurant recommendations to studying for exams, drafting business plans, coding, preparing for important conversations, or simply having fun discussing a hobby, Pi is proving to be a versatile and valuable tool for users.
Technical results show that Inflection-2.5 performs at more than 94% the average performance of GPT-4, despite using only 40% the training FLOPs. The model shows substantial gains over Inflection-1 on the MMLU benchmark, a diverse benchmark measuring performance across a wide range of tasks from high school to professional-level difficulty. Inflection-2.5 also performs well on the GPQA Diamond benchmark, an extremely difficult expert-level benchmark, and on two different STEM examinations: the Hungarian Math exam and the Physics GRE.
In an exclusive interview with Axios, Inflection CEO Mustafa Suleyman expressed his satisfaction with the new model's performance, stating that Inflection 2.5 is now "neck and neck with GPT-4 for quality." Suleyman also shared user metrics for the first time, revealing that Inflection has 1 million daily active users and 6 million monthly active users, who have exchanged more than 4 billion messages with Pi.
Pi is not multimodal and can only accept text as an input. Additionally, while nothing has been mentioned about the size of the model's context window, Inflection has limited inputs to 4000 characters.
Inflection's focus on creating a highly personal chatbot with a warm and friendly tone sets it apart from other AI companies. While Pi is currently free, Inflection's business model plans to generate revenue from its users, starting with a paid subscription. The company is also considering charging based on progress, such as when a user reaches a particular goal they had set out inside of Pi.
Despite its commendable progress so far, Inflection still has a lot of work to truly differentiate Pi as a standout offering. As it continues to advance Pi's capabilities and expand its user base, the company's 70-person team remains focused on fundraising to support its ongoing work.