By OpenAI's own testing,german swinger game sex video its newest reasoning models, o3 and o4-mini, hallucinate significantly higher than o1.
First reported by TechCrunch, OpenAI's system card detailed the PersonQA evaluation results, designed to test for hallucinations. From the results of this evaluation, o3's hallucination rate is 33 percent, and o4-mini's hallucination rate is 48 percent — almost half of the time. By comparison, o1's hallucination rate is 16 percent, meaning o3 hallucinated about twice as often.
SEE ALSO: All the AI news of the week: ChatGPT debuts o3 and o4-mini, Gemini talks to dolphinsThe system card noted how o3 "tends to make more claims overall, leading to more accurate claims as well as more inaccurate/hallucinated claims." But OpenAI doesn't know the underlying cause, simply saying, "More research is needed to understand the cause of this result."
OpenAI's reasoning models are billed as more accurate than its non-reasoning models like GPT-4o and GPT-4.5 because they use more computation to "spend more time thinking before they respond," as described in the o1 announcement. Rather than largely relying on stochastic methods to provide an answer, the o-series models are trained to "refine their thinking process, try different strategies, and recognize their mistakes."
However, the system card for GPT-4.5, which was released in February, shows a 19 percent hallucination rate on the PersonQA evaluation. The same card also compares it to GPT-4o, which had a 30 percent hallucination rate.
In a statement to Mashable, an OpenAI spokesperson said, “Addressing hallucinations across all our models is an ongoing area of research, and we’re continually working to improve their accuracy and reliability.”
Evaluation benchmarks are tricky. They can be subjective, especially if developed in-house, and research has found flaws in their datasets and even how they evaluate models.
Plus, some rely on different benchmarks and methods to test accuracy and hallucinations. HuggingFace's hallucination benchmark evaluates models on the "occurrence of hallucinations in generated summaries" from around 1,000 public documents and found much lower hallucination rates across the board for major models on the market than OpenAI's evaluations. GPT-4o scored 1.5 percent, GPT-4.5 preview 1.2 percent, and o3-mini-high with reasoning scored 0.8 percent. It's worth noting o3 and o4-mini weren't included in the current leaderboard.
That's all to say; even industry standard benchmarks make it difficult to assess hallucination rates.
Then there's the added complexity that models tend to be more accurate when tapping into web search to source their answers. But in order to use ChatGPT search, OpenAI shares data with third-party search providers, and Enterprise customers using OpenAI models internally might not be willing to expose their prompts to that.
Regardless, if OpenAI is saying their brand-new o3 and o4-mini models hallucinate higher than their non-reasoning models, that might be a problem for its users.
UPDATE: Apr. 21, 2025, 1:16 p.m. EDT This story has been updated with a statement from OpenAI.
Pinkwashing the Timeline
The Ovid of Loserdom
Fresh Hell
Amazon Big Spring Sale 2025: Best gaming deals on PlayStation, Nintendo, more
JUMBO TEAM получила квоту на LAN
NOAA reveals most detailed satellite view of Northern Hemisphere
Amazon Big Spring Sale 2025: Best stick vacuum deals on Shark, Tineco, more
The world's oldest recorded spider built herself a hole, then just stayed there till age 43
Sakura Sunday
Blue Origin launches its first test flight of 2018
Mariye Receives DGA Nomination for ‘Just Add Magic’
Trump appointee Jim Bridenstine confirmed by Senate to lead NASA
A Show Trial that No One Watched
Best smartwatch deal: Save $40 on the Fitbit Versa 4
Getty Images Confuses 'Star Wars' Actress with Olympic Figure Skater
Astronomers are out looking for long
A 2016 heat wave may have forever changed the Great Barrier Reef
Watch harrowing footage from a couple's dangerous brush with a tornado
Lecture, Exhibit on Akita Dogs at Japan Foundation
Cradle to Grave
Matsui Statement on Trump Walking Away from COVID Relief TalksMatsui Statement on Trump Walking Away from COVID Relief TalksHernandez a NoTime Is Running Out to Complete CensusOni wa Soto! Fuku wa Uchi!‘Tokyo Godfathers’ Screenings on March 9, 11Time Is Running Out to Complete CensusNewsom Signs ToxicPacifica Plan to Remove ICF Draws Strenuous OppositionHawaii Five Here are the funniest reactions to Kendall Jenner's terrible 'woke' Pepsi ad This little boy reuniting with his doggie best friend will make you ugly cry Vladimir Putin HATES this meme so much Russia banned it Don't look down: These slacklining photos will give you full A personal thank you to the people who Snapchat their entire concert experience Mama alligator leads her 16 babies across a golf course, because Florida Little girl photobombed by huge venomous snake You should probably think twice about buying Girl Scout cookies on Amazon Racist posters targeting Chinese students stir controversy at the University of Texas at Austin Do you use dating apps at your desk? You're not alone.
0.2815s , 14255.9609375 kb
Copyright © 2025 Powered by 【german swinger game sex video】OpenAI's o3 and o4,Feature Flash
