Meta’s Maverick AI Falls Short in LM Arena Benchmark Rankings

After drawing criticism for submitting an unreleased experimental version of its Llama 4 Maverick model to LM Arena, Meta has now seen its officially released version ranked—and it falls well behind top models from OpenAI, Anthropic, and Google.

The reassessment followed Meta’s earlier submission of Llama-4-Maverick-03-26-Experimental, a version specially tuned for chat performance. That optimization initially gave the model an advantage on LM Arena, which relies on human raters to evaluate model responses.

Following pushback, LM Arena moderators issued an apology, updated their submission policies, and rescored the model using the unmodified, publicly available version: Llama-4-Maverick-17B-128E-Instruct.

Performance Falls Short

The result? The unmodified Maverick model was outperformed by several older rivals, including GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro.

Meta said the experimental variant had been “optimized for conversationality,” an advantage in LM Arena’s human-comparison format. But experts caution that optimizing for a single benchmark can reduce reliability across diverse applications—and beyond being potentially misleading, it makes it harder for developers to gauge a model’s true versatility.

Meta’s Response

In a statement to TechCrunch, a Meta spokesperson said the company often explores multiple model variants:

“‘Llama-4-Maverick-03-26-Experimental’ is a chat optimized version we experimented with that also performs well on LM Arena. We have now released our open source version and will see how developers customize Llama 4 for their own use cases. We’re excited to see what they will build and look forward to their ongoing feedback.”

In a previous statement, Meta denied allegations of cheating, insisting that its benchmark submission reflected normal experimentation practices.

What This Means

The rankings reveal the difficulty of relying on narrow benchmarks to evaluate complex AI models—especially when experimental tweaks can skew the results. While Meta's openness about experimentation may foster innovation, the episode highlights the need for transparency and consistent evaluation methods.

If Maverick is to prove its worth beyond benchmark charts, it will depend on how developers respond—and how the model performs when tested in the wild, where no leaderboard can offer shortcuts.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.