Benchmarks & Milestones

Features a 1-million-token context window and introduces agent teams — groups of agents that can split larger tasks into segmented jobs. Outperformed GPT-5.2 on several benchmarks.

State-of-the-art on SWE-bench Verified and OSWorld (61.4%). Can work autonomously for 30+ hours (up from 7 hours with Opus 4). Launched alongside the Claude Agent SDK.

Claude Sonnet 4 delivers superior coding and reasoning as a hybrid model. Claude Opus 4 is declared the world's best coding model, leading SWE-bench (72.5%) and Terminal-bench (43.2%).

OpenAI releases o3, its most capable reasoning model, making 20% fewer major errors than o1. The first reasoning model that can agentically use all ChatGPT tools (web search, Python, image generation) simultaneously.

First Llama models with native multimodality and mixture-of-experts architecture. Scout supports a 10M-token context window on a single H100. Maverick (128 experts) beats GPT-4o and Gemini 2.0 Flash.

Google's first 'thinking model' that reasons through steps before responding. Debuted at #1 on LMArena by a significant margin. Described as Google's most intelligent AI model.

The first hybrid reasoning model on the market, able to produce near-instant responses or engage in visible extended step-by-step thinking. Launched alongside Claude Code, an agentic coding CLI tool.

Trained with 10x the compute of Grok-2 on the 200,000-GPU Colossus supercluster. Features a 1M-token context window and large-scale reinforcement learning for reasoning.

OpenAI releases o3-mini, a cost-efficient reasoning model optimized for STEM. Excels in science, math, and coding, showing that reasoning capabilities can be delivered in lightweight form factors.

An open-source reasoning model rivaling OpenAI o1. Its mobile app briefly surpassed ChatGPT as #1 free app on the US App Store. The release caused an 18% drop in Nvidia's share price.

A 671B MoE model (37B active per token) trained for only ~$6 million. Competitive with GPT-4o and Claude 3.5 Sonnet on most benchmarks, demonstrating radical training cost efficiency.

Outperforms Gemini 1.5 Pro on key benchmarks at twice the speed. First Gemini with multimodal output (native image generation + text-to-speech). Positioned for the 'agentic era.'

A 70B text-only model delivering performance comparable to the much larger Llama 3.1 405B at a fraction of the serving cost, demonstrating significant efficiency gains through improved training.

Full release of o1 with 34% fewer major mistakes and 50% faster than the preview, now multimodal. Launched alongside the $200/month ChatGPT Pro subscription tier featuring o1-pro mode.

Claude 3.5 Haiku matches the performance of the prior Claude 3 Opus at the speed and cost of the smaller Haiku tier. Scored 40.6% on SWE-bench Verified, outperforming many agents using larger models.

Upgraded Claude 3.5 Sonnet with across-the-board improvements, especially in coding. Introduces computer use in public beta — the first frontier AI to control a computer (cursor, clicks, typing) via API.

Meta's first multimodal open models (11B and 90B vision LLMs) plus lightweight models (1B and 3B) for edge and mobile devices. Announced at Meta Connect 2024.

OpenAI launches o1-preview, its first model trained with reinforcement learning to think step-by-step before answering. Ranked 89th percentile on Codeforces and placed among top 500 US students on AIME.

xAI's first competitive frontier model, achieving performance on par with leading models on graduate-level science (GPQA), general knowledge (MMLU), and math. Includes image generation via Flux.

The first openly available frontier-level model at 405 billion parameters, rivaling top proprietary models in general knowledge, math, tool use, and multilingual translation. 128K context.

OpenAI releases GPT-4o mini, a smaller and significantly cheaper variant that outperforms GPT-4 Turbo on many benchmarks while costing a fraction to run.

Claude 3.5 Sonnet outperforms Claude 3 Opus on key benchmarks at 2x the speed and 1/5 the cost. Introduces Artifacts, an interactive workspace for AI-generated content alongside conversations.

A speed-optimized Gemini model for high-frequency tasks where latency matters most, while still supporting 1 million tokens of multimodal context (text, images, audio, video).

OpenAI releases GPT-4o ('omni'), which natively processes and generates text, audio, and images. 2x faster and 50% cheaper than GPT-4 Turbo. Made free to all ChatGPT users.

A MoE model with 236B total parameters but only 21B active, pioneering efficient inference with Multi-head Latent Attention. Dramatically undercut competitor pricing.

The most capable openly available LLM at launch, with significantly improved reasoning, coding, and instruction-following over Llama 2. Available in 8B and 70B parameter sizes.

Anthropic launches three models: Opus (most intelligent), Sonnet (balanced), and Haiku (fastest). Claude 3 Opus outperforms GPT-4 on most benchmarks. Anthropic's first model with vision capabilities.

Mistral's first true flagship model competing with GPT-4. Features 32K context, specialized RAG support, and best-in-class multilingual performance across French, German, Spanish, and Italian.

Gemini 1.5 Pro achieves a breakthrough 1-million-token context window, capable of processing 1 hour of video, 11 hours of audio, or 700K+ words in a single prompt.

A pioneering open-weight sparse mixture-of-experts model that outperforms Llama 2 70B on most benchmarks with 6x faster inference. Released under the permissive Apache 2.0 license.

Google DeepMind releases Gemini 1.0 in Ultra, Pro, and Nano sizes. The Ultra model is the first to claim human-expert performance on MMLU (90.0%). Natively multimodal from training.

OpenAI announces GPT-4 Turbo at its first DevDay conference, featuring a 128K context window, JSON mode, and significantly lower pricing — 3x cheaper input, 2x cheaper output vs GPT-4.

Meta releases Llama 2 in partnership with Microsoft — one of the first major open-weight foundation models for commercial use. Available in 7B, 13B, and 70B parameter sizes.

Anthropic releases Claude 2 with a 100K token context window, allowing users to process entire books in a single conversation. The first Anthropic model available to the general public via claude.ai.

OpenAI launches GPT-4, a large multimodal model that accepts image and text inputs. It scores in the 90th percentile on the bar exam and demonstrates significant reasoning improvements over GPT-3.5.

Anthropic launches its first public AI assistant, positioned as helpful, honest, and harmless. Available in Claude and Claude Instant variants, initially via API only.

OpenAI releases ChatGPT, a conversational AI assistant based on GPT-3.5. It becomes the fastest-growing consumer app in history, reaching 100 million users in two months.