Five LLMs Compared for Production AI: GPT-4.1, Claude Sonnet 4, Gemini 2.5 Flash, GPT-5.2 & Claude Opus 4.6

"Which model should we use?" It's one of the most common questions I get, and the most honestly answered by: it depends on what you're actually doing. The uncomfortable follow-up is that most people asking the question are comparing models on benchmark scores, which are about as useful for production decisions as judging a car on its top speed.

MMLU, HumanEval, MATH — the standard AI benchmarks — are valid scientific measurements. They're just measuring something different from what you need when you're deploying an agent that processes supplier invoices at 3am, a support bot handling an angry customer's billing dispute in three languages, or an extraction pipeline pulling specific fields from 40 different PDF formats reliably.

This article is about what we've found from actual production deployments. We compare five models across two tiers: the everyday workhorses — GPT-4.1, Claude Sonnet 4, and Gemini 2.5 Flash — run at volume in document pipelines, support agents, and extraction systems; and the frontier heavyweights — GPT-5.2 and Claude Opus 4.6 — for cases that genuinely need maximum capability. The comparisons come from projects where we had real data: domain accuracy, per-call cost at volume, latency under load, and failure modes on edge cases.

Why benchmark scores mislead production decisions

Benchmarks test the model in isolation on clean, standard problems. Production systems have noise. Here are the differences that reliably matter more than overall benchmark scores:

• Instruction following consistency. How reliably does the model return structured output across thousands of calls? Getting the format right 99.5% of the time vs 97% is the difference between 5 errors per 1,000 calls and 30 — a massive downstream impact on anything that has to parse the output.
• Graceful degradation on edge cases. What happens when the input is ambiguous, poorly formatted, or genuinely unclear? Does the model flag uncertainty, make a best-effort attempt, or confabulate confidently? Confident wrong answers are worse than honest "I'm not sure" responses in most production contexts.
• Cost at your volume. At 10,000 calls/day, a £1 per million token price difference isn't academic. Context window usage, output length control, and whether you can batch effectively all matter to the actual cost.
• Latency under load. Average latency is almost useless. You want p95 and p99 — real-world worst-case latency. How long is the slowest 1% of users in your support workflow actually waiting?
• API reliability. All major providers have had outages. They differ significantly in how quickly they resolve incidents, what their rate limits are at various pricing tiers, and whether there’s a fallback path (Azure OpenAI, Bedrock, Vertex AI, etc.) when the primary endpoint is degraded.

The comparison: at a glance

Prices are as of March 2026 and shift often — always check the current provider pricing pages for your exact use case. The first three columns are production mid-tier models; GPT-5.2 and Claude Opus 4.6 are the current frontier tier from their respective labs.

Document processing pipelines

This is the use case that shows up most often in our work: extracting structured data from PDFs, contracts, invoices, and forms. The inputs are messy — scanned documents, mixed formats, tables with irregular layouts, handwritten annotations alongside typed text.

Mid-tier production models

When to step up to the frontier tier

Customer support agent systems

This is multi-turn: the agent needs to understand context from conversation history, reference product documentation, handle queries in multiple languages, and decide when to escalate to a human without being trigger-happy about it.

Mid-tier production models

When to step up to the frontier tier

Structured data extraction at scale

Pure extraction — pulling specific fields from text, normalising them, and outputting a clean JSON structure. The kind of thing that runs millions of times and needs high reliability and low cost. This use case is where the cost differences between models hit hardest in practice.

The tiered routing approach we’ve landed on for several clients: use Gemini 2.5 Flash as the first pass at scale, route low-confidence outputs (where the model flags uncertainty or downstream validation fails) to GPT-4.1 for a second pass, and reserve Claude Opus 4.6 or GPT-5.2 only for the small slice where GPT-4.1 also struggles. In practice that means paying Gemini prices on ~88% of volume, GPT-4.1 prices on ~10%, and frontier prices on ~2%. Total cost is close to Gemini cost. Accuracy approaches frontier quality. You have to build the routing logic, but it pays off at volume.

Important note: Gemini 2.0 Flash is now deprecated by Google. If you’re running anything on 2.0 Flash in production, migrate to Gemini 2.5 Flash — it’s a meaningfully better model at a still-competitive price.

The honest recommendation

There is no universally best model for production. Here’s a clean decision framework across all five models:

• Gemini 2.5 Flash ($0.30/$2.50 per 1M) — high volume, cost-sensitive tasks. The pick for millions of calls on a well-defined task, with a fallback route to GPT-4.1 for low-confidence outputs. Skip the deprecated 2.0 Flash, and run standard mode (no thinking budgets) on latency-critical paths like live chat.
• GPT-4.1 ($2.00/$8.00 per 1M) — the reliable all-rounder default. The best starting point for most workloads: mature tooling, parallel function calling, 1M context, Azure fallback. Use it unless you have a specific reason to deviate.
• Claude Sonnet 4 ($3.00/$15.00 per 1M) — when quality and tone matter more than cost. Step up from GPT-4.1 when document reasoning or customer-facing tone is the bottleneck. The 15–20% output premium is usually justified on complex analytical work.
• GPT-5.2 ($1.75/$14.00 per 1M) — frontier capability at the lowest frontier price. Its input cost is actually below GPT-4.1. Reach for it when mid-tier models hit a quality ceiling on hard or ambiguous inputs; the $14/1M output cost is where the premium bites.
• Claude Opus 4.6 ($5.00/$25.00 per 1M) — the frontier ceiling for the hardest problems. Highest cost, highest sustained reasoning quality. Best for long-horizon agents, adversarial document review, and high-stakes automation where errors are expensive.

The thing that matters more than model choice

I'd be doing you a disservice if I finished this article without saying: model selection is typically the fifth or sixth most important decision in a production AI system. Far ahead of it are: prompt design and how you handle context, data quality and pipeline reliability, error handling and fallback behaviour, evaluation methodology (how you actually know the system is working), and deployment infrastructure.

A well-designed system using GPT-4.1 will consistently outperform a poorly-designed system using whichever model wins the latest benchmark. The models are close enough in capability on most production tasks that system design is the differentiator. The wrong model choice costs you maybe 10–20% on a given metric. Poor system design costs you the entire thing.

That said: model costs are real, model quality differences are real, and choosing deliberately rather than defaulting to whatever is most familiar is worth the hour it takes to think it through.