Anthropic Unveils Claude 3.7 Sonnet

Anthropic unveiled Claude 3.7 Sonnet this week, its newest AI model that consolidates all its capabilities instead of separating them into specialized versions.

This release signifies a major shift in the company’s model development philosophy, favoring a “do everything well” approach versus creating distinct models for different tasks, unlike OpenAI.

Incremental Update

This isn’t Claude 4.0; it’s an important but incremental update to version 3.5 Sonnet. The naming suggests that this October release may have been internally viewed as Claude 3.6, yet Anthropic has not confirmed that publicly.

Early testers have been impressed with Claude’s coding and agentic capabilities, with some tests demonstrating that this model surpasses other state-of-the-art language models in coding.

Pricing Structure

However, the pricing for Claude 3.7 Sonnet places it at a premium compared to alternatives. API access costs $3 per million input tokens and $15 per million output tokens—significantly higher than its competitors including Google, Microsoft, and OpenAI.

Capability vs. Features

While Claude 3.7 is a necessary update, it lacks features found in other models. For instance, it cannot browse the web, generate images, or match the research abilities of OpenAI, Grok, and Google Gemini.

Given the model’s capabilities, testing was performed across various scenarios to assess its performance in creative writing, political bias, math, coding, and more.

Creative Writing: The King is Back

Claude 3.7 Sonnet reclaimed the creative writing crown from Grok-3, which held it briefly. During creative writing tests, Claude 3.7 produced narratives rich in human-like language and structure compared to competitors.

The difference, while slight, was enough to give Claude 3.7 an edge overall, although it struggled with delivering cohesive endings. Interestingly, turning on Claude’s extended thinking feature hampered performance, producing outputs reminiscent of older models like GPT-3.5.

Summarization and Information Retrieval: It Summarizes Too Much

Claude 3.7 can effectively summarize lengthy documents, outperforming 3.5 by providing concise, ultra-brief headlines and bullet points. However, this approach sacrifices detail for brevity. Grok-3 has its limitations but offers more thorough breakdowns.

Sensitive Topics: Claude Plays It Safest

Claude 3.7 maintains stringent content restrictions. It refrains from handling sensitive prompts that competitors address more freely, making it less flexible for creative writers exploring mature themes.

Political Bias: Better Balance, Lingering Biases

Claude 3.7 reflects an improved balance in presenting political topics but retains a subtle bias towards U.S. perspectives. It presents multiple viewpoints but centers on American narratives in political questions.

Coding: Claude Takes the Programming Crown

Claude 3.7 excels in coding, processing complex tasks with deep understanding. It performed better than competitors in benchmarks requiring adaptability within programming frameworks but still led to higher output costs.

Math: Claude’s Achilles’ Heel Persists

Despite improvements, math remains a challenge for Claude 3.7, with scores significantly lower than competitors like Grok-3. The model struggles with complex problems, often yielding incorrect solutions.

Non-Mathematical Reasoning: Claude is a Solid Performer

In reasoning tasks, Claude 3.7 shows its strength, especially in complex puzzles. The model outperformed competitors in speed and accuracy in these scenarios.

Overall, Claude 3.7 Sonnet represents a significant step for Anthropic, consolidating its capabilities while showcasing both its advantages and shortcomings across various tasks.