Skip to content

Reviews

We run AI tools on real tasks and report the numbers: what it cost, how long it took, and whether we'd use it. Newest first.

All reviews

49 reviews

ChatGPT

GPT-6 Astra vs Terra: Same Contract Answers, 10x the Price

We gave GPT-6 Astra, Claude Fable 5.1, GPT-6.1 Terra and Claude Opus 5.5 a 15,000-word contract with four amendments and ten trap questions. All four got every trap right in every run; cost per read ranged from $0.02 (Terra) to $0.38 (Fable 5.1).

Claude

Claude Long Chat vs New Chat: Does It Get Lost? (Tested)

We hid three corrections in a 40-message office chat about a fake sales table and asked Claude Haiku 5.5 and Sonnet 5.5 five questions there, in a short chat and in a new chat built from a summary. All scored 5/5; the new chat carried 5–7 times less history per message.

Claude

Claude Code Cost and Limits: 7 Days of Real Usage Measured

We added up every Claude Code request one machine made in a week: 2,818 requests that would have cost $149.44 at API prices, 60% of it writing to the prompt cache. Here are the official plan prices and limits next to those numbers, plus seven tests on spending less.

Claude

Anthropic Skills on GitHub: 19 Official Skills Measured, 5 Run for Real

We measured all 19 skills in Anthropic's anthropics/skills repository with a script and put the numbers next to our five hands-on Claude Code tests. Descriptions decide whether a skill loads: the only one we tested that never says when to use it stayed unused until we named it.

DuckDB

DuckDB 2.0 on a CSV too big for Excel: 3 million rows queried in 0.2 seconds

We ran DuckDB 2.0.0 on the day after release against a synthetic 3,000,000-row, 123 MB CSV, almost three times Excel's 1,048,576-row limit. A GROUP BY took 0.17 to 0.22 seconds, against 4.04 seconds for a plain Python csv loop that returned the same totals to the cent. One 2.0 API change broke our script.

Mistral

Voxtral 3 vs Whisper, tested: a tie in English, about 4× fewer errors in Vietnamese

We scored Mistral's Voxtral Mini Transcribe 3, whisper-1 and gpt-4o-transcribe against the script of two narrations. English: 1.6%, 1.7% and 2.1% word error rate. Vietnamese: 1.7% against 6.8% and 6.6%, with 37 of 39 listed names and terms right against 16 and 21. Voxtral also lists at USD 0.0035 a minute against 0.006.