How we test
Every hands-on guide on RanThis starts with a real run. We install the tool, give it a task with an answer we can check, and write down what happened, including what went wrong.
We install it and run it
- We install the tool, Claude skill or MCP server the way its documentation describes, on our own machine and accounts. Most tests so far ran in Claude Code in a Linux terminal, or in the web app in Chrome.
- Each test is one concrete task: merging four Excel files with mismatched columns, turning three PDF invoices into a register, building a dashboard from a CSV file.
- We use made-up sample data (fictional companies, invoices and sales figures) and say so in the article.
- Many of our tests are run in Vietnamese, the working language of the people behind RanThis. When that's the case, the article says so and translates the prompts; the screenshots stay as they were.
We record the run
- We capture the real session as it happens: the terminal, the app window and the file that comes out. Many tests are also screen-recorded.
- Our hands-on guides state the test date, the tool version, the plan and the settings, for example Claude Code 2.1.301 on a Claude Max plan in auto mode.
- Times are measured from sending the prompt to a finished result.
We check the output
- When the result is a number, we recalculate it ourselves: row counts and totals against the source files, dashboard figures recomputed in Python.
- When we compare models, the tasks have answers worked out before the run, and a script grades every reply item by item: valid JSON, each extracted field, each figure, each condition of a written reply. The grader is strict, so a correct answer in the wrong format loses the point, and the article explains when that happens.
- We report the misses as well as the wins: an incomplete warning, a script that needed a retry, an instruction that only works in desktop Excel.
We measure cost on the real task
- For API tests, cost is the token counts the API reports for each request multiplied by the price on the vendor's official pricing page on the day of the test. We publish the cost of the whole task, not only the price per million tokens.
- For subscription tools, we name the plan we used. Plan limits come from the vendor's own pages and from what we saw on screen; we don't repeat numbers we can't find on an official page.
We keep private data out
- Tests run on sample files, not on real company or customer data.
- Before a screenshot is published, we hide account details such as organization names, email addresses and keys.
What hands-on and verified mean
Inside our publishing system, every guide carries origin labels. They are not shown as badges on the page yet; the article itself tells you what we ran.
- Hands-on: the guide is built on a test we ran ourselves, with the screenshots and numbers from that run. Our automated writing pipeline can't set this label.
- Verified: hard facts such as prices, plans, versions, dates and limits were checked against the vendor's official pages, which are listed under the article.
- A guide that only summarizes other sources can't be published without at least one listed source, and ranks below hands-on guides in our own scoring.
What a single test can't tell you
- One run is an observation, not a benchmark. Most setups ran once or twice; a rerun can word things differently, use more tokens or take longer.
- AI tools change fast. Claude Code shipped several versions during our October 2026 tests alone, so a result can change with the next update. Check the date and version in each guide.
- Your plan, operating system, settings, language and files can change the outcome. A test on our machine is not a promise about yours.
- We test short tasks with clear answers. Long documents, large codebases and multi-step agent work need their own tests, and our guides point out what they didn't cover.
- Prices and plans change often. Check the vendor's page before you pay.
See it in practice
- Claude Opus 5.5 vs Gemini 4 Argon: three tasks graded by code, with the cost of each task from the API's token counts.
- Merging mismatched Excel files with Claude Code: totals checked against the source files, and the warning Claude got wrong.
- Claude Dashboards from a CSV: every figure recalculated in Python, including after the data changed.
Got a different result?
Tell us your tool version, plan and operating system on the Contact page. If we can reproduce a mistake, we correct it.