Claude Long Chat vs New Chat: Does It Get Lost? (Tested)

Does a long AI chat really "get lost", or is starting a new chat just habit? We gave Claude a fake 36-row sales table, slipped three corrections into a 40-message office chat full of side requests, then asked five questions whose answers depend on those corrections. We asked the same questions in a short chat and in a fresh chat that only got Claude's own summary. All three got 5 out of 5, on both models we tried. What changed was how much the AI had to re-read for each new message.

Measured on 27 Oct 2026 with Claude Haiku 5.5 and Claude Sonnet 5.5 through the Anthropic API, from a Python script on a Windows 11 PC (WSL) in Vietnam. The data is invented; the script and raw answers are kept with this article's source files.

At a glance

Key results

• No "lost" answers at 40 messages. Haiku 5.5 and Sonnet 5.5, two runs each, answered all five questions correctly in the long chat: 20 of 20 per model. The three corrections (one changed number, one "exclude rep Lan from now on" rule, one changed price) were all applied.
• The summary hand-off worked too. When we asked for "a summary I can paste in a new chat", every summary carried all 36 rows and all three corrections, and the fresh chat again scored 5/5 in every run.
• The new chat was 5–7 times lighter. Each question in the 40-message chat carried about 7,300–8,100 tokens of history on Haiku and 12,000–14,400 on Sonnet; in the fresh chat it was about 1,300 (Haiku) and 2,100 (Sonnet).
• Why that matters: Claude's Messages API is stateless, so the whole conversation is sent again with every new message. A heavier chat means more to re-read each time, even when the answers stay right.
• What we did not test: chats with hundreds of messages, long PDFs or images, or the ChatGPT and Claude apps themselves, which may trim or compress old messages on their own.

Hands-on test

Setup

• Models: claude-haiku-5-5 and claude-sonnet-5-5, default settings, two runs each.
• Data: a CSV of 36 invented sales rows (id, region, rep, month, units, price), pasted as the first message.
• Short chat: table, then the three corrections, then the questions (5 messages).
• Long chat: table, then 36 everyday requests (draft an email to the manager, explain XLOOKUP, suggest slide titles, a COUNTIF vs COUNTIFS question…), with the three corrections placed at messages 8, 19 and 30.
• Hand-off: at the end of the long chat we asked for a summary to paste into a new chat, then opened a new conversation with only that summary.
• Questions: total units, South revenue, top rep by revenue, number of rows above 50 units, average March price. Each question was asked separately from the same history and graded by code against answers computed from the corrected table.

What "lost" usually means

The idea that long chats go wrong is not made up. Drew Breunig's write-up of his talk on context engineering names four failure modes: a bad fact the model can't get past (poisoning), a context so long the model leans on it over what it knows (distraction), unrelated material that sways the answer (confusion), and pieces of the context that start to conflict (clash). Our test aimed at the office version of that: a number corrected halfway, a rule given once, and dozens of unrelated requests in between.

At this size it did not happen. Forty messages of office chatter came to well under 15,000 tokens, small next to the context windows of current models, so there was little to get lost in.

When to start a new chat anyway

Accuracy was not the reason to switch in our test; weight was. Use a new chat when:

1. The topic changes. A fresh chat for a new report keeps the old one's numbers from leaking in.
2. The chat is long and you keep going. Every new message re-reads everything above it. Moving to a summary cut that from about 12,000 to about 2,000 tokens per question on Sonnet.
3. You uploaded big files you no longer need. We did not test files, but they add to what is re-read each time.

How to carry a chat over

The prompt we used, word for word: "I'm going to start a new chat. Write a summary I can paste there so you can continue this report exactly: include the data and every correction or rule I gave."

Then check the summary before pasting it. In our runs it held the corrected table and a numbered list of the rules. If a correction is missing from the summary, the new chat will not know it.

What worked

What worked

• Saying "from now on" for a rule. Both models kept the "exclude Lan" rule for 20+ messages.
• Asking for the summary to include the data and every correction. All four summaries kept every row.

Where it fell short

What didn't work

• Our first run returned empty answers: with a very small reply limit, the models used it all on internal reasoning and printed nothing. With a normal limit the problem went away. This is an API setting; app users will not see it.
• The test cannot show where long chats start to fail. Forty messages was not enough, and we did not push to hundreds.

Our verdict

For a normal working session of a few dozen messages, you probably don't need to restart the chat to keep the answers right: in our test, Claude kept corrections and rules for 40 messages. Start a new chat when the topic changes or the chat has grown heavy, and carry it over with a summary that includes the data and every correction. It was just as accurate and five to seven times lighter.

How this was made: an AI (Claude) helped write this article; the RanThis editor ran the test script, checked every number against the raw results and checked facts against the linked sources.