Here is the thing nobody selling you a “bigger context window” wants you to hear: past a certain point, feeding your AI more data makes it worse, not better. Not slower. Worse at the actual thinking.
That is the uncomfortable finding at the centre of a sharp new piece by Angus Sewell, who writes Angus the Nontechnical. In “I Analyzed 12,130 Records To Build an AI-Native CRM” he points an AI agent at 11,000 people and 12,130 interactions, roughly 5.2 million tokens of history, and watches it fall over. His fix was not a bigger brain. It was a smaller plate. Credit where it is due, it is a genuinely useful build and worth reading in full.
The research is worse than the vendors admit
Quick definitions first, because the marketing relies on you not knowing them. A “token” is roughly three-quarters of a word. The “context window” is how much text you can hand a model at once. The pitch is that a bigger window is a better assistant. The evidence says a bigger window is a trap you walk into.
Chroma’s “context rot” study tested 18 frontier models and found every single one degrades as the input grows, sometimes shedding 30 to 50 per cent of its accuracy well before it hits the advertised limit. A model sold with a 200,000-token window can start wobbling at 50,000. Separately, the NoLiMa benchmark from Adobe’s research team clocked GPT-4o at 99.3 per cent accuracy under 1,000 tokens, collapsing to 69.7 per cent at 32,000. Sewell is not exaggerating for effect. The floor really does fall away.
His answer: three resolutions, read top-down
Sewell stores the same data at three levels of detail. A rolled-up view with one row per person, short factual summaries of each interaction, and the untouched raw records underneath. The agent is instructed to read from the top and never crack open a raw transcript first. It is a clever design, and it maps onto how a good salesperson actually remembers a client: the headline, then the detail, then the receipts only if pushed.
Where I would push back
The three-layer build is elegant if you are the sort of person who runs a Postgres database for fun. Most businesses reading this are not. Sewell is honest that his “CRM” is a second database sitting beside the real one, which means somebody now owns another system: keeping the summaries current, re-running them when records change, and paying each time. He mentions summarising the full set cost about thirty dollars once. “Once” is doing a lot of quiet work in that sentence. Your data is not a photograph, it is a river, and every new meeting nudges the rollup out of date.
The deeper risk is the summary layer itself. Sewell nails it when he calls a summary of a summary “fiction with good manners.” The three-layer approach reduces that risk but does not remove it. Every compression step is a decision about what does not matter, taken by a model that cannot know which throwaway line in a transcript is the one that closes the deal. Get a summary wrong and the agent never finds out, because you told it not to look underneath. You have swapped a model that drowns for one that is confidently, tidily incomplete.
The part that should annoy you
Sewell’s audit of the big CRMs is the real story here. Attio’s summariser can read your attribute values but not your notes or emails. HubSpot hands you a call-summary field but not the transcript. Close gives you both. Only Salesforce could, in theory, build all three layers. So you are paying enterprise prices for tools that will not let you control the one thing that now matters most: how your own data gets compressed before your AI ever reads it. That granularity is not a feature they forgot. It is a feature they have not decided to sell you yet.
What to actually do on a Tuesday
If you have an engineer, Sewell’s blueprint is worth copying. If you do not, the honest move is smaller. Stop dumping your entire CRM export into a chatbot and expecting brilliance. Feed it the one account you are working, and treat any AI summary of a long history as a lead to check rather than a fact to trust. The bigger-window advertising will keep coming. You can safely ignore most of it.
Did you know: the line everyone repeats, that an AI starts struggling once you fill about 40 per cent of its context window, appears to trace back to a conference talk rather than a study. Sewell flags it, and the peer-reviewed work suggests the real drop-off is messier, model-specific, and often earlier than any tidy percentage.
Sources
- Angus Sewell, “I Analyzed 12,130 Records To Build an AI-Native CRM”, Angus the Nontechnical.
- Chroma Research, “Context Rot: How Increasing Input Tokens Impacts LLM Performance”.
- Modarressi et al., “NoLiMa: Long-Context Evaluation Beyond Literal Matching”, Adobe Research.