Isaac Kargar once found about 1,000 records in a dataset missing information his agent needed. The fix he settled on was not a better final check. It was checking every step before the agent was allowed to take the next one.
The short answer: Trust belongs to the harness that decides what to extract, keep and forget, and the dangerous failure is an agent's decision that nobody verified.
What Isaac Kargar trusts, and for what
| Evidence | Trusted for | Not trusted for |
|---|---|---|
| Raw chat logs and agent traces | Nothing, until something filters them | Decisions, as they stand |
| A context layer that extracts decisions and signals | The part worth keeping | Remembering everything, which degrades it |
| A simple chain of small model calls | Predictable results, cheaper models | Open-ended autonomy |
| Verifying each step | Catching choices an agent made alone | Speed |
Traces are not data until something filters them
Chat logs, agent traces and team chat history are noise until a context layer pulls out the decisions and signals worth keeping. To Isaac, that filtering harness is the system. The same idea runs his personal news channel: an agent pulls links, summarises them and stores everything in a local SQL database.
Knowing what to forget
Building memory systems taught him that storing too much hurts as much as storing too little. His answer is a hierarchy: recent context close at hand, mid-term facts merged periodically, rarely used items dropped, and some information parked in a database and fetched only on demand.
The hardest part, especially for memory, was finding the balance
Verify the step, not just the result
An agent left alone on an ambiguous task makes choices about data and scope that nobody checked. His discipline is to tell the agent what matters and to break the work down so each step can be verified. He also found that a plain chain of predictable model calls beat the fully autonomous agent he built first: easier to build, cheaper to run and friendly to small models.
So I started breaking the steps down: the agent does the step, but I verify the result before continuing.
What to do with this
Every pipeline that writes data your team will read has intermediate choices in it. List them, and put a check after the ones where a wrong choice would still produce output that looks fine.
For seven checks drawn from across the series, see the questions to ask before trusting a number.
Read next in The Question
Source and disclosure
Read Issue 9 on A Founder's Friend. Quotations on this page are copied from the published issue.
Skene sponsored this research and commissioned it on a single question. A Founder's Friend ran the interviews, wrote them up and published them, and the series states that Skene paid for the research and did not direct the questions, the analysis or the conclusions. Each of the thirteen interviews was reviewed and signed off by the person in it, named or anonymous by their choice. Appearing in the series is not an endorsement of Skene, and nobody here is presented as a Skene customer.