For AmirBahador Bahadori, product data is not one thing to trust or distrust. The database, the observability stack and the behavioural analytics overlap in tooling. To him they answer three different questions, and mixing them up is the mistake.
The short answer: Product data is three separate layers that answer different questions, and a dashboard pattern is a hypothesis until a person confirms it.
What AmirBahador Bahadori trusts, and for what
| Evidence | Trusted for | Not trusted for |
|---|---|---|
| Database state | The current state of a thing, judged on correctness | What users want |
| Observability: traces and logs | Fixing problems, including questions nobody planned for | Explaining behaviour |
| Behavioural analytics | Patterns worth testing | Cause, especially in B2C |
| Talking to the customer | The reason behind a B2B account's behaviour | Covering every user at a large customer |
Three layers, three questions
The database records the state of things and is judged on whether it is correct. Observability exists to fix problems, one request at a time. Analytics describes who did what. People use one for another all the time, and he keeps them deliberately apart so each is judged on its own terms.
In my head, those three are different concepts answering different questions.
The test is the question you did not plan for
A dashboard that only answers the queries it was built for does not meet his bar. A trustworthy observability system lets him ask an open-ended question nobody designed it for and still rely on the answer.
That is why he keeps as much useful telemetry as security and GDPR rules allow, anonymised where needed and moved to cheaper storage as it ages. The business will eventually ask about a metric that did not seem to matter when it was recorded.
A proper observability system is one where I can ask an open-ended question I didn’t build the tool for
A dashboard shows a hypothesis
Behavioural data shows you something without telling you why. A churned consumer might be tired of the app or might have hit a bad translation. In B2C he leans on aggregation and never treats one number as the reason.
In B2B the customer is usually reachable, and a conversation beats a pattern on a chart. Smartlane itself provides analytics to its customers through Metabase, so this is not a team that distrusts dashboards. It is a team that knows which questions they answer.
In a dashboard you can never really know what it is.
The machine never holds the wheel
He is happy for AI to review code, read logs and open tickets. He would not let it write, submit and deploy on its own. Fully automated loops are fine for low-stakes internal work; anything important needs a named person who owns the outcome, because an apology from a model is not accountability.
AI should never be the driving person. It should not have the wheel. The final decision should be kept in your hand.
What to do with this
Before a number goes into a plan, label which layer it came from. A database count, a traced error rate and a behavioural funnel can share a dashboard and still deserve very different amounts of trust.
For seven checks drawn from across the series, see the questions to ask before trusting a number.
Frameworks this interview tests
- Customer health score: A customer health score blends usage, support and sentiment to predict renewal risk. Usage carries the most weight, and nothing in the score checks the usage events.
- Jobs to be done (JTBD): Clayton Christensen's jobs to be done says customers hire a product to make progress. The job decides which events matter, so interviews come before the tracking plan.
Read next in The Question
Source and disclosure
Read Issue 2 on A Founder's Friend. Quotations on this page are copied from the published issue.
Skene sponsored this research and commissioned it on a single question. A Founder's Friend ran the interviews, wrote them up and published them, and the series states that Skene paid for the research and did not direct the questions, the analysis or the conclusions. Each of the thirteen interviews was reviewed and signed off by the person in it, named or anonymous by their choice. Appearing in the series is not an endorsement of Skene, and nobody here is presented as a Skene customer.