Analytics validation playbooks
Step-by-step playbooks for the analytics work a team running coding agents has to do. Each playbook covers problem context, what breaks, a system approach, execution steps, metrics, and failure modes.
Playbooks
Start with the audit. Then pick whichever covers your current situation.
Audit your current event taxonomy
Before you can stop instrumentation from drifting, you need to know what you have. A one-time audit produces a manifest that every later decision can sit on top of.
Job-to-be-done: Get a written, accurate snapshot of every analytics call your repo emits today.
Open playbook →Set up Skene for a PostHog codebase
Step-by-step for repos that use PostHog. Covers the MCP install, the GitHub Action, and how to baseline the existing surface.
Job-to-be-done: Wire Skene up against an existing PostHog instrumentation surface so future PRs get validated.
Open playbook →Fix dashboards that have already drifted
Recovery playbook. When you discover the funnel has been wrong for weeks, here is the order of operations to get back to honest data.
Job-to-be-done: Recover from a known instrumentation drift incident: the dashboard is wrong, find out why and stop it from happening again.
Open playbook →Validate analytics in CI as part of code review
Treat instrumentation drift like any other code regression: caught on the PR, fixed before merge, reviewed by humans on the same surface they review everything else.
Job-to-be-done: Make analytics drift a blocking signal on PRs, the same way type errors and failed tests are.
Open playbook →Catch instrumentation breakage from coding agents
When the agent is writing the code, the agent is the first reviewer. The earlier Skene's signal reaches the agent, the cheaper the fix.
Job-to-be-done: Make sure your coding agent (Cursor, Claude Code, Codex, Devin) finds out about analytics drift before it commits.
Open playbook →Migrate from one analytics tool to another without losing fidelity
Migrating Segment to PostHog, Mixpanel to Amplitude, or any pair. The hard part is not the SDK swap; it is not silently losing or renaming events along the way.
Job-to-be-done: Move from one analytics backend (or CDP) to another while preserving the event taxonomy you already trust.
Open playbook →