Migrate from one analytics tool to another without losing fidelity

Problem context

You are moving from one analytics tool to another (cost, capabilities, or strategic reasons).

Years of dashboards, funnels, and warehouse models depend on the current event taxonomy.

A naive migration is a refactor of every call site in the codebase, multiplied by the number of events.

What breaks if this is not solved

• Events get silently renamed during the SDK swap because the new SDK's docs use different example names.
• Payload property keys diverge between old and new tools, breaking historical queries.
• Some events get dropped because the migrating developer did not know they were live in production.

When this playbook applies

• You have a non-trivial number of analytics calls (more than ~30).
• You have at least some dashboards or downstream models you are not willing to rebuild from scratch.
• You have a window to run both tools in parallel (even briefly).

System approach

Baseline the old surface in detail before touching anything. The migration target is to reproduce this exact shape in the new tool.

Run both tools in parallel during the migration window. Compare event counts daily.

Use the baseline manifest as the spec for the new tool's instrumentation. If the new tool's call shape cannot match, decide explicitly what to drop or rename. Do not let the migration decide for you.

Execution steps

1. Audit the existing instrumentation: `npx @skene/cli audit` against the current state. Commit the manifest.
2. Generate a list of every event name and payload shape that needs to exist in the new tool. The manifest is your spec.
3. Build a parallel instrumentation: dual-fire to old and new tools at the same time. Keep both running.
4. Compare event volumes between old and new for two weeks. Investigate any mismatch.
5. Rebuild critical dashboards in the new tool. Verify the numbers match the old dashboards to within noise.
6. Once the new tool's numbers are trusted, cut the dual-fire and commit the cleaned manifest (now baselined against the new tool).
7. Update `skene.config.json` to reference the new tool. Turn on the GitHub Action to catch any post-migration drift.

Metrics to watch

• Event volume parity (old vs. new)

  Within 1-2% for every tracked event during dual-fire window.

  Larger gaps usually mean a missed call site or a payload mismatch.

• Critical dashboards reproduced

  100% before cutover.

Failure modes

• Skipping the audit. You will discover events you forgot about, mid-migration.
• Not running parallel. You will only see what is missing after cutover, when it is expensive to fix.
• Letting the migrating developer rename events 'for cleanliness'. The cleanliness is downstream of consistency, not naming.

Related concepts