Define the denominator before comparing conversion rates
The denominator defines who had the opportunity to perform the measured action. Changing it can change the rate without changing user behaviour.
Read field note12 PRACTICAL FIELD NOTES
Growth measurement
Define funnels, cohorts and experiments carefully before making decisions from business metrics.
Pair a growth metric with a guardrail against unwanted side effects
A guardrail tracks a consequence that should not deteriorate while the main outcome improves. It helps the team notice when a change makes one number look better by moving costs or problems elsewhere.
Read field noteUse cohorts when calendar totals hide different user journeys
Use a cohort when you need to compare people who started under similar conditions and observe them for the same length of time. Calendar totals mix new and established users, which can obscure onboarding or retention changes.
Read field noteMeasure retention at the frequency of the task your product supports
No. The useful return interval depends on how often the task occurs.
Read field noteSeparate leading signals from the business outcomes they may support
A leading signal is an earlier observation that may help anticipate or influence an outcome. It is useful only when its relationship with that outcome makes sense and is checked in context.
Read field noteCreate a baseline before changing a business workflow
Record the current outcome under the conditions relevant to the experiment: the task, period, eligible cases and important constraints. A baseline makes the comparison more interpretable, but it does not automatically isolate the effect of your change.
Read field noteInterpret small-sample metrics without false certainty
Show the underlying counts and treat the result as preliminary. When the denominator is small, one additional event can move the percentage substantially.
Read field noteChoose average or median task time according to the question
The average reflects total time divided by the number of tasks, while the median is the middle value after ordering the observations. Long exceptional cases can pull the average upward.
Read field noteUse last-click attribution as a limited view of the customer journey
Last-click attribution assigns the recorded conversion to the final tracked interaction. It can be useful for understanding that interaction, but it does not establish which earlier experience caused the decision.
Read field noteWrite a metric definition sheet so reports answer the same question
Specify the business question, unit, event definition, formula, time window, exclusions and source. The sheet prevents different teams from using the same label for different measures.
Read field noteCheck data quality before explaining a growth trend
Check completeness, duplicate records, event meaning, time alignment and any recent tracking changes. An apparent trend can arise from delivery or measurement errors rather than user behaviour.
Read field noteReview growth results without selecting only favourable numbers
Start with the question and measures agreed before the work, then include relevant negative or unchanged results. A review should improve the next decision rather than defend the previous plan.
Read field note