Back to the playbook

Acting on what you find

Testing the insights you have collected, working out whether your ad spend creates demand, and telling the story so somebody acts on it.

This part is about what to do once you have an insight: test it, find out whether your marketing actually works, and get someone to act on it.

Experimentation covers testing an idea when you don't have the traffic for a textbook A/B test. Incremental testing and media mix modelling answer the question every ad budget raises: did this spend bring customers who wouldn't have come anyway? Data storytelling turns the result into a dashboard people understand, and the last chapter covers what it takes for AI to answer your analytics questions accurately.

Read the two marketing chapters together. An incrementality test measures one channel at a time; media mix modelling covers the channels you can't switch off or track with a link.

  1. 24How to run experiments at different stages?Most companies collect insights and never test them. You might not have the traffic for a textbook A/B test, and that is fine. Run a smaller version instead, such as a before-and-after comparison or a group of users held back, and keep running it.3 min read
  2. 25Incremental testing: are your ads creating demand?Incremental testing tells you whether your ad spend brings signups that would not have arrived anyway. You switch ads off in a holdout group of markets, leave them running in a matched group, and compare signups. The gap between the two groups is your incremental signups.4 min read
  3. 26Media mix modelling: measuring untrackable channelsMedia mix modelling measures the channels you cannot track with a link, such as influencers, offline campaigns and brand spend. You give it one row per week with spend per channel, total signups and what else was going on. It works backwards to split your signups into base and the part each channel caused.3 min read
  4. 27Data storytelling: building a dashboard people understandA dashboard people understand opens on a question, orders its reports so each one follows from the last, and lands on an answer.3 min read
  5. 28Analytics in the age of AI: making agents trustworthyAI can answer your analytics questions, but only with clean data, a written explanation of what it means, and the exact calculation for each core metric. Skip those and it still answers but a good chunk of the time the answer is wrong.5 min read