Is your incrementality testing giving you accurate data or are you burning budget on a broken signal?
Austin Harrison (CEO, Northbeam) joins Sean Frank (CEO, Ridge) and Katy Mimari (CEO, Caden Lane) to challenge the most repeated advice in ecommerce measurement: that longer tests are always better, holdouts are interchangeable, and incrementality alone is enough to make media decisions. They break down when a growing brand needs incrementality versus MTA, and how running tests inside a noisy environment can produce weeks of data worth nothing.
Sean pulls from Ridge’s own results to show how layering incrementality on top of MTA unlocked YouTube spend that in-platform data never would have justified, while Katy’s botched first test proves clean test conditions matter as much as the tool itself. They also dig into why moving up funnel on Meta is outperforming conversion campaigns at scale, and what separates the fastest-growing ecommerce brands in 2026 on creative.
Powered By
Fulfil
https://bit.ly/3pAp2vu
Saras Analytics
https://bit.ly/9OP-Ytdesc
Aftersell
https://9ops.co/4i3bb5
Richpanel
https://9ops.co/richpanel
Northbeam
https://www.northbeam.io/
Postscript
https://9ops.co/postscript
Chapters
00:00:00 How Long Should Tests Run
00:06:10 MTA vs Incrementality
00:12:00 Katie’s Messy First Test
00:17:00 Snorkel to Scuba Analogy
00:23:00 Holdout Tests Explained
00:30:00 The Over-Testing Trap
00:37:00 When the Data Disagrees
00:43:00 AI Ads Are Already Here
00:49:00 Sean as Caden Lane CMO
00:57:00 North Beam’s Next Move
01:01:00 Creative Volume Wins
01:06:30 What’s Working Right Now


