Introducing testing concepts via a real-world scenario that we all can relate to: Dessert

Published by Rommil on

A/B Testing can be intimidating to the uninitiated, but it doesn’t have to be! I want to introduce testing concepts via a real-world scenario that we can all relate to: Dessert. I’ll cover what an MVT is, hypotheses, independent variables, dependent variables, results (of course), and design flaws. And if you need any more convincing, I have four words for you: Monte Cristo Bread Pudding.

Here’s a snippet of my talk track:


But then, a couple of difficult questions arose. Should the bread be crusted or crustless? Should the sandwich be pressed before being battered? I knew my decisions could have major repercussions on the texture, and therefore the flavor, of my dessert-for-breakfast meal.

I can’t pretend I didn’t spend a day or two thinking about the trade-off between texture contrast and burnt bits resulting from keeping the crust, or that I didn’t lose sleep trying to work out the impact of compression on batter absorbency. What was I to do?

Oh, wait! I can answer these questions—WITH SCIENCE! After all, empirical analysis is my bread and butter. So I decided to run a multivariate test.

A multivariate test, or MVT, is an experimental design that allows the researcher to investigate the effects of two or more independent variables on one or more dependent variables. An independent variable is the thing the researcher is systematically changing and a dependent variable is the thing the researcher is expecting to be impacted due to that change. The MVT I planned to run is called a 2×2 MVT, because I have two independent variables and each variable can take one of two values (called levels):

Variable 1: Crust
Levels: Crusted, Crustless

Variable 2: Compression
Levels: Pressed, Unpressed