What is AB testing? (in an eCommerce environment)

[robiulhasan]

AB testing is a specific type of controlled experiment. AB tests are used to determine the best option out of a possible range. In an eCommerce context, this could be determining which offers to show, what products to recommend, or even which product images or layouts work best for a given success criteria.



Like other multi-arm bandit tests, there are two distinct parts. First, there is an exploration phase where multiple options are considered. Once a winner is determined, the second phase of exploitat cell phone list ion starts. In this phase, the winner is chosen and the loser is discarded.

Properly constructed AB tests are comprised of the following parts:

Hypothesis - a testable statement predicting what will happen in the test.
Independent Variable - The independent variable is what we are changing to see what effect it has. In the context of eCommerce AB tests, it might be changing which bundles we offer, or different incentives (such as free shipping vs discounts).
Dependent Variable (Success Criteria) - The dependent variable is how we are measuring success. For an eCommerce AB test the dependent variable is often a KPI you are trying to improve such as conversion rate, increasing average order value, or repeat orders.
Test and Control Groups - Test and control groups are the visitors, prospects, or customers that experience the distinct independent variables and whose actions represent the dependent variable.
“AB tests are used to determine the best option out of a possible range. ”

While defining an AB test is nice, it is helpful to give a few illustrative examples for how AB testing can benefit eCommerce stores.