Below is the conclusion of a three-part series in which Michael Spitzer-Rubenstein talks about A/B testing your web pages, which can make a huge difference in how well they convert site visitors into supporters and donors. Michael is the president of Spitzer-Rubenstein Strategies, a digital campaign consulting firm specializing in email, social media, online advertising, and website development and optimization. Note that if you leave questions comments below or tweet them to @MSRPOTUS, Michael will answer them in tomorrow’s conclusion, and be sure to check out Part One and Part Two of the series.
How Do I Get Started with Website A/B Testing? Questions and Potential Pitfalls
So far, we’ve talked about how one campaign managed to increase their online donations by 158% with A/B tests. We went over how A/B tests work. Now, lets make sure you’re ready to avoid problems.
A/B testing seems complicated. Why can’t I just make a change to my website and then compare the new results to the old results, without bothering with an A/B test?
Once you have a system up and running, an A/B test can take only a few minutes to set up. And if you don’t set up an A/B test, you could end up misguided because of external events and actually hurt your website. What if you make the change just before a big event (in the campaign world, perhaps a debate) and then all of the sudden, you get more donations? How do you know whether the increase was a result of the debate or because your website is better? You don’t. With an A/B test, you know that any changes are solely because of the website change, not because of anything else, because you’d see the effects of anything else on both variants, not just one. The upshot: A/B tests help you isolate variables.
How many things can I test at once?
You can run an unlimited number of tests with an infinite number of variants; the only limitations are your time and the number of visitors you can test website changes on. However, if at all possible, only test one aspect at a time. Maybe you want to test this headline or that image — in that case, split them up into separate tests so that you can identify whether the changes are because of the headline or the image, and then run those tests sequentially.
The only time you want to make several changes for one test is if those changes only make sense together. For instance, if the donation page currently asks for contributions to support a local soup kitchen but you think it might be better to emphasize the kids fed by the organization, changing just the headline wouldn’t make much sense. Instead, it’s probably better to just test the new page against the old page and see which does better. Keep in mind, though, that if you’ve worked on one page much more than another, you might be comparing a Picasso to a school-kid’s drawing: they might be superficially similar — both are paintings on paper or both are donation pages — but they are very different in quality.
Where can I learn more about A/B testing?
- The founder of Visual Website Optimizer had a great article in Smashing Magazine about A/B testing with several case studies that might inspire you. While it’s a few years old, it’s still a great resource.
- KissMetrics has a great list of resources specifically for Google’s Content Experiments (previously called Website Optimizer). Their blog is also a great source of advice about online marketing in general.
- How to create an A/B testing plan.
I’m also happy to talk to anyone about how to do A/B testing and increasing donations, volunteer signups, or otherwise making your website better. Reach out at http://michaelspitzer-rubenstein.com.
Thanks Michael! Great work on the series — I bet a lot of readers will be trying this out at home.