Comparing A/B Testing Tools in 2014
A/B testing tools are an essential part of a marketers’ toolkit, but with so many options, choosing the right one isn’t easy. A key factor in this decision is the amount of web traffic on your website, the type of experiments you want to run and of course budget.
I went around the web and created a comparison of 9 different a/b and multivariate testing companies to help you decide which service to go with. Whether you’re looking for a light-weight A/B testing tool or a more robust platform for complex multivariate tests. This article will let you understand the different options that are out there.
Company Overviews:
Optimizely – A website optimization platform enabling A/B and multivariate testing for users to enhance their websites. Co-founders Pete Koomen and Dan Siroker originally started out building an educational site to teach kids math, but felt disconnected from the target users and weren’t confident the service could grow. After they got into Y Combinator they pivoted into what would become Optimizely an a/b testing platform that is widely used by tech companies.
Unbounce – A landing page platform for marketers to build, publish and A/B test their pages without developers. Co-founders Rick Perreault and Oli Gardner developed Unbounce after experiencing first hand the frustration of trying to get effective landing pages launched for their own online marketing campaigns.
Monetate – A cloud technology company providing marketing and website optimization solutions, services and content resources. Co-founders David Brussin and David Bookspan wanted to create a platform that allowed retailers to successfully serve each individual customer, rather than force them into one-size-fits-all marketing.
Maxymiser – They offer cloud-based testing, personalization and cross-channel optimization solutions for brands to boost engagement and revenue. Co-founders Max Polyakov and Mark Simpson wanted
SiteSpect – Is a tag-free A/B and multivariate testing and targeting tool that enables large enterprises to test content, features and functionality. Founded by Eric Hansen.
Visual Website Optimizer – A testing and website optimization tool allowing enterprises to create A/B tests and geo-behavioral targeting campaigns. Founded by Paras Chopra.
Google Analytics – A free service offered by Google that generates detailed statistics about the visitors of a website.
SiteTuners – An agency that developed their own a/b testing and multivariate testing suite. Founded by Tim Ash best-selling author of Landing Page Optimization and was one of the first agencies to focus on CRO.
ClickThroo – A landing page software platform for marketers. Co-founders Dave Bird and Ignas Bernotas founded ClickThroo to help marketers not only build landing pages without IT, but to help test, measure and optimize the conversion process.
Comparison Chart
| Company | Pricing | A/B or MVT | Traffic Segments | Multiple goals | Able to create webpages | Email Support |
|---|---|---|---|---|---|---|
| Optimizely | Free Trial then usage-based tiers from $19-$400 per month | Yes | Yes | Yes | No | Yes |
| Visual Website Optimizer | Free Trial then usage-based tiers from $59-$875 per month | Yes | Yes | Yes | Yes | Yes |
| Google Analytics | Free | A/B | Yes | Yes | No | No |
| Unbounce | Usage-based tiers from $49-$199 per month | A/B | No | Yes | Yes | Yes |
| Sitespect | Contact for Pricing | Both | Yes | Yes | No | Yes |
| Monetate | Contact for Pricing | Both | Yes | Yes | No | Yes |
| Maxymiser | Contact for Pricing | Both | Yes | Yes | No | Yes |
| SiteTuners | $1500-$2500 per Test | Both | Yes | Yes | No | Yes |
| ClickThroo | Usage-based tiers from $49-$1195 per month | Both | Yes | Yes | Yes | Yes |
Bonus Definitions:
A/B testing – An A/B test involves running live web traffic at two different versions of a web page — an A version (the control) and a B version (the variation). Then measuring the results each version has on your conversion rate.
Multivariate testing – A multivariate test involves different content for many elements across one or more pages to identify the combination of changes that creates the highest conversion rate.
- Full Factorial – The most straightforward method whereby all possible combinations of content variants are served with equal probability. In this method, you distribute website traffic equally among all combinations. If there are 16 combinations, each one will receive one-sixteenth of all the website traffic.
- Taguchi Method – A more complex method to test the position of different content across different webpages. For example, if 3 different images are to be tested in 3 locations, there are 9 combinations to test.
Heatmap data – Web heat maps are used to display areas of a Web page most frequently scanned by visitors. Visualizing your visitors’ every mouse move, click and scroll can help improve conversion rates and site usability.
Geotargeting – Is the method of determining the geolocatione of a website visitor and delivering different content to that visitor based on his or her location.
Conclusion
Even though these companies may seem to offer similar functionality, the way they handle things is very different. There’s plenty of factors you have to consider to make sure their solution is the right fit, don’t go with a company because that’s what everyone else does.
For example, If you don’t have technical chops, VWO is more user-friendly with their click and play interface, but can be limited in other areas. Where Optimizely requires a bit of front-end knowledge to fully take advantage of their solution.
If you have suggestions on how this article can be improved feel free to let me know in the comments!
Some Survey Data Sourced from Conversion Rate Experts
Image courtesy of bplanet/freedigitalphotos.net