ECMS A/B Split Testing

From NewHaven Software Wiki

If you want to do split testing in eCMS using Webmaster Tools one of the cleanest ways to do it is with a promo or customer group code set via a URL parameter. (To use a promo code you must first set one up in a customer group rule [[Promo_Codes|as described here].) The advantage of using this method is that it does not mess up any forms redirection and keeps the URLs the same, sans the snippet at the beginning setting the promo/customer group.

An example of the two URLs for use in Webmaster Tools would be something similar to the two below:

http://www.example.com/checkout_shipping/?p=pageb
and
http://www.example.com/checkout_shipping

Then in if you wanted to test variations in the the checkout_shipping template just do a simple test:

{if $promo.code == 'pageb'}
Do this.....
{else}
Do something else.....
{/if}

You will also need to add a test condition to the _meta.tpl, something like this:

{if $view == 'checkout_thanks'}
{literal}
<!-- Google Website Optimizer Tracking Script -->
....code for confirmation page.....
<!-- End of Google Website Optimizer Tracking Script -->
{/literal}
{/if}

{if $view == 'checkout_shipping' && $promo.code == 'pageb'}
{literal}
<!-- Google Website Optimizer Tracking Script -->
.....Code for Page B......
<!-- End of Google Website Optimizer Tracking Script -->
{/literal}

{elseif $view == 'checkout_shipping'}

{literal}
<!-- Google Website Optimizer Control Script -->
....Code for default page.....
<!-- End of Google Website Optimizer Tracking Script -->
{/literal}
{/if}

I tested this in Website Optimizer and it works. Two things to note:

• If you try to manually navigate to a checkout page you will be redirected to viewcart or the index page so you will have to manually upload the HTML for the pages for Google's validation service to work. Pretty trivial process and you would likely have to do this with any cart.

• If you set the promo code via the URL you may have this overwritten by a customer using a promo code. This could potentially skew your statistics a bit. There may be a way to compensate for this with proper setup in GWO. The other option is the use the customer group via URL feature. Similar to promo codes, ?c=CUSTGROUP.