ECMS A/B Split Testing

From NewHaven Software Wiki

If you want to do split testing in eCMS one of the cleanest ways to do it is with a promo or customer group code set via a URL parameter.

http://www.example.com/checkout_shipping/?p=pageb
and
http://www.example.com/checkout_shipping

Then in the checkout_shipping template just do a simple test:

{if $promo.code == 'pageb'}
Do this.....
{else}
Do something else.....
{/if}

You will also need to an a test to the _meta.tpl, something like this:

{if $view == 'checkout_thanks'}
{literal}
<!-- Google Website Optimizer Tracking Script -->
....code for confirmation page.....
<!-- End of Google Website Optimizer Tracking Script -->
{/literal}
{/if}

{if $view == 'checkout_shipping' && $promo.code == 'pageB'}
{literal}
<!-- Google Website Optimizer Tracking Script -->
.....Code for Page B......
<!-- End of Google Website Optimizer Tracking Script -->
{/literal}

{elseif $view == 'checkout_shipping'}

{literal}
<!-- Google Website Optimizer Control Script -->
....Code for default page.....
<!-- End of Google Website Optimizer Tracking Script -->
{/literal}
{/if}

I tested this in Website Optimizer and it works. Two things to note:

• If you try to manually navigate to a checkout page you will be

redirected to viewcart or the index page so you will have to manually upload the HTML for the pages for Google's validation service to work. Pretty trivial process and you would likely have to do this with any cart.

• If you set the promo code via the URL you may have this overwritten by

a customer using a promo code. This could potentially skew your statistics a bit. There may be a way to compensate for this with proper setup in GWO. The other option is the use the customer group via URL feature that was just released. Similar to promo codes, ?c=CUSTGROUP. This feature was just released and anticipating your wanting to continue to use promo codes in conjunction with A/B testing I did some testing of this feature today and found a minor bug where the customer group is not persisting in the template vars. I have spoken with CV3 regarding it and they are looking at it right now.