A/B (split) Testing with Google Analytics Experiments in Drupal

Submitted by Mark Kindred on Tue, 07/24/2012 - 14:35

I've been wanting to delve into A/B testing for some time, but I just haven't had the time available. Today, I finally decided to jump in and see how long it takes to set up.

Before I get too far, I need to tell you that the process I'm describing is to set up an experiment in Google Analytics for a site running Drupal 6. Recently, Google shut down its Website Optimizer product and rolled it into Analytics Experiments. There is an existing Drupal module that worked for Optimizer, but the module hasn't been updated in a long time, so I'm assuming it doesn't work for Analytics. (If that's not the case, please leave a comment correcting me.) There might be some minor differences between Drupal 6 and 7, but it should be easy to figure out what they are.

Define the Experiment

First, determine the pages involved in the experiment. One simple example might be: I want to determine if, by making the Contact Us menu link more visible on the home page, I can get more people to submit my contact form. (If this were a real experiment for a client, I would put more thought into it; this is merely a quick example to get you started.)

Create the Variation Page

In order to do this, we'll need to create an alternate version of the home page. Go to the existing home page, and copy the body content.

Now, create a new page with the same title and content, but give it a unique URL, like "/home2". I recommend that you keep the page out of your sitemap.xml and assign a nofollow meta tag to the page, so it doesn't end up as duplicate content in search engines.

Before going to Google Analytics to set up the experiment, let's make a change to the "home2" page. (This change is, after all, what we're testing.) For this test, I'm just going to make the "Contact Us" link in the main menu bold.

First, find the link and its associated identifier in Firebug:

Finding Contact menu item in Firebug

OK, so the menu list item's class is "menu-mlid-512". But how do we target it on just this page? Grab the page-xxxxx class from the <body> tag. In this case, the page class is "page-home2":

Finding page's body tag class

A quick addition to the appropriate stylesheet:

/* *********************************************************************** */
/* Experiments                                                             */
/* *********************************************************************** */
body.page-home2 #block-menu_block-1 li.menu-mlid-512 a {
    font-weight: bold;
    color: #000;
}

nets us this:

Contact link after style change

Yeah, I know; it's not the greatest experiment. If this were real, we'd probably want to create a nicer call to action. We would perhaps design a nice graphic to put in the sidebar. In that case, instead of using stylesheets, we would probably add it via a block, visible only on /home2.

Set up the Experiment in Google Analytics

Before continuing on to create the experiment, you'll need to have a Goal set up in Analytics. I already have one called "Contact Form Submitted".

Next, go to the Experiments page in Analytics. Press the "Create Experiment" button above the list of existing experiments:

Create Google Analytics Experiment

Enter the URL of the page you want to improve, and press "Start Experimenting". In this case, it's our front page:

Add URL for experiment

On the following page, enter the basics of the experiment, and press "Next Step":

Define experiment parameters

BTW, you'll notice that Google attempts to verify that all pages (the original, plus all variations we create) exist.

On the following page, set the Goal and administrative parameters:

Set goal

We're almost there. On the next page, choose "I'll add the experiment code myself", and copy the script code:

Copy Google Experiment javascript code

If you press "Next Step", Google will attempt to verify you've added the code to the correct page:

no experiment code found

Oops, I forgot to add the code to the original page. Let's correct that.

Add the Code Snippet to your Original Page

This is the point where the Drupal module would be useful. But that's OK, we don't need it for this simple experiment. In your default theme, find the template file that contains the <head> element. This assumes that you are working on a contributed theme, not on a core theme. You should never modify files in the /themes directory of your Drupal installation. In my case, the correct file is /sites/all/themes/arsnova/page.tpl.php. Depending on the Drupal and theme versions, this might be slightly different.

Find the appropriate Drupal template file

We know it's the right file if we see the <head> tag:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="<?php print $language->language; ?>" lang="<?php print $language->language; ?>" dir="<?php print $language->dir; ?>">

<head>
  <title><?php print $head_title; ?></title>
  <?php print $head; ?>
  <?php print $styles; ?>
  <?php print $scripts; ?>
</head>
<body class="<?php print $classes; ?>">

Remember the script snippet we copied from Google? That goes here, directly after the <head> tag. Note that we've added PHP code to show the code only for node number 1:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="<?php print $language->language; ?>" lang="<?php print $language->language; ?>" dir="<?php print $language->dir; ?>">

<head>
<?php

    if ($node->nid == 1) {
        echo <<<EOD
       
<!-- Google Analytics Content Experiment code -->
<script>function utmx_section(){}function utmx(){}(function(){var
k='2085658-2',d=document,l=d.location,c=d.cookie;
if(l.search.indexOf('utm_expid='+k)>0)return;
function f(n){if(c){var i=c.indexOf(n+'=');if(i>-1){var j=c.
indexOf(';',i);return escape(c.substring(i+n.length+1,j<0?c.
length:j))}}}var x=f('__utmx'),xx=f('__utmxx'),h=l.hash;d.write(
'<sc'+'ript src="'+'http'+(l.protocol=='https:'?'s://ssl':
'://www')+'.google-analytics.com/ga_exp.js?'+'utmxkey='+k+
'&utmx='+(x?x:'')+'&utmxx='+(xx?xx:'')+'&utmxtime='+new Date().
valueOf()+(h?'&utmxhash='+escape(h.substr(1)):'')+
'" type="text/javascript" charset="utf-8"><\/sc'+'ript>')})();
</script><script>utmx('url','A/B');</script>
<!-- End of Google Analytics Content Experiment code -->

EOD;

    }

?>
  <title><?php print $head_title; ?></title>
  <?php print $head; ?>
  <?php print $styles; ?>
  <?php print $scripts; ?>
</head>
<body class="<?php print $classes; ?>">

This is where the module would come in handy. I assume it would insert the code for us, so we wouldn't have to change a template file. But, hey, we're not hacking the core, so I have no problem with this. Just remember to take this code out when your experiment ends.

Note that node id 1 is not necessarily your front page in Drupal. In fact, most sites I develop don't even use a single node for the front page; I often use Panels or other similar modules for complex front pages.

Now, let's see if Google says we did everything to their standards. We'll try to re-validate:

Now it validates

Well, that worked better. Sometimes, it pays to follow the instructions. Sometimes. :)

Google gives us one more chance to review before we begin running the experiment:

Review the experiment before we start it running

That's pretty much it; our experiment is up and running as soon as I press the "Run Experiment" button. The JavaScript that we inserted into the home page redirects half of the home page visitors to the variation page. From there, it tracks which visitors trigger our goal and breaks it down by original/variation.

Give it a day or so to begin collecting data, and you should start seeing results.

Notes

One thing to note: As with many other changes and additions to GA, the new Experiments area has a couple bugs they need to work out. For example, although you can stop an experiment, you cannot delete it (as of this writing). If you search through Google's help documentation, they explain how to do it in the old system, but it seems the controls are missing in the new version.

Give them time; I'm sure they'll get the kinks worked out.

Comments

question

Submitted by bryce (not verified) on Wed, 10/17/2012 - 13:37.

is this able to work by not directly inserting the code into the template file? for example, can you just add the code to the body section if you have a custom page setup? (just a regular page file in drupal?)

i added the code and it seemed to be placed because google said OK, but then I realized that this google code automatically redirects the user, doesn't it (when they hit your original page?) so I don't think it will actually work.... not sure though.

According to Google, you need

Submitted by Mark Kindred on Wed, 10/17/2012 - 13:43.

According to Google, you need to place the code directly after the <head> tag, so by the rules, you couldn't just place it in a node. My experience is that sometimes you can fiddle with the placement of analytics-related code, and sometimes you cannot.It's OK that it redirects you; that's what it's supposed to do. It redirects half (I assume exactly half in a test with two variants) to the alternate page.I hope this helps.

no actually it didn't

Submitted by bryce (not verified) on Thu, 10/18/2012 - 14:41.

no actually it didn't redirect me, sorry what i was saying was that i realized it is supposed to redirect, so i'm not sure if this code added to say body area of a drupal created cntent page (actually placing the code into the content area within drupal) would give it the proper permisisons (?) to allow it to redirect. i set up a test though and let it run.. we will see.. but i'm thinking i may have to do it your way. but yeah, i know what google said.. was just hoping for an easy setup as opposed to hacking the header.

What ever happened with this?

Submitted by Anonymous (not verified) on Tue, 01/08/2013 - 15:55.

What ever happened with this? Did you need to hack the header or did the dropping the code into the content area work for you?

thank you i figured out the

Submitted by Rekha (not verified) on Mon, 09/24/2012 - 16:49.

thank you i figured out the solution .
regards,
Rekha

Good to hear.

Submitted by Mark Kindred on Mon, 09/24/2012 - 16:51.

Good to hear.

Split testing not working

Submitted by Rekha (not verified) on Mon, 09/24/2012 - 16:39.

Hi ,
We have a Drupal 7 website and we want to split testing on the home page
www.lasvegasrecovery.com

but when i implemented the code as you had specified , we are getting a error saying that google is unable to find the code . No experiment code found
Could you please guide me on this
regards,
Rekha

Rekha, I'm not sure how much

Submitted by Mark Kindred on Mon, 09/24/2012 - 16:50.

Rekha,

I'm not sure how much help I can be from here, but it appears that your Experiment JavaScript is being commented out.

This is not multivariate

Submitted by Anonymous (not verified) on Sat, 08/11/2012 - 10:10.

This is split A/B testing not multivariate testing.

Thanks for the correction.

Submitted by arsnova_admin_dude on Sat, 08/11/2012 - 11:05.

Thanks for the correction. I've been using the term incorrectly for quite some time.