4 Replies Latest reply on Jan 30, 2017 10:38 AM by Juli James

    Keeping Track Of A/B Test Results

      I need to do a better job tracking and getting long-term insights from all A/B tests. With that in mind, does anyone have a method/template that they use to track and analyze A/B tests?

       

      I envision creating an Excel template with different columns such as:

       

      Asset Type Tested: Email

      Type of Test: Whole Emails

      Date of Test: 1/24/17

      Time of Test: 6:00 pm EST

      Variable Tested: Button Color

      Variables: Red Button Vs. Blue Button

      Winning Criteria: Clicks

      Test Results: Red button scored better

      Variable 1 Results: Red had X clicks

      Variable 2 Results: Blue had X results

      Additional Comments: ....................

       

      Does anyone have any thoughts on this? Is there a better way to manage A/B test insights? Are there other things I should be tracking? Any insights are appreciated!

       

      Steve

        • Re: Keeping Track Of A/B Test Results
          Sara Greaves

          Hi Stephen,

           

          Personally, I do something very similar to what you do. I just maintain mine in a narrative style format. I like being able to describe the test in detail and write out any conclusions. Since I share this information out to stakeholders, it seems like this format is easier for them to digest. This may not be relevant to your business, but I track who requested the test, what product line the test was related to, and what the specific CTAs or subject lines were, for example. It sounds like what you're doing will work!

          • Re: Keeping Track Of A/B Test Results
            Kim Allen

            Stephen,

             

            Unfortunately testing results are a beast to keep up with. I always found a spreadsheet worked best for me as well.

             

            I'd usually pick a variable to focus on for a while (qtr or so at a time) and really get a good sample size and testing in and compile in one tab per variable. This way I could include industry specific testing all in one variable in one tab (but it totally depends on your business).

             

            Wish there was a better way!

             

            Kim

            • Re: Keeping Track Of A/B Test Results
              Jamie Gregory

              Hi, Stephen!

               

              This is something I've struggled with also. A while back I created a Google Doc to capture this information (see screenshot below) but I've only used it for two A/B tests so far. Maybe if I switch to a spreadsheet format it would be easier/faster to enter the A/B test details. It would be cool if there was a place in Marketo where we could see all of the A/B tests we've done and record insights/notes, etc.

               

              Screen Shot 2017-01-26 at 10.12.16 AM.png

              • Re: Keeping Track Of A/B Test Results
                Juli James

                Hi,

                 

                I've always worked in an Excel Spreadsheet, and have all the headings as you've suggested above.  What I also tend to do is highlight the winners for each type of test (and even use different tabs in a spreadsheet for different types of tests), so that if I want to go back and look at Button colors for example I can see easily which colors worked and which didn't. 

                 

                Its always really useful having the spreadsheet for others to see so that they can either make suggestions or see the results to back up why you might be making template changes. 

                 

                Thanks

                 

                Julz