Keeping Track Of A/B Test Results

[Anonymous]

I need to do a better job tracking and getting long-term insights from all A/B tests. With that in mind, does anyone have a method/template that they use to track and analyze A/B tests?

I envision creating an Excel template with different columns such as:

Asset Type Tested: Email

Type of Test: Whole Emails

Date of Test: 1/24/17

Time of Test: 6:00 pm EST

Variable Tested: Button Color

Variables: Red Button Vs. Blue Button

Winning Criteria: Clicks

Test Results: Red button scored better

Variable 1 Results: Red had X clicks

Variable 2 Results: Blue had X results

Additional Comments: ....................

Does anyone have any thoughts on this? Is there a better way to manage A/B test insights? Are there other things I should be tracking? Any insights are appreciated!

Steve

[Sara_Greaves]

Hi Stephen,

Personally, I do something very similar to what you do. I just maintain mine in a narrative style format. I like being able to describe the test in detail and write out any conclusions. Since I share this information out to stakeholders, it seems like this format is easier for them to digest. This may not be relevant to your business, but I track who requested the test, what product line the test was related to, and what the specific CTAs or subject lines were, for example. It sounds like what you're doing will work!

[Kim_Allen]

Stephen,

Unfortunately testing results are a beast to keep up with. I always found a spreadsheet worked best for me as well.

I'd usually pick a variable to focus on for a while (qtr or so at a time) and really get a good sample size and testing in and compile in one tab per variable. This way I could include industry specific testing all in one variable in one tab (but it totally depends on your business).

Wish there was a better way!

Kim

[Anonymous]

Hi, Stephen!

This is something I've struggled with also. A while back I created a Google Doc to capture this information (see screenshot below) but I've only used it for two A/B tests so far. Maybe if I switch to a spreadsheet format it would be easier/faster to enter the A/B test details. It would be cool if there was a place in Marketo where we could see all of the A/B tests we've done and record insights/notes, etc.

[Julz_James]

Hi,

I've always worked in an Excel Spreadsheet, and have all the headings as you've suggested above.  What I also tend to do is highlight the winners for each type of test (and even use different tabs in a spreadsheet for different types of tests), so that if I want to go back and look at Button colors for example I can see easily which colors worked and which didn't. 

Its always really useful having the spreadsheet for others to see so that they can either make suggestions or see the results to back up why you might be making template changes. 

Thanks

Julz