Testing Principles and Techniques in Marketo

Josh_Hill13
Level 10 - Champion Alumni
Level 10 - Champion Alumni

As you gain experience with Marketo, you will build increasingly complex flows to manage leads as well as to nurture them.

Marketo, in essence, is a rules engine. You decide the rules for your system and your audience. As with all computers, the rules you decide on are executed faithfully and without question. Thus, if your rules are not properly setup, they will go ahead anyway (if they are logically correct). For example, if you set your Smart List to ANY instead of AND, you will likely bring in many more leads than you intended, possibly ruining data or worse, sending out 100,000 emails to the wrong people.

Fortunately, there are ways to build workflows and test processes to avoid disasters. If you follow these principles and any other policies your firm has, you can reduce the error rate greatly.

Technique 1: Pay Attention

When to Use: all the time

Time Involved: 1 minute

Level: All

Paying attention seems like an obvious way to avoid mistakes. It is also prone to many human biases such as “Glossing over work you just did,” and “I’ve done this a thousand times before.” Be careful and follow a few of my rules when I operate alone.

  • Carefully create the smart lists. It is easy to drag, drop, and dash only to see the batch campaign send to 10 times the number you intended. When you think you are done, stop and look at the AND vs. ANY rules as well as the Counts.
  • Watch your Flows – I always check these three times. Remember a Flow step will run once for every lead that goes through.
  • Watch for red squiggly lines in Flow Steps and Filters. Even if it looks right, it means Marketo did not like something.
  • Watch the Schedule Count – does this count match what you thought? Did you subtract the blocked email count from the total? If something seems off, STOP.
  • Qualification Rules – Every vs. Once vs. something else. One of the most frequent questions on the Nation are related to this feature.
  • Scheduled Time – I always schedule a run for 10 minutes in the future because it is very easy to realize that the Email Subject Line is missing 9 minutes after you press Run.

Technique 2: Review Thrice

When to Use: always

Time Involved: 1 minute

Level: Any I use this technique in combination with Technique 1, cycling through the steps three times…or maybe I’m a little OCD about sending emails to thousands of people.

  1. Smart List x 3
  2. Flow x 3
  3. Schedule x 3

Technique 3: Paired Campaign Managers

When to Use: always

Time Involved: 1 minute to 1 day

Level: Any

A technique the Marketo marketing team uses is paired campaign managers. One person builds the Program, while the other prepares the creative. Then they switch to review each other’s work.If you have the staff, I highly recommend setting up this system as it helps to avoid the human ability to ignore errors and typos after working on something for 4 hours.You can go further and setup an entire approval process, even with just 3 to 4 team members:

  1. Build Program
  2. Add Creative
  3. Review Creative
  4. Review Program
  5. Test Program
  6. Approve by Director
  7. Launch

The one challenge with a full blown approval process is Marketo does not have an “approval system.” It may be possible for you to break out Roles according to the process above. For example:

  • Approver: can access all Marketing Activities
  • Program Builder: Marketing activities, but cannot send or approve emails or Pages.
  • Creative: Design Studio, Build Emails or Pages only. No Approval rights.

Technique 4: One Email, Multiple Leads

When to Use: Any system

Time Involved: 1 minute

Level: Any

This trick works on any email platform, although I tend to only use it on Gmail. You can create as many individual Leads in Marketo as you want and have them all go to the same email box.your.email@gmail.com

your.email+test1@gmail.com
  • Marked somewhere as a Test – IGNORE.
  • Fields values as you want them to be for the Test.
  • Qualification Rules set to Every Time. (otherwise, the lead goes through once regardless of test changes).

Once ready, be sure to copy the email address to a Form or notepad so you can keep using it. Then, make the change of value using any of the following steps:

  • Direct Edit
  • Edit in CRM
  • Fill Out Form
  • Change Data Value in Flow Action

Once your test is done, adjust your flow (if needed) and keep testing. To re-set your Lead, just undo the Data Value Change you made using the Direct Edit or another flow action.

Technique 5: One More Time…

When to Use: Basic Trigger workflows, Drip campaigns, Engagement Nurture, Lifecycle Testing, Lead Routing

Time Involved: 1 minute

Level: Any

This technique is just to modify the Schedule to use a Qualification Rule of Every Time. This way you can continually run the same set of Test leads.

Technique 6: The Brake

When to Use: Basic Trigger workflows, Drip campaigns, Engagement Nurture, Lifecycle Testing, Lead Routing

Time Involved: 1 hour to 2 days

Level: Any

This is by far the best technique and it is the most simple. At the end of your Smart List, add one of these two filters:Member of Smart List IN “Internal Test”

Email Address CONTAINS “@yourcompany.com”

Edit Wait Steps to 1 second or 1 minute – when you Clone, Brake, and Wait, you need to reduce all Wait Steps to 1 minute Any Time. Otherwise, you will Wait 2 days until Tuesday for the next email to go out. It can be a bit time consuming, which is why Cloning and Testing work better.

Technique 7: Software Style Testing Process

When to Use: Lifecycle Testing, Lead Routing, Very Complex systems

Time Involved: 1-5 days

Level: Advanced

This process involves setting up leads that meet various criteria to flow through your workflow. Each time you run each lead, you should have an Expected Result and an Actual Result. Once complete, you will have a clear list of potential flaws in the workflow and possible ways to resolve them.

Do not let “software” intimidate you. The test cases you setup will likely be a bit short of what a full Engineer in Test might do, but it’s close. Here are some terms you may come across:

  • Edge Case: used by engineers to discuss unlikely scenarios that could happen, but may not be worth the effort to test or fix. Be very careful that edge case leads do not bring the system to a halt.
  • Test Case: this is a planned test and lead that meets certain criteria we expect to happen. For example, the Lead is entered in Form X with State=CA and Country=Canada. What do we expect will happen? Test Cases may be called “Use cases” if created before the build.
  • Test Plan: The combination of Test Cases and materials to run through the system with Expected Results vs. Actual Result.

Portions of this post originally appeared on my blog and in collaboration with Steven Moody​.

3096
3
3 Comments
Jenn_DiMaria2
Level 10

I can't stress enough the importance of Technique 7. Nothing like importing a list of thousands of leads that accidentally change a data value for everyone Maybe try one or two of them first to see what happens.

Anonymous
Not applicable

Josh this is an incredibly insightful post that will help us out as newbies, but is likely something we'll continue to refer back to as we grow into our instance. Many thanks!

Anonymous
Not applicable

Great info! I feel like our team has a great Marketo setup but when it comes to testing/approvals, the attention to detail goes out the window! We just implemented a formal approach to Technique 3 but if only Techniques 1-2 were ALWAYS followed, we would avoid so many costly errors