Engagement Program Standard Metrics Setup

[Anonymous]

I recently launched my first engagement program and put in about 2.800 members and after two casts (once a week) I'm staring at the dashboard pictured below. You'll notice there is no Engagement Score, nor are there any unsubscribe metrics.

What am I missing?

 


Thanks!

Grant

[Anonymous]

Thank you for clarification, Cheryl!

But it's not clear for me why engagement program success is represented in a wrong way with a dashboard. The programs I'm using are local to engagement program, and success is calculated normally for each of them.

Do you have any recipe? 

[Anonymous]

HI Everyone,

To reiterate, the engagement score is calculated differently for Programs inside engagement versus emails.  First of all, unsubscribe rates are NOT calculated for Programs, at this time.  It is not a bug.  Programs calculate the engagement score using the Program success, so the score is not as granular as for the email.  It will be different if you calculate the same email as standalone and then calculate it as part of the program.  Hope this helps to clarify any confusion.

Thanks,
Cheryl

[Anonymous]

Hi Alex,

I cannot answer your question, but I see the same in my programs. If I add programs = smart campaigns to the Engagement Program, neither engagement (always 90) nor unsubscribes (always 0%) are calculated correctly. Only in my very first Engagement Program, where I had added emails (instead of programs) to the streams, both engagement and unsubscrbes look somewhat real.



I haven't had time to follow this up with support, but IMHO there are still some bugs here.

[Anonymous]

Hi all,

Very interesting topic! And I have some issue to post here. 

I lauched an engagement program with local programs in there. I launched one smart campaign which tracks overall engagement program success and another campaign which tracks activitities (e.g. clicks in emails) I defined as successes for local programs. I launched local programs success tracking campaign several casts after the main success tracking campaign and collected previous success with batch campaign. 

Now that's what I see in the program dashboard: 


 At the same time overall program status is reflected correctly:



I saw that local programs successes are calculated correctly and have such numbers as 466, 1066 and others 

Why the program dashboard doesn't reflect correct score and only 90 for each cast? 

Thank you,
Alexander

[Anonymous]

Michael-

You are spot on in your understanding of my delema. And you are confirming my suspicions.

In my testing, however, I did find one thing that worked (for one scenerio). I added an invitation by dragging in an invitation email from the program. Next, I pulled in a campaign from the program just as you outlined above. I tested on two email addresses. When I got the emails, I openned one email and one I did not. The one that I openned, as expected, I did not receive the second invitation. But for the one I did not open, I received the second invitation. I believe it's because the address I did not open email was not added to the program as a member.

This seems to be one way to re-invite those who don't open the first email (or any), but it really is not the holistic solution we are looking for. 

In any case, the search continues. Thanks for your detailed response.

-Michael.

[Anonymous]

Michael,

I don't have a definitive answer to your question, but I suppose that this won't work.

Let's see if I have understood your setup: 

You have a webinar program in Marketo. This program contains of emails (at least 3, I guess), landing page(s) and smart campaigns. And in order to send an invitation for that webinar from your Engagement Program you have a smart campaign in your webinar program that you have dragged&dropped into your Engagement program. This smart campaign looks something like this:



This is how you would send one invitation, isn't it?

Now what you can do to achieve your 2) is to add a condition to the flow step that says exclude all registrants for the webinar. Like this:



So far so good. But 1) is difficult, I think. You can't add two emails from the same program to a stream and you can't add two smart campaigns from the same program to a stream. So what you would have to do is to add a smart campaign to the Engagement Program that contains the sending of two (or three) emails like:

Flow1: Send Email 1
Flow 2: Wait until day webinar date minus 1
Flow 3: Send Email 2

But with a flow like that you'd defeat the purpose of the engagement program with its set cadence. So the things I don't know are: Can you actually enter wait steps into smart campaigns inside of Engagment Programs and how do they interfere with the cadence of the stream?

What you could be doing is have invitation 1 and invitation 2 for your webinar in different programs. But nobody would want to do that, I guess.

I myself have added live webinar invitations to Engagement programs in the described way (Cheers to the "Edit Availability" option in streams) , but I haven't found a way of adding an invitation #2. 

Hope this is - maybe only a little bit - helpful.

Best,
Michael

[Anonymous]

Great thread. I too use stand alone programs for the same reasons. But I have an additional problem I'm trying to overcome. It concerns this statement:

3.  Programs give you more control over when to send a piece of content again or not - if you add a piece of content to a stream, and it already has members, then if the lead is a member already, it will skip it.

We do monthly webinars as part of our nurture program. I want to add more than one invitation for my webinar into the engagement stream. So my stream may look like this:

Week 1: Webinar invitation
Week 2: Whitepaper content
Week 3: 2nd Webinar invitation
Week 4: Blog article  
(And so on...)

So my question is, what would be a safe way to send a second (or 3rd) webinar invitation within one stream to those who didn't respond the first time within the stream? Again, these webinars are intended to be part of our ongoing nurture program. I want to:

1. Prevent the stream from not skipping a member with the second invitation since they would have become a member of the [external] program from the intial send.
2. Only send it to those who didn't respond (register) the first time. 

Is this possible to do in one stream in conjunction with an external program?

Thanks.
-Michael.

[Anonymous]

Ah, standalone... Yes, all emails in my stream are standalone. Sorry, didn't read carefully enough. So the missing metrics apply only to emails sent from programs within the engagement program, right?

Thanks,
Michael

[Anonymous]

No we didn't.  Do you have a standalone email in the stream as well?  Can you send me a link to the program?

[Anonymous]

2.  We cannot calculate the unsubscribe rate for an email inside of a program at this time.  This is an enhancement.  So it is not in error that you see 0%.

Has this just been fixed? I haven't seen any unsubscribe numbers until five minutes ago. But now they are there:

[Anonymous]

Jeff,

We're on the same page re non-local content. That's exactly the reason I structured it that way as well.

Thanks all!

[Anonymous]

Thanks for the clarification. Glad to hear the missing unsubscribe % isn't an error.

Grant, I chose to use non-local programs for the very reason that Cheryl's #3 addresses. Since I may have content used in conjunction with other channels, like display advertising, I don't want a lead that originally came into my system by downloading Whitepaper A to enter nurture only to then be served Whitepaper A again. I also keep my nurture content in non-local programs so I can try different email messaging with the same content safely. I can verify the skip mechanic works beautifully in this scenario. 

[Anonymous · Accepted solution]

I'm going to clarify a couple of points here in bullet form:

1.  Engagement Score takes 72 hours to calculate, whether it's a program or an email in the stream
2.  We cannot calculate the unsubscribe rate for an email inside of a program at this time.  This is an enhancement.  So it is not in error that you see 0%
3.  Programs give you more control over when to send a piece of content again or not - if you add a piece of content to a stream, and it already has members, then if the lead is a member already, it will skip it.  If there are no members, then it looks at whether or not the lead ran through it.  If you use the same program across multiple streams, it will not send the content more than once.

Hope this helps clarify!

[Anonymous]

Jeff,

Yes, that makes sense, and the same is happening with mine.

Do you have any idea whether the other automated functions like not allowing the same email to be sent to someone who has already received it applies to programs as well? Or is it still based on the email itself?

In other words, say have a program that sends an email and I put it in more than one stream - will my contacts that were cast the program the first time skip it when they see it again? Or is it the email that they are skipping? For example, what if I swap the email in the same program - will they skip it or will be sent the new email?

I suspect that the answer is that they will skip the program entirely, without any regard for what's inside - as it should be.

Hope that makes sense - can someone confirm?

[Anonymous]

Hey Grant,

I've built my engagement program the same way as yours (with all the content linking to non-local programs). What Cheryl is suggesting is that if you set the program successes of those programs (those ones you're referencing in-stream), the engagement score will calculate, but I believe you'll need to wait a couple days for it to acutally show up. 

As for the unsubscribe metric, I've been unsuccessful in getting that data to map over to the nurture program. So you're not alone there. Because I'm using non-local programs, the engagement program dashboard is actually pretty light on data for me. The only data I can see is the top two charts - Summary (except for Unsubscribed) and Members. The Progression status and "highest engagement" areas are completely blank for me. 

[Anonymous]

@Cheryl Are you talking about the program success of the programs that are in the stream?

What about unsubscribes? All programs have have an unsubsribed status that's been setup, but there is no unsubscribe data in the engagement program.

[Anonymous]

If you are using programs, the engagement score is calculated off of Program Success.  So you need to be changing the program status for the leads in the program.  If this isn't present, then there is no engagement score.

@Mike - we found a bug which is causing the date to display this way and we are working on a fix.

[Anonymous]

We have an issue with a dashboard showing dates from 1969 to 1970 without a way to modify them. I'm wondering if there isn't a wider dashboard issue at hand. We're opening a case to have the support team look into it. I'll let you know if hear anything.

[Anonymous]

The casts sent - the program is functioning as expected, it's just the Dashboard that's not working. The program seems completely unaware of all unsubscribes and there is not engagement score.

Are there additional trigger campaigns I need to be setting up to feed this information to the program?

[Anonymous]

My apologies, I was thinking more of the reporting and setting the report up properly. Have you set up a report to show what and how it was executed? That is where the smart list and set up filters can be set. If the casts didn't actually send, that might be a place to start troubleshooting.