Skip navigation
All Places > Products > Blog > Authors Sanford Whiteman
1 2 3 Previous Next


44 Posts authored by: Sanford Whiteman

Or not so much “beware” as ignore an Email Bounced Soft that doesn't have an associated Category.


Far too many posts and practices imply that grouping Email Bounced Soft-s together with a simple filter is harmless. The thought is that you may want to separate Category 3, Category 4, and Category 9 but you don't have to.


The approach is implicitly encouraged by the official docs:



But this doc is misleading, because there's one type unlisted: call it the No-Category (NC) Soft Bounce.


Guess what falls into NC? If you read my posts avidly, you might suspect it's something code-related. Yep: a Velocity token parsing error results in an NC Soft Bounce.


Here's one I triggered at a client just now by accidentally deleting the close parenthesis ) from a #set directive:



I quickly fixed the error before sending again in this case.


But imagine if it took more troubleshooting, and each send triggered an operational campaign (or qualified for the equivalent daily batch) that counted Email Bounced Soft activities without a Category constraint, setting Marketing Suspended = true when people exceeded a threshold. There are well-known recipes for Marketo database cleaning out there that do just that!


Don't make that mistake. Constrain Email Bounced Soft by Category [is not empty] to catch only the bounce types which may call the lead's info into question. Velocity coding errors must not be held against the lead, as they're completely in your court.


You can view just the Velocity parsing errors like this:




P.S. and N.B.: the Velocity errors that are surfaced as NC Soft Bounces are Velocity Template Language (VTL) parsing errors. That is, forgetting the #end of an #if, missing parentheses or brackets, or other broken syntax. That doesn't include Java language errors thrown by syntactically valid VTL, like trying to get() a nonexistent index in an ArrayList of Custom Objects. The latter type of error shows up verbosely in the Preview UI, but if it makes it to send time, it's swallowed by the system. You will only see the Sent Email without a corresponding Delivered Email in this case. Obviously that's pretty ambiguous. So test, test, test your Velocity!

Marketo's Known Visitor HTML (If Known Visitor, Show Custom HTML in Form Editor » Settings) feature is the obvious answer to a few questions:


  • How can I completely ungate an asset no form fillout at all, just a redirect if a web session is is already associated with a known person in my instance?
  • How can I show just a Download button if a session is already associated?
  • How can I auto-submit a form if a session is associated, so I can still switch between the different Advanced Thank You pages in my form setup?


Just redirect

This first one is easy: put a <script> that calls location.redirect in your KV HTML. (You do have to manage the redirect URL in JavaScript; it won't use the Thank You URL(s) as you're skipping the form post entirely.)


Just a button

The second one is straightforward, too.[1] In the Rich Text editor that pops up when you select Custom HTML, strip everything but the built-in {{form.Button}} token:





Auto-submit for Known Visitors

The third goal above isn't as easy as you'd expect. If you've dabbled in the Forms 2.0 JS API before, you might think you could do this (purposely screenshot-only so you're not tempted to copy it):




Nope, that won't work!


The reason is a classic bug-you-eventually-round-up-to-intentional: the JS API is not fully supported in KV HTML mode. Important methods like addHiddenFields work, and the whenReady listener itself works, but submit on the Marketo form object doesn't.


So we need to go back to the old-school method of simulating a click event on the button. It works just fine in all browsers, even if primitive:






var formEl = form.getFormElem()[0],
submitEl = formEl.querySelector(".mktoButton");;





[1] KV HTML does have an unexpected hidden field autofill (i.e. UTM tracking) gap that relates to the 2nd and 3rd bullets equally, but that's separate enough to be covered in another upcoming post.

If you've been around the block with Marketo Smart Lists, you know there's no Ends With operator, only Starts With and Contains.[1]


This puts a damper on a common need: accurately searching for an email domain (, or TLD (firmographic clues like .edu, geographic ccTLDs like .cn).[2]


Some have attempted extravagant combos of Contains and Not Contains, which require a whole lot of prep just to determine that... they don't work. (Read my comments on the linked post for some examples of how such approaches are broken.)


There's a much easier way: maintain a custom field, here called Matchable Email, that always holds the value of {{Lead.Email Address}} followed immediately by two quotation marks "":






Then, to do a domain search, search that Matchable Email field for Contains"", which is equivalent to searching the original Email Address for Ends With




Pretty easy, right?


Why two quotation marks (“”)?

The key is to add a sequence of characters to the end of the email address that can never occur in the middle of the email address, so Contains @{{domain}}{{characters}} is functionally equivalent to Ends With @{{domain}}.


Finding those appropriate {{characters}} is a lot harder than it sounds. For awhile I was lazily appending $ to the end, because I like the fact that the dollar sign represents end-of-line in regular expressions, so it was easy to remember. But the email address "$ke$$" — note the quotation marks around the mailbox part, it wouldn't be valid without those — is a Gmail account, but would match Contains$.


Yes, RFC 5321 is just that generous. There are so many crazy-but-valid email addresses, however inadvisable it would be to use them in the real world, that it's hard to find something that, without exception, can only occur outside of a valid address and so can be used as your anchor point.[3]


I think I've found that something, though. Two quotation marks in a row "" can occur inside an email address, but they can never be preceded by a character that is a valid part of a domain name.


Let me explain.


First of all, as you may already be confused by this part, it's possible to have a quoted mailbox name (called a quoted-string in the standard). That's how you can add spaces on the mailbox side of the @: "sandy spacebot" is a valid SMTP address.


You can also put quotation marks inside an already quoted mailbox name, but if you do so, you have to escape them with a backslash.  Thus "Clarence "Frogman""" is not a valid email address, but if you escape the quotes as "Clarence \"Frogman\"" it is valid. Even though this address has two quotes in a row "" (see the characters right before the @?) they are by necessity preceded by a \.  And the \ can never be at the end of a domain name. 


Therefore you can accurately search the Matchable Email field for a string that Contains"", knowing that that sequence of characters cannot be found at the start or middle of the value, only at the end.








[1] As a sometime database architect, I've never understood the technical reasoning and figure it must be just legacy-code-nobody-wants-to-touch syndrome. When searching strings, Starts With is faster than Ends With unless specific indexing is used; yet Contains and Ends With have equivalent performance — often terrible performance, don't get me wrong, but roughly the same either way. Plus, it's way easier to add indexing to speed up Ends With than it is to optimize Contains (an index on the reversed value in the first case, n-grams in the second case, FWIW). But here we are.


[2] My colleague EU points out that Marketo attempts to optimize a search for a domain-like pattern, one that begins with the character @, and turn it into an SMTP domain search. The problem is that it still doesn't work: The valid address "" will (as we would expect given the concept of contains) match both Contains and Contains so it doesn't successfully emulate Ends With. It will also false negative on Contains, which is just plain wrong.


[3] The way to do this in a technically complete manner is to add an ASCII control character (like ASCII 30 RECORD SEPARATOR, one of my faves) which is never allowed, not even in quotes. But while you can append such a character with a specially hacked Change Data Value, searching for those characters is, unless it's just a one-time thing, effectively impossible. So we'll have to make do with "".

Hadn't even heard of Appointlet until the other day, but when user SS mentioned he was trying to use the REST API to integrate an Appointlet widget with Marketo, I knew there had to be a better way. (There's almost always a more reliable + scalable alternative to server-to-server API calls for what should be browser-side integrations, Unbounce being another example.)


In line with services like ChiliPiper, TimeTrade, Calendly, et al. Appointlet is a service I wish I'd thought of because I'd be rich right now dedicated to scheduling. It interacts with a cloud calendar — O365 or Google calendar in this case — in real time to check availability, alerts reps of new bookings, and sends periodic reminders. (Again, no endorsement intended, just describing the published features... only spent 1/2 hr figuring out the API, so perhaps the platform might turn out to have humongous bugs, but it definitely looks useful enough so far!)


The Appointlet embed code gives you a button, which when clicked brings up the rep's availability:



And then a place for the lead to enter their personal info (more fields can be added but these are the defaults):



Naturally, when you're offering an Appointlet Book Now instead of a full Marketo form, the questions are:


  • How do you insert/merge the lead's info into Marketo?
  • How do you make sure past + future web activities are associated with the newly identified lead, i.e. how do you associate the Munchkin cookie with the lead, the way it works with a Marketo form?


The best answers are definitely not found in the Marketo REST API. Appointlet does offer outbound HTTP callbacks (accurately called webhooks, but they must not be in any way confused with Marketo's outbound 'hooks). So yes, you could set up your own gateway to receive said callbacks, and you could map them to the Marketo REST API endpoints (plural) that sort-of-maybe emulate a Marketo form post. But that means raw coding labor, new servers to maintain, and Denial of Service exposure. And no upside.


Instead the answer, as usual, is to simply post a Marketo form in the background, relaying the lead info from the Appointlet UI.


To do this reliably, Appointlet needs to have a client-side JS API. And indeed they do!


The Appointlet widget itself is rendered in an IFRAME, and like other sophisticated IFRAME-based embeds (the YouTube player, for example) the widget sends standard browser events to the parent document (that is, to the outer Landing Page) that include interesting info from the widget. We just have to listen for those events, add corresponding values to a Marketo form, and submit. Then we'll get a standard Filled Out Form activity in the Marketo Activity Log, which you can trigger and filter on like any other, and past + future Visited Web Page and Clicked Link on Web Page activities from that browser get merged in, too.


Step 1 of 3: Create a form

So first, set up a form that'll catch submissions from your Appointlet widget. (You can set up more than one form if you want to see cosmetically different Filled Out Form names for different pages, but it's not necessary and you don't want to create complexity.)


It doesn't need any fields at all, since we'll be populating the fields via API, but you can leave the default 3 fields in place. Just don't make any of them Required.



Step 2 of 3: Add the Marketo form to your page, with the <form> element not displayed

Inline style="display:none;" is easiest. With the embed code:


<form style="display:none;" id="mktoForm_787" class="mktoForm"></form>


With a Guided Marketo LP:


<div class="mktoForm" id="appointletForm" mktoName="Appointlet Hidden Form" style="display:none;"></div>


Or you can put it in a separate <style> which is more professional I suppose.


Step 3 of 3: Add the Forms 2.0 API custom JS

This is of course the meat of the solution.


MktoForms2.whenReady(function(mktoForm) {
var appointletUserConfig = {
allowedOrigins : [""],
formFields : [
appointletName : "first-name",
marketoName : "FirstName"
appointletName : "last-name",
marketoName : "LastName"


window.addEventListener("message", function(message) {

var appointletGlobalConfig = {
messageType : {
TYPE_BOOKING_CREATED : "booking:created"
pattern : {
RE_AL_POSTMSG : /^appointlet:/
err : {
ERROR_NON_ORIGIN : "Message received from non-Appointlet origin",
ERROR_BAD_JSON : "Message received from Appointlet API but could not be parsed"

var appointletEvent,
mktoFieldsObj = {};

isAlOrigin = appointletUserConfig.allowedOrigins.some(function(origin){ return origin == message.origin; });
if (!isAlOrigin) {

try {
appointletEvent = JSON.parse(, ""));
} catch (err) {
return console.log(appointletGlobalConfig.err.ERR_BAD_JSON);

if (appointletEvent.type == appointletGlobalConfig.messageType.TYPE_BOOKING_CREATED) {
mktoFieldsObj["Email"] =;
mktoFieldsObj[fieldDesc.marketoName] ={
return alField.field.slug == fieldDesc.appointletName;


Most of the code is no-touch, but there's a short config area at the top where you put your company-specific variables. From your Appointlet settings, get your Booking Page URL. That goes in the allowedOrigins config property:



Then the formFields property is an array that maps each Appointlet field name to its corresponding Marketo field name. (You didn't think it would be so easy that the separate products would miraculously use the same names, didja?) I filled in the First Name and Last Name mappings for you. Names of additional custom fields can be found via browser inspection, the Appointlet » Form Fields UI, and the SOAP API Name column of a Marketo UI » Field Management CSV export.


And that's it! Now, any confirmed Appointlet booking will post the form to Marketo.


What about the rest of the Appointlet setup?

That's on you. I found it very easy to set up an Appointlet account, link to a test Google Calendar, and grab the button code. But since I don't want to imply an outright endorsement, better to leave the rest of the product evaluation in your hands.

To switch up the Nancy Sinatra song, Booleans keep truthin’, when they ought to be falsin’.


As explored in earlier blog posts, when Marketo exposes Boolean fields in Velocity fields on the Lead/Person object, not on other objects they become Strings, not real Booleans.


And they're not even very Boolean-like Strings: they have values "1" and "" (the empty string) which in Velocity are both truthy values.[1]


As a result, you can't use a standard Boolean expression


#if( $isCustomer )
You're a customer.
Wouldn't you like to be a customer?


because everyone will match the first condition.


You have to be more exact, unfortunately making your code less self-documenting:


#if( $isCustomer == "1" )
You're a customer.
Wouldn't you like to be a customer?


Now, this more verbose version may not seem like a big deal, but I consider it to be poor programming practice because it relies on a “magic string”: a person reading your code has no way to know that "1" has some special significance and that the variable could not hold any other string value. That is, any Boolean-ish thing should be an enumeration only allowing 2 values (one representing true and one representing false, whatever those values might be) but since it's a freeform String it has no such restriction.


So here's something you can add to your global {{my.velocityIncludes}} token. (You do have such a token, don't you?  All the cool kids do.)



#set( $mktoBoolean = { "1" : true, "" : false } )



With that one-time include (put it in the <head> of your templates) now you can refer to those Boolean-ish fields like so:


#if( $mktoBoolean[$isCustomer] )
You're a customer.
Wouldn't you like to be a customer?


Now it's clear that you're using the variable as a Boolean.


I've recently decided this simple method is good enough. In the past I'd been  using a list (manually maintained in {{my.velocityIncludes}}) of known Boolean field names, then “massaging” those fields on the lead to turn them into real Booleans before using them. But that takes prep work and IM(new)O isn't worth it.


Code breakdown (if you need it)

The snippet above is just the kind of thing that can make new devs think VTL syntax works a certain way, and then try to adapt it to other scenarios only to find syntax errors.


So let me explain exactly what's happening, as short as it is.


First, let me add line breaks for readability:


#set( $mktoBoolean = { 
  "1" : true, 
  "" : false
} )


By using Velocity's map literal syntax we're creating a simple Map object with 2 keys.


(Informal/imprecise terms for such an object are Hash, HashTable or Dictionary, and the exact type is LinkedHashMap. Also feel like noting that even though Velocity's map literal syntax looks the same as JavaScript's object literal syntax, it creates an object that is different in one critical way, though that difference isn't relevant here.[2])


The keys in the Map can have any values, even including null, and can certainly include any kind of string, including an empty string. You access the keys in a Map using .get, bracketed-property syntax or, when it can be parsed unambiguously, dot-property syntax.


So for a more general example, if we defined this Map:


#set( $someOtherMap = {
  "FaveFruit" : "apple",
  "FaveVeg" : "broccoli"
} )


Then we can use one of 3 equivalent ways to access the person's favorite fruit:




Those all address the same key and will all show apple.


In the specific case of the $mktoBoolean Map, we can't use the 3rd option of dot-property syntax though, because $mktoBoolean.1 isn't a valid expression in Velocity Template Language since it starts with a number.  We're limited to




This limitation isn't a big or surprising deal, by the way. Just one of a zillion cases where certain accessing syntax might be unusable, but that doesn't mean the initial definition of the variable was wrong. Sometimes you end up limiting the ways to refer to object keys — another common case is when a string key has a space in it ({ "My Other Car" : "Lambo" }), which also doesn't work with dot-syntax so you have to use $someOtherMap["My Other Car"] or $someOtherMap.get("My Other Car") — but you get other benefits in return.


Aaaaanyway, so we have a Map with 2 keys, both Strings. The value of each key is a real Boolean: I used the literal Boolean values true and false, not Strings.


That means we can use bracket-syntax to access the corresponding key in the Map, which will return a Boolean.  When I do


#if( $mktoBoolean[$isCustomer] )


I'm getting the value from the $mktoBoolean Map that matches the key $isCustomer. That is, within our reserved world of "1" and "", I'm always getting either $mktoBoolean["1"] or $mktoBoolean[""], and those values are both Booleans so the result can be used clearly and consistently.


Hope that all made sense!






[1] In Velocity's underlying Java environment, these would both be String instances and thus neither true nor false; they'd throw fatal errors if you tried to use them as Booleans. (Unlike other languages you might have used, in pure Java only an actual java.lang.Boolean can be used in a Boolean expression; there's no concept of truthy or falsy strings or numbers.)


But VTL is an often frustrating “friendlier” dialect on top of Java which minimizes visible errors. In VTL, #if ($someStringVariable) won't chuck an error message into your email; yet the logic it uses is basically “everything but exact Boolean false or null is true” which can be very misleading.


[2] That way being that the keys in a LinkedHashMap are ordered. Key order has no bearing on the way $mktoBoolean is used in this scenario, but it's a classic source of confusion in object-land. In JavaScript, object literal syntax creates a plain JS Object, which is unordered and is roughly like a simpler Java HashMap.

Custom Forms 2.0 JavaScript behaviors are best managed via an external JS file and <script src="/yourfile.js"> tag in the LP. This allows your code to be updated without touching any other part of the page or form, sharing behaviors across forms & pages, and so on.


At tonight's NYC MUG meeting, my man Nick asked if you could put the custom behaviors JS into the form itself via Form Editor.


Indeed, if you want a quick-and-dirty JS enhancement, and you don't want to figure out where in the LP to put your <script> tag[1] or talk to your webmaster, yes, it's possible to use the Forms JS API from a Rich Text area. If you insist.


That should be good news! The only, let's say, more guarded news is that you have to do it right or can get craaaaazy results.


There's one major concern and one minor concern:


    (1) Major: You must ensure the code in your embedded <script> only runs once. Because of the curious way in which forms are rendered, this is a harder than you probably think.

    (2) Minor: You have to completely hide the Rich Text area so it doesn't show up in the layout, which means hiding its entire form row (margins, padding, et al.).


(2) is easy to accomplish with some CSS. So let's wait on that.


Run only once

Let's see what happens if we naïvely add a RT area containing a <script> with a simple whenReady listener function inside. Note I've put some text at the top of the RT so it looks in-use in Form Editor (“[Form Behaviors JS - Do Not Delete]”). Such text is optional but recommended; otherwise, the RT might be accidentally deleted as it looks empty until you double-click it.




When you load a page with just that one Marketo form in it, you might see the following Console output:




That's the same function run 4 different times even though we only have one Rich Text. Really bad if you're adding event listeners!


This happens because of the way the <form> DOM is built out. As the <script> is ejected and injected into the page repeatedly, it ends up executing its code repeatedly.


And that's not the same function running, technically speaking, but 4 functions that happen to have the same code running in a row. Because they're all separate from each other, they don't share a parent scope in which you could add behaviorsLoaded = true or something like that.


Instead, you can set an HTML data- attribute on the <form> element, since that will of course persist across executions. Each time the code runs, check for the attribute and return immediately if it's already true:




In copy-and-pastable form:


[Form Behaviors JS - Do Not Delete!]
<script type="application/javascript">
  var formEl = form.getFormElem()[0];

  if( formEl.getAttribute("data-inline-behaviors-loaded") == "true" ) {

  formEl.setAttribute("data-inline-behaviors-loaded", "true");

  // now continue
  console.log("Doing something special");


Now you can see the meat of the code only runs once:




Back to CSS

If you make the Rich Text area the first row in Form Editor, it's easy to select and hide:






.mktoFormRow:nth-of-type(1) {
    visibility: hidden;
    position : absolute;


I'd typically recommend a more resilient method of selecting the right row. But that would likely involve loading my FormsPlus::Tag Wrappers helper JS first… problematic if the whole idea is to consolidate the JS all within the form!




[1] As a reminder, when not using the Rich Text method described here, put the behaviors <script> just inside the closing </body> tag on a Marketo LP, or anywhere after the form embed code on a non-Marketo LP.

As you dive into the world of webhooks for advanced database tasks, the number of webhook definitions in the Admin UI can get pretty crazy. I've seen instances with 100 different ’hooks!


Many webhooks do need to stand alone, but some come in pairs or groups. For example, an Add to Event Registration Counter webhook needs a companion Remove from Registration Counter. A Lookup in Suppression List webhook running against an external db likely means Add to Suppression List is also defined. And so on.


To consolidate webhooks and make the UI (and your code) more manageable, a cool trick is to send the {{Campaign.Name}} token along with the hook, so the remote service can decide which direction/action to take, and you only need to define one webhook to cover a group of related actions.


For example...

I have a webhook that maintains, in a Textarea custom field, all the Static Lists a lead is currently in. (This view is notoriously hard to get at, in both the UI and the API, unless you flatten it onto a field like this.) The field uses a semicolon-delimited format, as is typical for such things: Apples;Peaches;Pumpkin Pies.[1]


Naturally, the webhook is triggered on Added to List and on Removed from List. So I have those 2 campaigns set up with informative names:





And the onRemove flow is exactly the same as the onAdd flow pictured above. They both call the same webhook, rather than there being 2 webhooks, one for each direction.


Then within the webhook itself, I can detect the direction because {{Campaign.Name}} is included in the payload (in addition to {{Trigger.Name}}, which is the List name):




This I find infinitely more manageable than the alternative, since the onAdd and onRemove handlers sit right next to each other.


Obviously, the exact way your webhook service switches between different directions/modes depends on your architecture. Since this webhook uses JavaScript, there's a literal JS switch statement, while in other cases you might pass {{Campaign.Name}} in the URL path or query string (the dotted Program Name.Campaign Name could represent the dot-path to different Java class methods, that would be cool!).




[1] In this case, a single Static List can't have a semicolon within its name, for obvious reasons.  You can switch delimiters as necessary.

Isn't it annoying to add a Wait step before sending a registration confirmation, crossing your fingers that the provider has phoned home with the {{Member.Webinar URL}} token?


I've always hated this step. Not only because of the experience for the end user, who might think you've forgotten about them, but on a pure technical level. Techies hate timers that are “probably sufficient”: we want tools that are predictable, not best-case guesses with fatal downsides. Not only might a 15 minute Wait not be sufficient under load,  there's no Wait that will fix a major problem with the integration, so users will eventually get an email with a big blank in the middle anyway.


What if I told you you don't need to hold off on such emails at all?


Instead, set up an LP in the webinar program that redirects to the {{Member.Webinar URL}}. Send them a link to that LP with no delay. Unless it's very close to the webinar's start time, most people aren't going to click a Join Webinar link right away, so you've got lots of breathing room for the API to sync up the {{Member.}} token.


When they do click the email, you'll be ready for them and will immediately bounce them over to the webinar, hands-free.  If it does happen that the URL isn't yet ready when they click, well, you weren't going to get it to them any quicker with an arbitrary delay! You can give them a countdown timer and reload the page in 30 seconds, and you can send them a follow-up email since you know they're eager (that second email might as well have a Wait step).


Here's the briefest possible copypasta to add to that LP:


<style type="text/css">
.webinar-pending-note { 
  display: none; 
[data-url-ready-state="pending"] .webinar-pending-note { 
  display: block; 
var memberWebinarURL = "{{Member.Webinar URL}}";
if ( memberWebinarURL ) {
  document.location.href = memberWebinarURL;
  // just for completeness
} else {
<div class="webinar-pending-note">
Your customized webinar URL is still being assembled. 
Please refresh this page or re-click the email link in a few minutes!


More ideas

Once the LP is in place, you can extend the user experience in several ways:


1. As noted above, you can add a periodic document.location.reload(true) in case they did hit the page before the API phoned home.

2. You can adapt some of my Redirector Page JS to add a progress indicator before reloading. Note sending leads to the Webinar Redirector LP also enables Munchkin tracking, which is a benefit in its own right.

3. You can add Agical Add to Calendar links to the page, referencing the {{Member.Webinar URL}} using the alternate separator syntax originally designed for this very case.

4. If you want to send the lead an Add to Calendar link that works from the start, then reference the Redirector LP's URL in your .ICS file or Agical link, not the {{Member.Webinar URL}}.

5. You can retarget the lead with rich contextual content if they land on the page after the webinar is over (check another {{my.}} token that stores the event date) or well before it begins, instead of taking them to this beautiful page:


For any Velocity project, I've taken to offering clients a separate config token, call it {{my.ThisIsWhereYouChangeStuff}}, where they can manage some output settings without having to email me all the time.


Then there are one or more {{my.PleaseDontChangeAnythingInHereItsFragile}} tokens with the meat of the code.


Velocity #define directives are really handy for the config token. They're a bit more fault-tolerant than #set statements, where the non-technical person has to remember to escape quotes, close parentheses and such.


That is, instead of:


#set( $baseURL = "" )
#set( $linkText = "Say \u0022Hello\u0022 to our new preference center." )


I give them a token like so:


#define( $baseURL )
#define( $linkText )
Say "Hello" to our new preference center.


As long as they leave the #define/#end lines alone they can change anything in-between (especially good for multiline text, as you might imagine).


There's a little trick to using #define, though, and that is like everything in Velocity, it preserves whitespace.  What whitespace, you may ask? Well, look at the end of this line:


That has a carriage return + line feed (or just LF, depending on the OS) at the end.


So if I output a link like so:


<a href="${baseURL}">${linkText}</a>


The email will contain:


<a href="
">Say "Hello" to our new preference center.


Instead of what you intended:


<a href="">Say "Hello" to our new preference center.</a>


Which is bad because it will wreck your links even though you may not even see the wreckage in Preview because of the way HTML itself swallows line breaks.


Now, you can suppress the trailing whitespace by adding a comment ## at the end of the line


but I daresay that's not an improvement, since the idea is to offer this token as a not-too-fragile place for a non-technical person to make adjustments, and adding ## is something they're bound to forget or mess up.


So what you want to do is let them enter text in as close to free-form fashion as possible. Then in your code, strip out extraneous whitespace at the beginning or end to be tolerant of minor messups.


How trim() works

The documentation of the trim() method in Java, which exists on any Java String and therefore on any Velocity String is almost lovable in its complexity.


trim() does exactly what we want, but you have to understand the ASCII table to know that! Not that a programmer shouldn't understand ASCII, but it's a particularly circuitous explanation IMO:


[L]et k be the index of the first character in the string whose code is greater than '\u0020' (the space character), and let m be the index of the last character in the string whose code is greater than '\u0020'. A new String object is created, representing the substring of this string that begins with the character at index k and ends with the character at index m-that is, the result of this.substring(k, m+1).


Let me put that in clearer terms:


If a contiguous block of characters between ASCII 0 and ASCII 32 is found at at the beginning and/or end of the string, the whole block is removed.


ASCII 0 through ASCII 32 means the nul (0) through space (32) characters, inclusive. In that range are the quite common carriage return (13), line break (10), and tab (9) characters, and some more obscure ones like vertical tab (11).[1]


So though it only explicitly mentions the space character \u0020 (hex 20 is decimal 32), which you're probably familiar with as %20 in URLs, in fact it covers line breaks as well. If there's a long intro or outro of spaces, line breaks, and tabs, trim() will clean 'em all out.


trim()-ing what's inside a #define

So trim() is perfect, but you can't simply do this:


<a href="${baseURL.trim()}">${linkText.trim()}</a>


That'll throw an error. The reason is that any #define, when you address it directly, is a Velocity-specific Block$Reference, not a generic java.lang.String.


A Block Reference doesn't itself have a trim() method. But it does have a toString() method. (In fact, toString() is called under the hood when you output a plain ${reference} in Velocity, otherwise you couldn't output it at all.)


So I know this was long-winded but hopefully you learned something you need:


<a href="${baseURL.toString().trim()}">${linkText.toString().trim()}</a>


And you're done!





[1] But not all whitespace characters, since some as common as non-breaking space (the famous &nbsp; in HTML) are above ASCII 32. And over in JavaScript, the almost-identically-purposed trim() does strip non-breaking space. Is there nothing in programming that's not complicated when you care to learn the details? ☺

Quick Velocity learner EC asked a seemingly simple question:


How do you sort Velocity objects based on a Boolean field, such that false values come first?


What makes it not-so-simple is that it's actually 3 questions in one!


Realize that something labeled as Boolean in Marketo (in the Field Management or Custom Objects admin sections) may or may not end up as a Boolean field in Velocity.


Booleans on Leads

As I've delved into before, regardless of the data type set in the database, Lead/Person fields in Marketo become String properties of the $lead object in Velocity:

  • Dates and DateTimes are not dates in Velocity, but merely date-like Strings (that's why you need to go through all that pain to convert them).
  • Integers are number-like Strings (which can wreak havoc on comparisons if you don't realize it).
  • Boolean fields are boolean-ish Strings. They have the possible values "" (empty String) if the original Boolean was false and numeric String "1" if true.


But the same doesn't hold for Boolean fields on other objects besides the Lead.


Booleans on Oppties and Salesforce COs

On Opportunities and other SFDC Custom Objects, bizarrely, Booleans become a different type of String, which has the possible values "0" for false and "1" for true (this is admittedly a more traditional conversion than empty String and "1" but why, oh why, must it be different?).


Booleans on Marketo COs

On Marketo Custom Objects, Booleans are presented as true Booleans! As in real Java Booleans with the reserved values true and false, the way we dream they'd be everywhere.


Wow, that's confusing!

Uh-huh. As I'm sure you realize, such differences all have to be considered for sorting.


So the 3 questions EC was implicitly asking were:


     (1) How do you sort objects based on a String field that can be empty string "" or numeric string "1", such that "" comes first?

     (2) How do you sort objects based on a String field that can be numeric string "0" or numeric string "1", such that "0" comes first?

     (3) How do you sort objects based on a Boolean field that can be true or false, such that false comes first?


Now, at the code level, the answer to all 3 questions happens to be the same. But this is coincidental because it derives from different Java sorting rules. The fact that there's one answer that covers these 3 cases must not be interpreted to mean any ways of representing true and false will follow the same sorting logic.


At any rate, that answer is:


#set( $sortedList = $sorter.sort($originalList, "fieldName:asc") )


Because ascending (:asc) is the default order in SortTool, this can be shortened to:


#set( $sortedList = $sorter.sort($originalList, "fieldName") )


So if you have these 3 objects in a list $originalList:


  "Name" : "A",
  "IsActive" : false,
  "ExternalCreatedDate" : "2017-07-01"
  "Name" : "B",
  "IsActive" : true,
  "ExternalCreatedDate" : "2017-08-01"
  "Name" : "C",
  "IsActive" : false,
  "ExternalCreatedDate" : "2017-09-01"


And you do:


#set( $sortedList = $sorter.sort($originalList, "IsActive") )


Then $sortedList will be:


  "Name" : "A",
  "IsActive" : false,
  "ExternalCreatedDate" : "2017-07-01"
  "Name" : "C",
  "IsActive" : false,
  "ExternalCreatedDate" : "2017-09-01"
  "Name" : "B",
  "IsActive" : true,
  "ExternalCreatedDate" : "2017-08-01"


And the same code also works if IsActive uses either of the 2 "boolean-ish" String conversions instead of true Boolean.


But again (and I'm going to pound you over the head with this) it's a coincidence. If Marketo happened to use yet another way of String-ifying Booleans, you might need to sort descending ($sorter.sort($originalList, "fieldName:desc")) or use another sorting method entirely to get the falses to the top.


The challenge

Now, the fun part. Rather than jump right to Part II (where I go deep into the technical details) I'll keep that follow-up post as a draft for a week, until September 18, 2018. 


From now 'til Sept. 18, if you can answer the below challenge correctly, I'll give you a major shout-out in the follow-up... and hey, I may even have some side gigs for you! (Leave your answers in the comments, and please only try once per person.)


The background:


If Marketo used the Strings "-1" for false and "+1" for true then you'd need to use :desc instead of :asc. (And there's no standard that says they couldn't String-ify Booleans this way.) And same if they'd used symbols "" and "" , which appropriately denote truth and falsity (verum and falsum) in logic. Ditto if they'd used the Unicode characters for thumbs-up and thumbs-down.*


The question:


Why, in these 3 alternate cases, do you need :desc to get the originally false values to come first?


Be precise... no credit for "Because that's what Velocity requires."



*Would've displayed the characters here, but the Jive editor strips them

out! You can see them on the original version of this post on my blog.

Can't believe someone other than me has thought about cryptographic techniques in Marketo! But in this recent Nation thread, a user asked about attaching the SHA-256 hash of a recipient's email address to (email) links.

The aim being to send a secure representation of a lead's email to an external (non-Marketo) site, where it could be looked up as part of a 3rd-party subscription center.

In other words, a link would look like this:

The target site's database would hold pre-computed hashes of all their registered users' emails, the hash being stored in a separate column from the email itself.[1]

A match would then be done from hash-to-hash (the way hashes are always used, since they can never be reversed to the original input) to load the user's record.

Mind you, I have no idea why this would actually be done as opposed to sending the URL-encoded email address:

Or the easily reversed, Base64-encoded (not encrypted) address, which wouldn't require a hash lookup:

Being a realist, I suspect the justification is merely that they want the links to look more “technical” than they would with Base64 (?) and they don't truly have any need for security (as it makes little sense to care about the security of the recipient's own email address![2]). UPDATE: OP says the client now wants AES encryption instead of hashing, so they really are trying to securely transport stuff in the URL. That gives me a chance to bring out the big guns and show you how encryption is done in VTL... at some point.

Anyway, should you need it, here's a utility Velocimacro to gen a SHA-256 hash:

    HashTool in VTL v2
    @copyright (c) 2018 Sanford Whiteman, FigureOne, Inc.
    @license MIT License: all reproductions of this software must include the data above
#macro( SHA256_v2 $mktoField )
## reflect some dependencies
#set( $Class = $context.getClass() )
#set( $java = {} )
#set( $java.lang = {
     "StringBuilder" : $Class.forName("java.lang.StringBuilder"),
     "Appendable" : $Class.forName("java.lang.Appendable")
} )
#set( $ = {
    "MessageDigest" : $Class.forName("")
} )
#set( $java.util = {
    "Formatter" : $Class.forName("java.util.Formatter").getConstructor($java.lang.Appendable)
} )
## get an MD and make it go, then (important) reset MD singleton for reuse
#set( $MD = $"SHA-256") )
#set( $digestBytes = $MD.digest( $mktoField.getBytes("utf8")) )
#set( $void = $MD.reset() )
## gen hex string representation
#set( $hexString = $java.lang.StringBuilder.newInstance() )
#set( $hexer = $java.util.Formatter.newInstance($hexString) )
#foreach( $B in $digestBytes )
#set( $void = $hexer.format("%02x",$B) )
## return uppercase, but note a hex string should be treated as case-insensitive

After including the above in its own token (best to not pollute your user code with utility functions) use the macro like so:

#set( $emailHash = "#SHA256_v2($lead.Email.toLowerCase())" )
<a href="${emailHash}">Click here</a>

Note that I toLowerCase()d the Email before passing it, as that's appropriate for that particular field (as I've written about before, though SMTP addresses are actually case-sensitive, they're commonly matched case-insensitively for sanity's sake).


[1] Hope I'm not hoping for too much here. They'd better be already pre-computing hashes for all the stuff in their database, or this idea goes from merely frivolous to very bad. If they didn't pre-hash, they'd have to hash every record in the database, on-the-fly, for every lookup. This would absolutely destroy performance and be a sign that the back end was not well thought out.

[2] Long as the destination site runs SSL. But if the site doesn't run SSL then the connection could be intercepted and you have a lot worse problems than showing the attacker an email address.

The Forms 2.0 Email type — just like a standard HTML <input type="email"> — won't throw an error on email addresses like:





That is, domains with only a single DNS label to the right of the @ sign (@example), rather than multiple labels separated by dots ( or, are valid entries.

Confusing, sure. But no, it's not a bug. A mailbox @ a single-label domain isn't an “incomplete” or “partial” address, and while most such addresses happen to be invalid on the public internet, it's impossible to know whether they're valid using JavaScript alone.

To understand why, you have to know more about how SMTP domains are looked up in the global DNS.[1] (To my admittedly unreasonable dismay, nothing about SMTP or DNS is taught to marketing students!)

A laughably brief overview of SMTP and DNS

This is silly to try to go over quickly. But here are the basics:

  1. (1) For an email address to be theoretically routable over the public net, it must have a domain part, the part to the right of the @.
  2. (2) For an email address to be factually routable over the public net, the domain needs to have an MX record[2] in the global (public) DNS.
  3. (3) A web browser cannot perform arbitrary DNS lookups on its own, so it cannot know whether a domain actually exists, let alone what records are in the DNS zone. It can only know if it's well-formed: that is, if it matches the string syntax rules for a domain.
  4. (4) A single-label domain like gmail (or, for that matter, a single label like com) is no less a well-formed domain than one with multiple labels, like or looked up, these will differ greatly in terms of being private (assigned year-by-year to a company or other entity) or public (registered semi-permanently to a country or independent NIC operator).[3] And they may or may not be registered, or even legally eligible for registration, at a given point in time. But they are nevertheless all domains.
  5. (5) A single-label domain, if it exists in the global DNS, may also have an MX record (and some, as we shall see, do). It's a rare case, yes — but it's technically valid, and since form inputs don't perform DNS lookups, there's no way to know whether the particular single-label domain somebody entered is routable or is a dead end.

In sum, the only thing a web browser alone can know about a domain, without invoking a remote lookup service, is that it follows proper syntax. It can't have commas instead of dots; it can't have anything other than a tiny subset of ASCII characters (though it may be displayed as if it has more exotic characters, which is another topic); it can't have an individual label longer than 63 characters; it can't have 2 dots in a row; the total length including dots can't exceed 253. That's pretty much it: one string that matches those requirements is as valid as any other.

You might be getting it already, but let's go further

From the intro above, you should already have your mind opened.

But what exactly do I mean when I say a single-label domain may still be a publicly routable domain? Most likely, you've never sent or received mail from joe@gmail, only But that's just because you haven't sent enough email. (Like, you haven't personally sent billions of messages!)

As of today, these TLDs have MX records, meaning the email address user@tld is publicly routable:

.ai .ax .cf .dm .gp .gt .hr .km .lk .mq .pa .sr .tt .ua .ws

These are mostly small island countries, like Anguilla, Dominica, and Trinidad and Tobago. With full respect to the governments of those countries, with relatively limited budgets they may have set the records up accidentally. But the list also includes Guatemala (.gt), Ukraine (.ua) and Sri Lanka (.lk) — so we must assume it's no accident that you can address mail to sirisena@lk.

I get that you still want to “fix” your forms, though

Even if you believe everything above, you probably still want to hand-wave it and require multi-label domains! So here's a little snippet if you insist:


MktoForms2.whenReady(function(form) {   
  form.onValidate(function(nativeValid) {      
    if (!nativeValid) return;      
    var currentValues = form.getValues(),         
    formEl = form.getFormElem()[0],         
    emailEl = formEl.querySelector("[name=Email]"),         
    RE_EMAIL_ASCII_PUBLIC = /^[a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)+$/;      

    if (!RE_EMAIL_ASCII_PUBLIC.test(currentValues.Email)) {         
        "Must be valid email. <span class='mktoErrorDetail'></span>", 
    } else {         

Obviously if you already have custom onValidate behaviors, you have to integrate this code with those. (When you're flipping the Forms 2.0 global submittable flag on and off, you need to make sure all conditions are considered together.)

A deep note about commercial gTLDs

OK, there is one glitch in my claim that browsers are completely justified in accepting all single-label domains in Email inputs.

In fact, newfangled domains like .space and .lawyer and .futbol are expressly prohibited from having MX records at the top level. From the guidebook for applying to run one of these “new gTLDs”:

ICANN receives a number of inquiries about use of various record types in a registry zone, as entities contemplate different business and technical models. Permissible zone contents for a TLD zone are:

• Apex SOA record.

• Apex NS records and in-bailiwick glue for the TLD’s DNS servers.

• NS records and in-bailiwick glue for DNS servers of registered names in the TLD.

• DS records for registered names in the TLD.

• Records associated with signing the TLD zone (i.e., RRSIG, DNSKEY, NSEC, and NSEC3).

In other words, non-essential records like A or MX records are not permitted at the apex level of these new gTLDs.

The guide does go on to imply that another record type might be permitted by special exception, but it seems awfully doubtful that any would ever be approved:

An applicant wishing to place any other record types into its TLD zone should describe in detail its proposal in the registry services section of the application. This will be evaluated and could result in an extended evaluation to determine whether the service would create a risk of a meaningful adverse impact on security or stability of the DNS.

So, if you know you're dealing with one of these new gTLDs, .lawyer for example, you in fact can be sure without a DNS lookup that joe@lawyer will not be routable over the net, since that top-level domain is not legally allowed to accept mail.

So browsers could, in theory, throw an error on <user@new gTLD> (since they do have a copy of the Public Suffix List internally and don't have to hit DNS for that). But, well, they don't. Go figure!


[1] Plus, it helps to understand that SMTP delivery doesn't even require a DNS lookup. The SMTP standard predated DNS by a few years, and even today billions of messages are passed from server to server without DNS being consulted. But I'm going to ignore this other contributing factor today.

[2] Yes, for the purists, an MX or an A record. I'm not trying to overwhelm people here.

[3] Or of course the 3rd variant: private on the pure DNS level, as they're children of a public TLD, but to be treated as public for purposes of browser security. This is why the PSL exists: it tracks the effective public domains, a list which can't be derived via DNS alone.

When choosing a Thank You/Follow Up URL, Form Editor presents 3 largely self-explanatory options:


Choosing Landing Page will give you a nice dropdown of all your approved Marketo LPs.

Choosing External URL and entering the full URL of a Marketo LP seems to work, too — but it's a bad move. Form Editor won't tell you why, but I will.

The problem

If you choose Landing Page or Stay on Page, Marketo uses a special read-your-write feature to ensure Pre-Fill on the next pageview.[1]

If you choose External URL, Marketo looks up the URL in your known LPs but it doesn't look in your Redirect Rules.  If the URL is not seen as a core LP URL, even if it does lead to a Marketo LP in reality, then the read-your-write Pre-Fill cache is not enabled.

What is “read-your-write” and why should I care?

“Read-your-write” or RYW is a desirable, but not always supported, feature of distributed systems. (A distributed system is any system that processes data at multiple tiers, like we all know Marketo does.)

RYW, in a nutshell, means:

If a user thinks they've made a change to the back end, then show them the new data as if it's been fully saved, regardless of whether system(s) may be still filtering and processing — or even discarding! — the data in the background.

It doesn't mean anybody else sees uncommitted data (there are some cases in which it won't be saved, so you don't want to propagate bad info more widely than necessary).

It means that for a less confusing user experience, you let a user see their own requested update immediately, instead of forcing them to see older saved data for a few seconds/minutes (even though showing saved data would technically be more accurate end-to-end).

Marketo implements read-your-write via the special query parameter aliId. It's a numeric identifier for a set of submitted form values, and it's usable regardless of whether the values are actually saved. When a page has an aliId in the URL, it's capable of (for the most potent example) filling a second form using the data from a first form, even if it was submitted mere milliseconds before.

Back to Form Editor

When you choose External URL in Form Editor, Marketo tries to append the aliId intelligently, but it can't know about your redirects (especially 3rd-party redirects) or anything that might disguise the Marketo-ness of the destination URL.  As a result, the next LP someone views may show their session's old data (maybe including someone else's email) or no data at all (the session still being effectively anonymous). You don't want that! So choose Landing Page if it's indeed a Marketo LP.

Other complications

When you use the Forms JS API onSuccess method to choose a Thank You URL dynamically (a very powerful option) set the form to Stay on Page in Form Editor, as this will ensure the aliId is appended.

Then clip out and append the original query string, which will include the aliId, to the dynamic URL.

A quick-and-dirty way to append the original query is like so:

    var originalThankYouDoc = document.createElement("a");
    originalThankYouDoc.href = tyURLFromFormEditor;
    var dynamicThankYouURL = "" +;
    document.location.href = dynamicThankYouURL;
    return false;

If you're using a fuller-featured URI parser/builder already, like uri.js, use that instead of the Link/Location method (though the breakdown works perfectly, since it's a key browser function).

Just don't write your own parser… I don't want to have to holler at you again!


[1] When you use Stay on Page the aliId is attached even if you're using the form embed and the page is a non-Marketo page. It's not fully honored in this case (since Pre-Fill isn't supported with the embed) but it's better to have it there than not.

Little-publicized, to the point that even Support might not be in the loop: the Visibility Rule Contains operator does allow multiple values.

The Contains value is interpreted as a full-fledged regular expression, so use the pipe character | to separate sub-values:


In fact, because of this unexpected/undocumented power, you must anchor an expression to avoid surprises:


Otherwise, it's doing a partial match of the full value against each of your pipe-delimited alternatives. This might not be a problem on your forms right now, but if it becomes a problem, you wouldn't know about it immediately.

For example, if you were using uppercase state names, then

     Contains KANSAS|IDAHO

would match both ARKANSAS and KANSAS, and you should use

     Contains ^(KANSAS|IDAHO)$


You can also of course do more fancy matches with regexes if you want. And Not Contains is also a regex!

There are 2 ways to access references (i.e. variables) in Velocity.


(1) Simple/shorthand notation, prefixed with a $:




(2) Formal/longhand notation, prefixed with $ and wrapped in {} (curly braces):




Simple notation should be your default. Formal notation should only be used when there's no alternative. As the Velocity docs say:


In almost all cases you will use the shorthand notation for references, but in some cases the formal notation is required for correct processing.


I'd go further: when used unnecessarily, formal notation can make your code confusing and fragile. Unfortunately, Marketo's script editor doesn't encourage best practices, because when you drag a field from the field tree to the editor canvas, it's automatically  wrapped in ${}:



My recommendation: remove the curly braces right away, and only restore them if it proves necessary during development.


What can go wrong?

A recent Community post shows how confusing things can get when you use formal notation unnecessarily (note how the poster titled it "... tokens behave unexpectedly" when it's actually established behavior, if not well-circulated).


The catch: a reference enclosed in ${formal.notation} cannot be chained with a method/property outside the curly braces.


OK, that probably didn't make sense unless you're fluent in OO-speak! Let's look at some examples.


This does work with simple notation:


#if( $lead.FirstName.isEmpty() )


It doesn't work if you only enclose part of the expression in formal notation:


#if( ${lead.FirstName}.isEmpty() )


Sure, it would work if you happened to enclose the entire expression in curlies


#if( ${lead.FirstName.isEmpty()} )


but you should just use simple notation instead, because it's harder to mess up during refactoring.


Don't believe me? (You probably do, or I'd like to hear who's more authoritative about Velocity.) Consider what happens when you move from a not-best-practice, but syntax-error-free, comparison using double-equals ==:


#(if ${lead.numberOfProducts} == 0 )


to a more forward-looking equals():


#if( ${lead.numberOfProducts}.equals(0) )


Suddenly, your Velocity token stops working because you can't have a . after a }. It may be clear, to you as a human, what you meant, but VTL doesn't allow it. If you'd used simple notation from the start you wouldn't have to make such adjustments.


When do you need to go formal?

Formal notation should only be considered inside strings or output, and only used when the separation between variables and static text would otherwise be unclear.


If you're in a line of code starting with #if or #set chances are very slim that you should be using formal notation.


Here's one case where it is necessary...


Read the full post on

Filter Blog

By date: By tag: