Skip navigation
All Places > Products > Blog > Author: Sanford Whiteman
1 2 3 4 Previous Next

Products

58 Posts authored by: Sanford Whiteman

While you should be loading embedded forms from your Marketo LP domain, it turns out just changing the URL is not enough.

 

Using a non-Marketo-owned domain to load forms2.min.js and call loadForm() seems to avoid collateral damage from Tracking Protection (Firefox’s built-in feature or the equivalent plugin for another browser). The form will at least show up on the page if you do that.

 

But – I’m sure you will be dismayed to learn – the form still won’t post under TP! You also need to upload a file to Design Studio and add a tiny bit of JavaScript. Then and only then are you good to go.

 

First, download (Right-click, Save link as) this file:

 

marketo-xdframe-relative.html

 

Next, upload the file to Design Studio. To be clear, you’re not creating an LP, you’re uploading the file as static HTML. This may not be something you’ve done before (you probably upload images, CSS, and PDFs) but Marketo actually supports any kind of file.

 

Screenshot after it’s been uploaded to Design Studio:

 

 

Then replace your embed code with this ever-so-slightly different version:

 

<script src="//pages.example.com/js/forms2/js/forms2.min.js"></script>
<form id="mktoForm_9999"></form>
<script>
MktoForms2.setOptions({
formXDPath : "/rs/123-ABC-456/images/marketo-xdframe-relative.html"
});
MktoForms2.loadForm("//pages.example.com", "123-ABC-456", 9999);
</script>

 

Where:

 

  • 123-ABC-456 represents your Munchkin ID
  • pages.example.com represents your Marketo LP domain
  • 9999 is o’course your form ID

 

A live demo (though all the code is above):

 

MktoForms2 :: XDFrame relative path for Tracking Protection

 

The why

Using your LP domain enables forms to show up. But there’s a special asset, only used upon submission, that will still be blocked by default. Naturally, this only reveals itself when you run end-to-end tests.

 

See, embedded forms use an IFRAME message relay (XDFrame) for cross-domain posts (there are other ways to do it, but they’re not as backward-compatible and the IFRAME method works fine + fast in all browsers, even back to IE8).

 

But the IFRAMEd document loads forms2.min.js from an absolute URL with your instance hostname, i.e. app-xxnn.marketo.com:

 

Oops! Tracking Protection ain’t gonna let that load, for the same reason it doesn’t like a Marketo domain in the main document. (You can’t avoid TP by using an IFRAME, that would be silly.) As you can see, the replacement XDFrame file uses a relative URL (/js/forms2/js/forms2.min.js), so it loads from your Marketo LP domain like the rest of the assets. Presto! No marketo.com for TP to get all paranoid about.

 



NOTES

forms2.min.js inside the IFRAME is only necessary because of its bundled  jQuery library, specifically the MktoForms2.$.ajax wrapper method. While not technically necessary, using jQuery keeps parity with the Forms 2.0 API.

New developers (and non-developers) often mistake values that might be converted to, converted from, or treated sort of like Booleans for a language’s actual Boolean type

 

This leads to internal contradictions like:

 

I have a Boolean with field with values “on” and “off”.

 

and

 

If my Boolean is set to 1, I want to do this. If it's set to 0, do this other thing.

 

If the value is really a string like “on” or a number like 1, you’re not (or no longer) working with a Boolean.

 

Boolean vs. sorta-boolean-ish-sometimes

In general, if a programming language includes a Boolean type, then a Boolean variable can have one of two canonical[1] values:

 

    one value will be the unquoted keyword true

    the other will be the unquoted keyword false

 

The keywords might be lowercase (true/false), uppercase (TRUE/FALSE), or mixed-case (True/False); some languages are case-insensitive.

 

But the quoted strings true” and “false”, the numbers 0 and 1, the numeric strings “0” and “1”, let alone strings like “Yes” and “No” or “Y” and “N” or “opt-in” and “opt-out” or the empty string “” — with rare exceptions in the world of SQL[2] (but not in Marketo) none of those are Boolean values.

 

Sure, those pairs of values might be a lot of useful things:

 

    labels for radio buttons on a form that map to a Boolean on the back end…

    values associated with a custom checkbox-lookalike field

    workarounds because some aspect of an app doesn't support Booleans…

    values that convert predictably to/from Booleans in a specific framework or language…

    the way an app displays Booleans in emails or web pages (hint, hint)

 

But not real Booleans, and the difference matters.

 

Just one of the reasons it matters in Marketo-land

A recent Velocity question from a client brought the difference into high relief:

 

  

With our Product Registration CO, how come the condition #if($product.isRecalled)always seems to match, even if isRecalled is set to false?

 

I immediately looked at their Custom Object setup in Admin, and check it out:

 


 

Bingo! The field isn’t a Boolean. It’s an Integer, presumably sent to Marketo exactly as it’s stored in their in-house SQL database.  And they were trying to use it in a Boolean context (an #if() condition) and expecting 1 and 0 to be truthy and falsy respectively. 

 

But Velocity doesn’t work that way. (Java doesn’t work that way, either: in Java, a strictly typed language, if something ain’t a Boolean, you’ll get a fatal type mismatch error, truthiness and falsiness do not apply.)

 

Note even real Boolean fields (CO or Person fields) require special treatment once they get into Marketo’s Velocity, as I’ve written about before. But here the field wasn’t even a Boolean to begin with! I’ll show you the slickest way to do the required conversion in Velocity in a moment.

 

But first, a little more about real Booleans vs. boolean-ish values, to help you code and/or talk to coders.

 

Truthiness doesn’t change the truth

I don’t blame folks for thinking 0 and 1 are low-level Boolean values at first. There are a bunch of things that can lead you down that path.

 

MISLABELING

Many SaaS apps don’t do you any favors by calling anything vaguely yes-ish/no-ish a “boolean” to sound cool (I guess?). It would be better to call such things “YepNopes” than to be misleading about the real type.

 

THE STRING-CENTRIC WEB

Bedrock web technologies operate exclusively on strings. Take a standard x-www-form-urlencoded form post or a URL query param: those are all characters on the wire.[3]

 

TRUTHINESS

This is a big one. Some languages assign “truthiness” or “falsiness” to certain non-Boolean values, so they don't have to be manually converted to the Boolean datatype to be used in a Boolean context. This can make for less complex-looking code (for what that’s worth) but with a cost of risk and just bad learnin’.

 

Make no mistake: just because a language considers if(1) or if("1") to be true, that doesn’t mean 1 and “1” are Booleans. They’re merely treated, on-the-fly, as Boolean true or false. And a value that’s temporarily truthy in an if() condition may not even be true when permanently converted using toBoolean(), bizarre as that might seem!

 

Moreover, truthiness rules are totally different across languages. One language’s truthy is another language’s falsy — or another language’s fatal TypeError. I’m not harping on hard-to-find cases: 0 and empty “” are falsy in JavaScript; they're both truthy in Velocity.

 

THE ELUSIVENESS OF SQL BOOLEAN

SQL databases are probably the main way people come to think 0 and 1 are Booleans, when in fact they’re just a very common way of storing things that were/will be Booleans.

 

SQL — the ANSI SQL standard, not all products based on that standard — does have a column type BOOLEAN. However, of the big 5 SQL products, only Postgres supports it. Microsoft SQL, MySQL, Oracle, and DB2 don’t. So, to avoid database vendor lock-in, you must use some other SQL type to store values that will be converted to/from Booleans in your app code. The narrowest possible INT, with values restricted to 0 and 1 via a constraint, is very widely used. (Though it’s not the only choice.)

 

Back to Marketo

You know I couldn’t resist that technical dive. But let me flip modes and get back to the matter at hand.

 

Velocity is more forgiving than Java, so it doesn’t throw an error[4] when non-Booleans are used in a Boolean condition. Velocity uses a very simple rule: anything other than null or the real Boolean false is truthy. That means an empty String, a non-empty String, an Object or List (regardless of whether it has any  properties/items), and 0 and 1 and all other Integers are all going to match #if($something)!

 

Now it’s clear why the client’s attempt to use #if($product.isRecalled) was doomed to fail.

 

So what’s the simplest way to turn 1/0 to true/false?

 

There’s clearly a plodding way to do it:

 

#if( $product.isRecalled.equals(0) )
#set( $product.isRecalledBoolean = true )
#else
#set( $product.isRecalledBoolean = false )
#end

 

But that’s going to be cumbersome to call every time. What about converting the values to Booleans using ConvertTool?

 

#set( $product.isRecalledBoolean = $convert.toBoolean($product.isRecalled) )

 

Nope. ConversionTool’s toBoolean() only treats the String "true" as convertible to Boolean true. Numbers 0 and 1 will both convert to Boolean false. (Notice how that’s the opposite of 0 and 1’s truthiness? That’s the kind of unpredictability I’m talking about!)

 

But there’s a secret weapon we can use to keep the conversion to a (slightly longer) one-liner: DisplayTool’s plural() method.

 

DisplayTool.plural() would be better called DisplayTool.notOneOrMinusOne(), because 0 is considered plural:

 

 

This quirk is great for our case, because now we can do:

 

#set( $product.isRecalledBoolean = $convert.toBoolean($display.plural($product.isRecalled,"true"))

 

 Here’s how this one-line contraption works:

 

      
  •  plural checks if the first argument ($product.isRecalled) is what it considers “singular” or “plural”   
          
    • if the argument is singular (that is, for our purposes, if it’s 1 – since we’re saying it’s never anything other than 0 or 1) then it returns the second argument, the string "true"
    •     
    • if the argument is plural (that is, if it’s 0) then it returns the second argument appending the letter “s”, to make the string "trues"
    •    
  •   
  • $convert.toBoolean then processes the returned value, which is always "true" or "trues"   
          
    • toBoolean("true") converts to Boolean true
    •     
    • toBoolean("trues") converts to Boolean false
    •    

 

So we’ve implemented the desired 0→false, 1→true conversion. Cool, eh?

 

 

 


 

Notes

 

[1] “Canonical” means a value that can be both written to and read from the variable. I use this qualifier because some languages allow alternate values to write a Boolean, but you’ll never see those values when you read a Boolean. For example, in XML, 0 and 1 are synonyms for true and false when setting an xs:boolean. But the values read back out are always true or false.  And the Postgres example below applies as well.

 

[2] Postgres goes bizarrely further than just the standard TRUE/FALSE keywords (though those are the preferred values for setting SQL BOOLEANs). They allow lots of aliases to be used for setting values: the strings “true”/“false” and “t”/“f”, even “yes”/“no” and “on”/“off” and a few more. Stranger still IMO is that the output values are the strings “t” and “f”. So you would say the canonical values are “t” and “f” in that particular dialect (since those are the only values that can be both written and read). Weird exception.

 

[3] Funny to call XML “newer” than anything as it’s so ancient, but both XML and JSON reserve special Boolean values (as well as distinguishing numbers from numeric strings) even though the overall encoding is still character-based. But regular form POSTs and GET params are strings through and through, until/unless the server chooses otherwise.

 

[4] This forgiveness is not necessarily a good thing. As I’ve droned on about before, Velocity suppresses a lot of fatal errors to keep your app running, but not all types by any means: you can trivially cause a fatal error by .get()-ing the 1st item in an empty List!

Community user JW asked a surprisingly unprecedented (AFAIK) question:

 

I am looking to send out a communication to our customers that includes that persons Policy Number. Unfortunately our legal department will not let us send the entire {{lead.Policy Number}} digitally. We are permitted to send the last 4 digits however...  with Velocity Scripting can I get it to show as *****3253?

 

Let’s be clear: this isn’t relevant to credit card numbers or SSNs because those fields would never be in Marketo in the first place! (Right, guys?)

 

But other fields – like Policy Number here, or other account/lead info like the person’s phone number – legitimately would be present in Marketo and your CRM, yet might be nice to partially redact in emails. In the unlikely event that an email is intercepted, the person’s privacy is protected, but there’s still enough data to know they’re the right recipient.

 

So here’s a quick one-liner to mask out 1234567890 to ******7890:

 

${lead.PolicyNumber.replaceAll(".(?=.{4,}$)","*")}

 

In brief, it’s a regex lookahead and replace. The pattern matches any character that has 4 or more characters between it and the end of the string, ergo, all characters except the last 4.

 

(Yes, there are ways to do this with indexOf and substring instead, possibly more efficiently than the regex… if you’re timing to the microsecond, that is. But as always with Velocity, saving a few lines of code is worth a few μs.)

For arithmetic in Velocity, the only safe approach is the so-called “tools-ish” one: use MathTool methods like $math.add, $math.sub, and  $math.mul.

 

The rule

Aside from the one exception later in this post, you should not use the Java-like math operators (+ and -).

 

The core reason is the plus sign + and minus sign - have different syntax rules in Velocity! You can easily create fatal errors – or worse, lines that are silently ignored – by forgetting that - must be surrounded by spaces.

 

That’s right, only one of these is correct Velocity syntax for subtraction:

 

#set( $a = 99 )
#set( $b = $a - 1 ) ## correct
#set( $b = $a -1 ) ## fatal ParserException!
#set( $b = $a- 1 ) ## fatal ParserException!
#set( $b = $a-1 ) ## non-fatal, but doesn’t subtract!!!

 

So I always teach people to use #set( $b = $math.sub($a,1) ). It’s longer, but you’ll never break your code by switching sub and add.

 

The exception

Despite the above, in the course of developing my Base64 encoding in Velocity code, I discovered a unique requirement that could only be met by purposely using the + operator instead of $math.add.

 

This is the only exception I’ve found, and it doesn’t override my recommendation above. But it is – to me and I hope to the couple of people like me out there – fascinating.

 

So.

 

In Base64 encoding, one of the steps requires Integers to be converted to binary Strings, 42 → “00101010”. (If this is complete gibberish to you, you can reread this post when you have a couple more years of development under your belt!)

 

To get those Integers, you need to convert signed Bytes (-128 through 127) to their equivalent all-positive Integers (0 through 255).

 

To convert signed Bytes, you need to use a bitwise AND. But the traditional AND operator, the & symbol in Java and most other languages, isn’t available in Velocity.

 

Since you can’t use notation like #set( $c = $a & $b ) in Velocity (that’s a fatal parser error) you need to find a method-based way – a “tools-ish” way, somewhat ironically – to do a bitwise AND: something like #set( $c = $a.and($b) ).

 

Miraculously, such a method does exist (after you do some hunting) so long as $a is a Java BigInteger.  

 

Hmm. So now you need to be able to construct a BigInteger from an Integer.

 
Getting a BigInteger

A BigInteger isn’t something you can explicitly create in Marketo’s Velocity (post-June 2019, that is), only implicitly. There’s one way to cheat it, and that is to set a variable to just outside of the range of a Long. If you know off the type of your head that the max positive Long is 9,223,372,036,854,775,807 (~9 quintillion) then you can add 1 to that:
    

#set( $aBigInteger = 9223372036854775808 )
${aBigInteger.class} ## will be class java.math.BigInteger

 

 

Skipping the magic

But let’s say you don’t know that magic number – you only know that Java’s

concrete integer Number types  are, in order of increasing width, Byte → Short → Integer → Long → BigInteger. (I think it’s reasonable for a developer to know that progression, but not reasonable to expect someone to remember that 9,223,372,036,854,775,807 is special.)

 

If you didn’t know the magic Long-BigInteger boundary, how would you make sure Velocity allocated a type big enough for a BigInteger? You’d want to say:

 

Gimme a Number that can hold Long.MAX_VALUE + 1.

 

Of course to do that, you‘d have to get a handle on a Long (in order to refer to the constant Long.MAX_VALUE). Since Velocity doesn’t give you a Long unless you’ve asked for something bigger than an Integer, you have to say:

 

First, gimme a Number with the value 0. I know that’s an Integer, since that’s the default type for whole numbers in Velocity.


Then, gimme another Number that can hold ThisMustBeAnInteger.MAX_VALUE + 1. That’s gonna be a Long.

 

Finally, gimme yet another Number that fits ThisMustBeALong.MAX_VALUE + 1. That third number must be a BigInteger.

 

And here is where + acts differently from $math.add.

 

MathUtils vs. MathTools

Hidden in the doc for Velocity’s MathUtils (part of Velocity’s internal API, not for public consumption) you’ll find this gem:

 

So now we know Velocity can do – in fact is explicitly documented to do – exactly what we need. If we add 1 to the max value of a Long, the result will be a BigInteger (and will also have the correct value, obviously).

 

Except this only applies to the literal + operator. MathTools’ $math.add also does addition, but doesn’t implement the “overflow correction” logic.

 

Let’s compare.

 

Using the + operator:

#set( $Integer = 0 )
Integer $Integer.class.getName() = $Integer
#set( $Long = $field.in($Integer).MAX_VALUE + 1 )
Long $Long.class.getName() = $Long
#set( $BigInteger = $field.in($Long).MAX_VALUE + 1 )
BigInteger $BigInteger.class.getName() = $BigInteger

 

The output will be:

Integer java.lang.Integer = 0
Long java.lang.Long = 2147483648
BigInteger java.math.BigInteger = 9223372036854775808


Exactly what we need. $BigInteger is a BigInteger we can use as a factory to make more BigIntegers ($BigInteger.valueOf) and then do fancy stuff like .and() and .or() and  .xor() which would otherwise be impossible in Velocity.

 

Let’s try using $math.add:

#set( $Integer = 0 )
Integer $Integer.class.getName() = $Integer
#set( $Long = $math.add($field.in($Integer).MAX_VALUE,1) )
Long $Long.class.getName() = $Long
#set( $BigInteger = $math.add($field.in($Long).MAX_VALUE,1) )
BigInteger $BigInteger.class.getName() = $BigInteger

 

 

The output will be:

Integer java.lang.Integer = 0
Long java.lang.Long = 2147483648
BigInteger java.lang.Long = 9223372036854775807

 

 

Whoa. We attempted to add 1 to the largest Long, but the attempt silently failed. $BigInteger (despite the name) remains a Long, and it never gets incremented: it’s still Long.MAX_VALUE or 9,223,372,036,854,775,807.

 

It’s notable, but doesn’t change the rule

The likelihood that you’d otherwise need to perform BigInteger-range arithmetic within Marketo is basically zero.

 

I mean, it would have to be something like… you’re using a webhook to take the squares of lead scores, and you have a score in the millions and must raise it to the power of 2 in a Velocity token. Let’s face it, relative to the chances you will at some point forget the spaces around -, that’s just not gonna happen.  The risk/reward overwhelmingly favors $math.add and $math.sub.

 

In this case we needed a BigInteger not because of its infinite range of values, but because we need to get at the cool methods it has – which Integers and Longs do not. (The underlying value is tiny, between -128 and 127 then converted to between 0 and 255.) So for this particular purpose, deliberately using + makes sense.

The removal of Reflection-based features from Marketo’s Velocity environment in June 2019 was a disappointment — however inevitable — to those of us who push email scripting to the limit.

 

Requirements like encoding, hashing, encryption, flexible rounding and randomization, XPath queries, and advanced filters/sorts on Custom Objects could no longer be met with Marketo alone. (Webhooks can fulfill a subset of those needs, but with nowhere near the performance and simplicity of Velocity.)

 

At the prodding of Community user TM, I set out to see if Base64 encoding, one of the Java methods that used to be callable from Velocity, could be written with “pure” Velocity and public Java methods, without relying on the Reflection API.

 

I was indeed able to do it... though unlike most of my Velocity work, this one wouldn’t File Under Fun!

 

You can call the #userlandBase64_v3 macro below in 4 ways:

 

#1: Basic usage

#userlandBase64_v3("some string")

The result uses the standard RFC4648 Base64 character set (A-Za-z0-9+/). (That is, the characters used/expected by browsers’ native btoa()/atob(), not a “URL-safe” variant.)

 

#2: Automatic URL encoding

#userlandBase64_v3("some string",true)

This option URL-encodes the input string (using $esc.url) before Base64-ing it, which is critical if (a) you have non-Latin-1 characters in the string and (b) you want to use the browser’s native atob() to decode the result.[1]

 

#3: Basic + URL-safe characters

#userlandBase64_v3("some string",false,"RFC4648URLSafePadding")

This result is always safe to embed in URLs directly, even in the path, because it uses - and _ in place of the more sensitive + and /.

 

#4: URL encoding + URL-safe characters: The works

#userlandBase64_v3("some string",true,"RFC4648URLSafePadding")

The safest route, as it accounts for both non-Latin-1 in the input string and for possible URL conflicts.[2]

 

Get the code

I don’t want to digress into how the code works in this post, but perhaps there’ll be a Code Anatomy follow up... my version of the code has lots of comments!

 

Enjoy...

 

#**
* Base64 encoding in Velocity without Reflection
* @version v3 2010-10-11
* @author Sanford Whiteman, TEKNKL
* @license MIT
* @param $inputString String String to transform
* @param $alsoURIEncode Boolean [default false] URI-encode the String first (support browser-native atob())
* @param $charset String ["RFC4648" | "RFC4648URLSafePadding" | default "RFC4648"] 64-character set
*#
#macro( userlandBase64_v3 $inputString $alsoURIEncode $charset)
#set( $Base64PaddingChar = "=" )
#set( $Base64ByteChunkSize = 3 )
#set( $Base64BitGroupSize = 6 )
#set( $Base64Charsets = {
  "RFC4648" : "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/",
  "RFC4648URLSafePadding" : "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"
} )
#set( $Base64Charset = $display.alt($Base64Charsets[$charset],$Base64Charsets["RFC4648"]) )
#set( $uriEncode = $display.alt($alsoURIEncode, false) )
#set( $Integer = 0 )
#set( $Long = $field.in($Integer).MAX_VALUE + 1 )
#set( $BigInteger = $field.in($Long).MAX_VALUE + 1 )
#set( $String = "" )
#set( $Base64Final = [] )
#if( $uriEncode )
#set( $inputString = $esc.url($inputString) )
#end
#set( $inputBytes = $inputString.getBytes() )
#set( $inputList = [] )
#set( $void = $inputList.addAll( $inputBytes.subList(0,$inputBytes.size())) )
#set( $inputSizeB = $inputList.size() )
#set( $padding =  $math.mod($inputSizeB, $Base64ByteChunkSize) )
#if( $padding > 0 )
#foreach( $pad in [1..$math.sub($Base64ByteChunkSize,$padding)] )
#set( $void = $inputList.add($null) )
#end
#end
#foreach( $byteGroup in [0..$math.sub($math.idiv($inputList.size(),$Base64ByteChunkSize),1)])
#set( $startOffset = $math.mul($byteGroup,$Base64ByteChunkSize) )
#set( $endOffset = $math.add($startOffset,$Base64ByteChunkSize) )
#set( $bytes = $inputList.subList($startOffset,$endOffset) )
#set( $integerList = [] )
#foreach( $byte in $bytes )
#if( $byte )
#set( $currentInteger = $convert.toInteger($byte) )
#set( $void = $integerList.add($convert.toInteger($BigInteger.valueOf($currentInteger).and($BigInteger.valueOf(255))) ))
#else
#set( $void = $integerList.add($null) )
#end
#end
#set( $binStrings = [] )
#foreach( $currentInteger in $integerList )
#set( $void = $binStrings.add($String.format("%8s",$Integer.toBinaryString($currentInteger)).replace(" ","0")) )
#end
#set( $allBinStrings = $display.list($binStrings,"") )
#set( $bitGroups = [] )
#foreach( $bitGroup in [0..$math.sub($math.idiv($allBinStrings.length(),$Base64BitGroupSize),1)] )
#set( $startOffset = $math.mul($bitGroup,$Base64BitGroupSize) )
#set( $endOffset = $math.add($startOffset,$Base64BitGroupSize) )
#set( $void = $bitGroups.add($allBinStrings.substring($startOffset,$endOffset)) )
#end
#foreach( $group in $bitGroups )
#if( !$group.contains("null") )
#set( $Base64CharsetPos = $Integer.parseInt($group,2) )
#set( $Base64Char = $Base64Charset.charAt($Base64CharsetPos) )
#else
#set( $Base64Char = $Base64PaddingChar )
#end
#set( $void = $Base64Final.add($Base64Char) )
#end
#end
$display.list($Base64Final,"")
#end

 

 


Notes

[1] Related browser-side decoding:

decodeURIComponent(
  atob(myEncodedString).replace("+"," ")
);

 

[2] Browser-side hint for this format:

decodeURIComponent(
  atob(
    myEncodedString.replace("-","+").replace("_","/")
  ).replace("+"," ")
);

 

Velocity does something magical when you just output a Lead field (without any other code at all).  The magic: it truly outputs the value as stored in Marketo. A {{lead.token}} doesn’t do that by default.

 

Community user AB wondered why a field containing an HTML table was revealing raw HTML (as text) in emails, as opposed to rendering the table.

 

That is, if a field looks like this in the UI:

 

 

Here’s what you see if you include {{lead.The Field With HTML}} in an email:

 

 

While you might have expected to see:

 

 

The reason you see the HTML-as-text is simple: Marketo HTML-encodes token values by default.

 

There’s nothing wrong, and a lot right, with this being the default behavior. It’s the same way that browsers deal with HTML-like text that’s not specifically inserted as HTML. To see what I mean, open a browser tab, go to  the Dev Tools Console, and run

 

document.body.insertAdjacentText("afterBegin","<table><tr><td>Stuff</td><tr></table>")

 

and you’ll see the code for a table, not a table!

 

In AB’s case the encoded value is:

 

&lt;table&gt;&lt;tr&gt;&lt;td&gt;Access Code&lt;/td&gt;&lt;td&gt;Remaining Uses&lt;/td&gt;&lt;td&gt;Start Date&lt;/td&gt;&lt;td&gt;End Date&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td&gt;hh7zh-lgyal&lt;/td&gt;&lt;td&gt;2.0&lt;/td&gt;&lt;td&gt;2018-12-31 08:00:00&lt;/td&gt;&lt;td&gt;2019-12-30 08:00:00&lt;/td&gt;&lt;td&gt;&lt;/table&gt;

 

Basic stuff, the < and > are encoded so the web browser/mail client won’t look for any deeper meaning in the text.

 

The old answer

The ready answer you might see for this is “Turn off HTML Encode Tokens in Field Management.”

 

I’ve given this answer myself. It works as a just-in-time fix. But I don’t feel it’s the right answer anymore, for multiple reasons:

 

  • you (the email author) may not have permission to make this change
  • if a field is used for multiple purposes, it may not be globally correct to leave the value unencoded
  • encoding is a secure default

 

The new answer

Using Velocity gives you local (per-email, even per-lead) control over whether the value is encoded.

 

To output a field without encoding, just create a {{my.token}} and drag that field to the canvas:

 

If you decide you do want to encode, Velocity has a method for that, EscapeTool.html:

 

${esc.html( $lead.TheFieldWithHTML )}

 

Now you’re in control of whether the output differs from the stored value.

 

Another related case

If you’re in the unlikely-but-not-unheard-of situation where an integration is plopping HTML into a field where you just want text, then it’s not just encoded vs. unencoded. You want the HTML tags outta there completely.

 

For that you use DisplayTool.stripTags:

 

${display.stripTags( $lead.TheFieldWithHTML )}

 

If you want to keep (whitelist) specific tags, pass all those tags as the 2nd+ arguments. With this VTL...

 

${display.stripTags( $lead.TheFieldWithHTML, "td", "tr" )}

 

... this original value...

 

<table>
<tr><td>Access Code</td><td>Remaining Uses</td><td>Start Date</td><td>End Date</td></tr>
<tr><td>hh7zh-lgyal</td><td>2.0</td><td>2018-12-31 08:00:00</td><td>2019-12-30 08:00:00</td></tr>
</table>

 

... gets output like so:

 

<tr><td>Access Code</td><td>Remaining Uses</td><td>Start Date</td><td>End Date</td></tr>
<tr><td>hh7zh-lgyal</td><td>2.0</td><td>2018-12-31 08:00:00</td><td>2019-12-30 08:00:00</td></tr>

 

Could be handy if you have a system that’s wrapping HTML in extra containers, disrupting your layout.

 

Note stripTags acts like textContent in a browser: all the matched opening and closing tags are removed, they’re not replaced by anything. So you might get unpretty output: <div>Sandy</div>Whiteman becomes SandyWhiteman (note the lack of space). Whether this works for your case depends on what’s coming in.

I obsess over questions like “What is a webhook?” like philosophers contemplate “What is consciousness?” (Or like others wonder “Are we merely 3-dimensional cartoons watched by N-dimensional beings?”)

 

It’s a different kind of search for truth, but better for your professional career.

 

  • Without the true (that is, RFC-based) answer to “What is a URL?” you’ll write a broken UTM parser.
  • If you use the term “CNAME” incorrectly, IT won’t take your requests seriously (or will botch the result).
  • If you don’t know what an SPF record can be, and when it, shall we say, ceases to be, your  deliverability will take a hit.
  • Without a (technical, not Shakespearean) dive into “What is a name?” you’ll mess up lots of things.

 

Standards, papers, and real-world experiences can reveal much wider definitions than you previously thought. The point of this post, for example, is that webhooks can take many more forms than you first imagine. And that should be exciting.

 

It starts from the start

The first mistake people make about webhooks is confusing (1) the webhook configuration (say, in Marketo Admin); (2) the webhook trigger (an event hook, which in Marketo means a triggered Smart Campaign); and (3) the remote web service that receives the webhook.

 

Far too frequently, (3) is mistakenly called the webhook. But it’s not.

 

You see, “the webhook” is the outbound HTTP request issued by a webhook-supporting app. It’s not the remote server that you connect to in hopes of a useful response – rather, it’s a feature of the platform that listens for the trigger. That is, for our purposes today, it’s a feature of Marketo.

 

Of course, without a webhook-compatible server on the other end, you won’t make successful use of webhooks! But you’re still sending a webhook, even if the other side is completely down.

 

The requirements for a server to be webhook-compatible are:

 

1. It runs over standard HTTP or HTTPS (that’s the “web” part). FTP servers, SMTP servers, servers that speak exotic binary protocols won’t cut it.[1]
2. It does not require multiple connections to perform a requested action. Webhooks are stateless. That means that OAuth-based systems requiring one HTTP connection to get an expiring token, followed by another connection do a lookup or update, aren’t webhook-compatible.[2]
3. It completes its requested action fast enough to handle invocation rates and within the HTTP timeout of the webhook side. “Fast enough” is a bit squishy because it depends on how many calls can be processed in parallel. Across the universe of webhooks, < 5s (including network time) is generally acceptable; < 1s is ideal. In Marketo specifically, < 30s is mandatory.[3]

 

Also note the creators of a service need not know it’s compatible to be compatible.[4]

 

The server doesn’t need to be a full-fledged service
You were supposed to notice something: the requirements above don’t include running a particular web language (PHP, Java, C#, JavaScript) nor any language at all!

 

That's because a webhook is still a webhook, even if it fetches static files – as long as the connection is over HTTP and finishes fast.

 

And a webhook need not fetch a different static file when called for different leads. Take a webhook you use to set default values. This could always get the same static JSON file defaults.json:

 

{
"FirstName" : "N/A",
"LastName" : "N/A",
"Company" : "[No Company Name]",
"Email" : "user@unknown.invalid"
}

 

This single ’hookable file serves a useful purpose, though it’s far from what people think of when you say “webhook”!

 

Taking it to the next level, you could use a set of static JSON files named by email address:

 

 

And a lead’s personal file could be dynamically fetched by the URL https://static.example.com/{{Lead.Email Address}}.json. For key accounts this would come in handy.

 

(You wouldn’t likely create tons of .json files by hand, in any case, but rather export them from a line-of-business app of some kind. The takeaway is that once static files are deposited on a webserver, no code needs to run to enrich a lead with their corresponding data.)

 

A static(ish) file that allows lookup

So far, the static file examples send back the  full contents of a static JSON file.

 

But what about doing a lookup within a large set of data to only return matches for the current lead?

 

That’s usually where people turn to a formal SQL or NoSQL database and, naturally, need to write the code to query that db. No longer is the webhook just fetching a prefab file with JSON. Instead, the webhook calls a web service, which is written in JavaScript, or C# or PHP or what-have-you. Then that service code queries the database using SQL or some other dialect, reading query info from the POST payload or from the GET query params.[5]

 

So when someone says, “Just use a webhook” (and I’m plenty guilty of this myself) unless they’re talking about calling an already-published service like Twilio SMS, they imply some kind of code. Perhaps simple code (to a developer’s eyes) but not no code.

 

But for certain types of data, it’s possible (and here is where this post starts to get strange) to create a webhook-compatible lookup service using only static files and standard HTTP headers. No code at all.

 

The key is HTTP Range (a.k.a. byte serving).

 

Homing in on Range:

The HTTP Range feature is amazingly ubiquitous: if you streamed a short video

today, you used it hundreds of times!

 

It works in such a simple way: you request a resource (URL) from a server and include a Range: header with a specific range of 1 or more bytes.[6]

 

The server, instead of responding with the entire resource, only sends you the bytes you requested.

 

So imagine you have this 26-byte, dirt-simple ASCII file shortsample.txt:[7]

 

ABCDEFGHIJKLMNOPQRSTUVWXYZ

 

And then you send this HTTP request:

 

GET /shortsample.txt
Range: bytes=0-4

 

The byte position always starts from 0. So this will get these first 5 bytes:

 

ABCDE

 

Presto! If you only wanted to peek at the first few characters in the file, you’ve just saved 80% of the potential network traffic. And now that you have the first 5, you can cache those and only get the remaining bytes, concatenating the files together afterward:

 

GET /shortsample.txt
Range: bytes=5-

 

FGHIJKLMNOPQRSTUVWXYZ

 

The Range: way of retrieving small segments at a time over standard HTTP is key to modern video streaming (Range Retrieval has been standard for 20 years, since RFC 2616, so it predates HTML5 video itself but has come into its own more recently). It’s also how we get pause/resumeable file downloads.

 

So how can Range: create a lookup-able, webhook-compatible service out of a single file? For that, you have to get inside my brain in the wee hours a few weeks ago (I never sleep because I’m thinking of things like this!).

 

Imagine the byte position (an Integer) can be a lookup key, and the character at each position (a one-letter ASCII String) can be the value at that key.

 

An example will — well, hopefully — help.

 

Say leads coming into your instance have a numeric License Code between 0 and 49 (in reality the range could start anywhere, just simplifying for now). The code indicates which kind of products they’re legally allowed to buy. Only people with certain codes are allowed to do business with your company. And it’s a binary/boolean thing: either they’re Allowed or Blocked, no nuances.

 

Create a text file like this that includes all 50 codes, in order from 0, with “A” for Allowed and “B” for Blocked:

 

ABBBAAAABBBABABBBAABBBBBAAAAAABABAAABAABAABAAABABA

 

Now, you can get the single byte at their License Code value to check if they’re allowed or not:

 

GET /licensecodes.txt
Range: bytes=23-23

 

That is, in a Marketo webhook’s Set Custom Headers config, where you can use {{lead.tokens}}:

 

Range: bytes={{Lead.License Code}}-{{Lead.License Code}}

 

 

That will return the one-character string

 

A

 

or

 

B

 

for every lead.

 

Did you have an “A-ha!” moment? I hope so.

 

Step by step

In the above example, the possible License Codes conveniently started at 0 and were all filled in through 49.

 

Let’s think of a somewhat more complex case, where the only codes used by the licensing bureau are 10-20 and (for whatever reason) 30-50.

 

Not a problem. Simply pad the file with 0s wherever you don’t have a possible value (you could use spaces or underscores or whatever, but 0s are easier to read on the blog, methinks):

 

0000000000BABABBBAABB000000000BABAAABAABAABAAABABA 

 

Byte #10 — a.k.a. License Code #10 — is still “B” for “Blocked”. Everything still works!

 

Why bytes and not bits?

If you’re with me so far you might be thinking, Why is he wasting whole bytes on letters, when there are 8 individual bits to fiddle with? I talk about that in the notes.[8]

 

We’re still getting underway (sorry!)

I know you’ve scrolled a lot already, but alas we’re only partway there.

 

Now, let’s walk through something that’s closer to a real-world scenario (I mean, the license code thing isn’t really that artificial, but I already had this other example in mind).

 

Imagine you‘re a US-based company and you assign every US ZIP code a certain grade that reflects its suitability for your product (please don’t assume I did anything but randomly generate a letter A-F, I didn’t pay attention to actual geography at all!). It’s a form of scoring, in other words.

 

Now, there are in theory 100,000 5-digit ZIP codes (00000-99999). In reality, only half of them are allocated (some are simply not used yet, some are permanently reserved). Here’s the top of an Excel sheet with the ones in use and their state/territory, in ascending order from the first one allocated (00501) to the last (99950), with lots of gaps of course:

 

 

Yes, I’m quick to admonish people for treating alphanumeric strings (like ZIP codes or, for other examples, credit card numbers or phone numbers) as if they’re truly numbers. So I’m not suggesting under any circumstances that ZIPs should be stored as Integers in a database. But for today’s purpose, a ZIP is usable as a numeric index. So let’s convert column A from Text to Number:

 

 

Then get the Sales team involved and assign grades:

 

 

Remember the lesson from the second License Codes example above. If we’re going to seek a specific byte position (ZIP 00501 = number 501 = byte 501 ), then every byte position must be occupied in some way. (To use technical jargon, you can’t have a “sparse array”, it needs to be “dense”.)

 

So we fill in the gaps so numbers that don’t have a state/territory are still present. Here’s how that looks, zoomed out a bit:

 

 

Then put a dummy grade of 0 for all the unused slots:

 

 

Now, the only thing we care about is Column C. That’ll be our database (if you will) of grades indexed by ZIP.

 

From here, we’re going to head over to a text editor to avoid any chaff that might come in from Excel.

 

Here’s just Column C, with carriage returns and/or line breaks:

 

We’ve got to strip out those CRLFs! They might add 1 or 2 bytes, throwing us totally off. Here’s the start of the file without CRLFs:

 

And another view after scrolling right a bit, so you can see the real grades after the hundreds of zeroes that have to kick it off (ZIP 00000/byte 0 through ZIP 00500/byte 500 are unassigned):

 

And finally, a deep look using a hex editor. You can see that the 501st byte, for ZIP code 00501, corresponds to the grade “B” (ASCII 0x42).

 

As far as the webhook-compatible file is concerned, we’re good to go! Now upload that file to a server somewhere (I’m using my Amazon S3 account) and let’s switch back to our beloved Marketo UI.

 

The webhook definition

Fetching the custom-crafted textfile with a lead-specific Range: header is simple. The main screen in Admin:

 

And the Set Custom Header dialog:

 

As you can see, we fetch the single byte at index {{lead.Postal Code}}.

 

Running a lead through the webhook, we see the expected single-byte response F:

 

 

So far, we’ve had success. The Range: is customized for the lead and honored by the remote S3 server, and we have a clean response.

 

Only the response is maybe too clean.

 

We’re not quite there

Here’s the rub.  Though there’s no particular response format that’s a requirement for being webhook-compatible, Marketo can only automatically map response values to lead fields if the response is valid XML or JSON. (It actually doesn’t matter whether the HTTP Content-Type is set, by the way, and Marketo, please don’t change this!)

 

In this case, the response is a single ASCII byte, which is totally valid text/plain of course but isn’t valid XML or JSON. Marketo doesn’t have a buit-in way to say “Map the entire response payload back to such-and-such field” so we have to take a more clunky approach (don’t worry, later in this endless post we’ll learn how to fix the clunk).

 

That clunky way is to use the undervalued Webhook is Called trigger. Since we only have 6 possible values, it’s easy enough to manage. You just need 6 Smart Campaigns with symmetrical structure, and the Response constraint matches each single-byte plain-text response:

 


Fixing the clunk

To avoid building multiple campaigns and use a standard webhook Response Mapping, we have to expand on another techno-ontological-philosophical question: “How do you generate a JSON response?

 

You probably have 2 valid answers in mind:

 

1. You return a static .json file stored on disk (that is, the entire file).
2. You generate the JSON text in server-side code, making sure to set your Content-Type to application/json.

 

Those are common, but what about an uncommon 3rd option: you store multiple valid JSON responses, end-to-end, in a single .txt file and use Range: to choose which block of JSON to return?

 

The file as a whole isn’t valid JSON. But you’re only reading parts, and those parts are valid.

 

For example, take this file:

 

{ "First Name" : "Marcus" }
{ "First Name" : "Cleopatra" }

 

 

That's not JSON taken all together (don’t mistake it for an array of 2 objects, as it’s missing the all-important [] and  , so would never parse correctly).

 

But the 29 bytes of the first line alone (including the CRLF) would be a single valid JSON object. And the 32 bytes of the second line alone would also be valid JSON.

 

Right now, the lengths of the 2 blocks are irregular. And since we don’t know in advance how long a First Name might be, that won’t yet work. So instead (for the sake of simplicity) let’s say it’s a maximum of 9 single-byte letters long (C-l-e-o-p-a-t-r-a). And we pad each line to exactly 32 bytes using harmless whitespace.

 

Using the ◌ character (in this post, not in the real file) to literalize the spaces and line break characters, and counting off the 32 bytes at the bottom:

 

{◌"First Name"◌:◌"Marcus"◌}◌◌◌◌◌
{◌"First Name"◌:◌"Cleopatra"◌}◌◌
{◌"First Name"◌:◌"Bob"◌}◌◌◌◌◌◌◌◌
00000000011111111112222222222333
12345678901234567890123456789012

 

Now we’ve created a file full of fixed width structured data. It doesn’t have any inherent meaning to the server hosting the file – it’s just like any text file – but to a client (i.e. Marketo) that knows how to seek into it, it’s like a giant, albeit simple, database of JSON objects.

 

We’re almost, sort of, there

With the above file structure, we know the width of an object is always 32. So if we want the Nth object, we seek to the position (N × 32 - 1) – recalling that positions start from 0 – and read the next 32 bytes.

 

But that li’l bit of multiplication there (N × 32) isn’t something we can do without... calling another more sophisticated webhook! Which would  defeat the purpose of this post. (Which was what again? Ah, to show how with a lot of ingenuity, you can make useful webhook-callable endpoints without writing any code.)

 

So we need to change our fixed block width to, well I’ll just cut to the chase: a multiple of 10.

 

Why? Because you can denote 10-byte zero-based ranges without doing the multiplication yourself.

 

Huh? Think about “0-9" – the first range of 10 bytes. “10-19" is the next range. 20-29 is next. Each of these strings can be created with simple variable substitution:

 

bytes={{some variable}}0-{{some variable}}9

 

Or with 100-byte blocks:

 

bytes={{some variable}}00-{{some variable}}99

 

100-byte blocks, though, could end up wasting a ton of space. Let’s not do that unless we have to. Instead let’s tighten our internal JSON objects as much as we can (the file doesn’t need to be human-readable, after all). If we use a single-character key v and remove other whitespace, we can get each block down to 8 bytes with an empty string value:

 

{"v":""}

 

Thus leaving us the headroom for 2 glorious bytes of string data! Since the example application here is single-letter grades A through F for every Postal Code, that's just fine.

 

And that's how the “Book of JSONs” file is laid out:

 

 

Now we need only set up a Marketo webhook that seeks 10 bytes at a time, offset by the Postal Code:

 

And the 10-byte response is valid JSON, ready for a standard Response Mapping back to our lead field:

 

Also notice how the Call Webhook dutifully records the HTTP 206 (Partial Content) as a success. Which it is!

 

Oh, and one more thing you can do with this is...

 

... basic score arithmetic (well, addition at least)

Y’know how Marketo can’t add 2 numbers on its own?

 

Here’s a primitive addition table created in Excel (via auto fill, I ain't that crazy!):

 

The column numbers are the first addend; the row numbers are the second addend.

 

You must consider the rows to start at Row 0, not Row 1 (something that Excel doesn’t allow, to this programmer’s dismay!). Count the columns from 0 in the same way, rather than as letters, so Excel’s Column D is Column 3.

 

The cell where a row & column meet is the sum of those numbers. For example, Excel cell D7 in zero-based terms = Column 3, Row 6. The value of that cell is 9 - which is the sum of 3 and 6!

 

By applying the same JSON-10-byte-block approach used above for ZIP codes, you can preload this addition table into Marketo as a bunch of individual files. Not to worry, I created the first 1000 columns for you, giving you the ability to sum any 2 numbers between 0 and 999.

 

This is one case where Marketo Sky (which, like most of you, I stil use only occasionally) shines over the legacy UI, since you can drag-and-drop multiple files. So download this file:

 

rangehook_mathjson_files.zip

 

Then unzip it and upload the contents to Design Studio into an appropriate folder:

 

 

Now you’re ready to do some simple addition.

 

Maybe you want to add the {{Lead.Demographic Score}} and {{Lead.Firmographic Score}} together into {{Lead.Aggregate Score}}.

 

You’d set up a webhook with the following settings (I’m too exhausted for another round of screenshots, but I trust you to know what to do!):

 

 

Where 123-ABC-456 is your Munchkin ID and na-sj01 is your direct asset URL (if you’re on instance app-sj01.marketo.com then the URL is na-sj01.marketo.com, etc.). Don’t use your Marketo LP domain here, it won’t work.

 

  • Custom Header: Rangebytes={{Lead.Firmographic Score}}0={{Lead.Firmographic Score}}9

 

  • Response Mapping: vAggregateScore

 

When run, this webhook will do a request like this, if Demographic Score is 12 and Firmographic Score is 30:

 

GET /rs/123-ABC-456/images/add-to-12.json
Range: bytes=300-309

 

Bytes 300-309 of add-to-12.json are, conveniently:

 

Giving us the correct sum, 42.

 

I’ll say it again: it ain’t pretty, but it is predictable and accurate. And it doesn’t require any external services, just a willingness to play on the wild side.

 

It’s not just a file download, it’s an indexed lookup!

It’s easy to confuse a Range request with a primitive file download. But it’s not, and the proof is in the performance.

 

Seeking the 1 billionth byte of a 1 GB file should be no slower than seeking the 1st byte. I’ve run convincing benchmarks with S3 and with IIS. Other HTTPds might not be as efficient, but that’s an engineering problem with those engines.[9]

 

Who is this for?

Hopefully, this post was interesting to anyone curious about HTTP, JSON, bytes and ranges and all that stuff.

 

But, to be clear, I don’t expect this far-out take on webhooks to be put into production by a totally non-technical person. It’s more for someone who may have some coding experience (from a past job or school) but – like most marketers – doesn’t have a corporate-approved place to host server-side code.

 

Even if you’re up to the challenge of writing secure, scalable webhook-compatible code... your employer may be (should be!) rightly uncomfortable about you “just” spinning up a production service somewhere in your personal cloud.

 

So the next best thing, for certain kinds of data, is a zero-code webhook running right out of your Marketo instance. No compliance issues, no security issues, no worries.

 

Or just enjoy the post as a peek into my sleepless mind.


Notes

[1] Counterexample: One of our clients runs an FTP server that accepts uploaded CSVs. A back end process periodically reads rows from the FTP’d files and upserts leads into Marketo. The deeper back end connects to Marketo via HTTP, sure – but the front end FTP server ain’t webhook-compatible.

 

[2] A webhook-compatible service needs to allow authentication + authorization info (if any) to be passed in the same HTTP connection as the requested action. Don’t be distracted by doomed hacks like storing access_tokens in lead fields. Even when they sort-of-sometimes work, they’re adding statefulness. You can’t expect two invocations of the same webhook, even for the same record in your database, to know anything about each other.

 

Note this doesn’t mean the service itself can’t call stateful APIs from the back end: we do this all the time! But the first-hop connection, from the webhook to the service, is stateless. Any additional network hops are hidden from the webhook.

 

[3] The “requested action” means the single GET or POST from the Marketo-like app. But that doesn’t mean all related actions on the other side are completed in that same short period!

 

Take an SMS webhook: its requested action is enqueueing the outgoing SMS message within the provider’s infrastructure, not delivering the message to the recipient’s handset. (Let alone listening for 2-way responses, which is way outside of webhook-land.)

 

Similarly, a service that inserts rows into a remote database need not have finished committing and/or replicating data (making it readable by other apps) by the time it returns 200 OK to Marketo. It might complete the insert a few seconds (or even minutes) afterward. What’s important is that the payload is eventually stored, not that it’s stored in real-time.

 

On the other hand: when the service offers data enrichment, field calculation, or remote lookup tables, the requested action does mean finishing everything before sending the HTTP response (typically JSON or XML). So some enrichment apps, particularly those that try to combine data from multiple back-end services (with each of those next-hop requests possibly requiring multiple connections) can end up being unusable via webhooks: they may have the other requirements down, but not the performance.

 

[4] I mentioned a while ago that Twilio’s Lookup API can be used by a Marketo webhook because it supports Basic Auth (username/password) credentials carried along with a lookup request. So it’s compatible, even though it’s not advertised with the word “webhook”.

 

A basic HTML form’s action URL is typically webhook-compatible as well. By definition, that URL expects x-form-www-urlencoded keys and values in a single GET or POST. So as long as the webhook-enabled app supports Form/URL encoding (Marketo does) then you can post from the server side as easily as from the client. That’s why you can use a webhook to call Marketo’s scriptless forms endpoint  /index.php/leadCapture/save to do some cool cross-lead stuff.

 

(Yes, CSRF tokens break this compatibility.)

 

[5] In some cases, a database has a native HTTP/S endpoint so there wouldn’t technically be a different tier for code. SQL Server used to have an XML service inside it, for example, but that’s been removed.

 

Document databases typically have an HTTP API built in. But if they require multi-request OAuth authentication, that endpoint would end up incompatible with webhooks. Similarly, OData services are always web-compatible but not necessarily webhook-compatible.

 

[6] Where supported, you can ask for more than one range, but for simplicity let’s assume it’s only a single contiguous range and a standard (non-multipart) HTTP 206 response.

 

[7] Simple ASCII to avoid confusion about the length of a UTF-8 file with or without the 2-byte BOM.

 

[8] True, there’s no general reason why a webhook response can’t be treated as binary and then chopped down further, 8 individual bits instead of 1 ASCII character (and in turn mapped to 8 Boolean fields).

 

But specifically within Marketo, this isn’t supported. In order for Marketo to treat B as 01000010 and then map 01000010 back to this...

 

Lead.Field 1 = false
Lead.Field 2 = true
Lead.Field 3 = false
Lead.Field 4 = false
Lead.Field 5 = false
Lead.Field 6 = false
Lead.Field 7 = true
Lead.Field 8 = false

 

... you’d have to pass the original B response to a whole other webhook, and that webhook would need to have real code behind it (albeit simple) as opposed to a static file. So it would defeat the purpose of today’s no-code experiment.

 

[9] Using Range: against dynamically generated resources (not static files) can be sketchy and cost more in server resources than it saves in bandwidth. But that’s a different situation from this post. Imagine you have a service that dynamically creates image files with certain transforms (sepia, bunny face, whatever). It’s likely impossible to know what the 999th through 1000th bytes will contain in advance. So it needs to render the transformed image to disk or memory, then jump to the 999th byte. If the whole image might only be 1K, that’s really wasteful and it would be better if it didn’t accept Range requests at all.

Say you have Badges Earned Custom Object ($BadgesEarned_cList in Velocity) to store a lead’s community achievements. Values are like so:

 

{description=Onboarding, points=200}
{description=Influencer, points=500}
{description=Helpful, points=200}
{description=Evangelist, points=350}

 

And you want to display the lead’s badges in a table with N alternating background colors. For example, with 3 alternating colors (leave aside the garish color scheme, that's not the point!):

 


This is is an old-school task that just about any template language can handle.[1] To start, we’ll stick with generic methods; later, we’ll see how Velocity’s Alternator helper class can save 1 or 2 lines of code.

 

As in other languages, alternation means a loop plus a modulo function (or native modulo operator[2]) which in Velocity is MathTool.mod:

 

#set( $rowColors = ["#ff4400", "#ccff00", "#0099cc"] )
#set( $numRowColors = $rowColors.size() )
#if( !$BadgesEarned_cList.isEmpty() )
<table style="border-top:10px solid #aaa;">
<tr bgcolor="#fff">
<td>Activity</td>
<td>Points Earned</td>
</tr>
#foreach( $badge in $BadgesEarned_cList )
#set( $rowColor = $rowColors[$math.mod($foreach.index, $numRowColors)] )
<tr bgcolor="${rowColor}" style="color:#333;">
<td>${badge.description}</td>
<td>${badge.points}</td>
</tr>
#end
</table>
#end

 

So I first set up an ArrayList, $rowColors, with the colors I want to alternate (in order from the top).

 

Then on every loop I get the modulo N of the loop index where N is the number of items in $rowColors.  (The list happens to have 3 items now, but the code dynamically adjusts if you add/remove colors.)

 

  • The first time through the loop, the loop index is 0.  0 modulo 3 is 0, so that means I use index 0 of $rowColors ($rowColors[0]) in turn.
  • Next time, the loop index is 1. 1 modulo 3 is 1. So $rowColors[1].
  • Next loop index is 2. 2 modulo 3 is 2: $rowColors[2].
  • Now the fun begins. Next time through the loop, the loop index is 3. 3 modulo 3 is 0. So we use $rowColors[0] again.
  • And that’s how alternating colors are done!

 

The HTML output is like so:

 

<tablestyle="border-top: 10px solid #aaa;">
<tr bgcolor="#fff">
<td>Activity</td>
<td>Points Earned</td>
</tr>
<tr bgcolor="#ff4400"style="color:#333;">
<td>Onboarding</td>
<td>200</td>
</tr>
<tr bgcolor="#ccff00"style="color:#333;">
<td>Influencer</td>
<td>500</td>
</tr>
<tr bgcolor="#0099cc"style="color:#333;">
<td>Helpful</td>
<td>200</td>
</tr>
<tr bgcolor="#ff4400"style="color:#333;">
<td>Evangelist</td>
<td>350</td>
</tr>
</table>

 

Simplifying a bit with AlternatorTool

You’ve seen above that without any “alternator-aware” code you can get exactly the output you want.

 

Velocity does offer a cool tool that abstracts away the modulo stuff. But as you can see in the Alternator source, it uses exactly the same method, just as compiled Java:

 

 

An Alternator might be infinitesimally faster because it’s compiled, but you’d never notice this in reality. The reason to use Alternators is to save lines of code, and every line does count in a language as verbose as VTL. Here’s how to get the same output using an Alternator:

 

#set( $rowColors = $alternator.manual(["#ff4400", "#ccff00", "#0099cc"]) )
#if( !$BadgesEarned_cList.isEmpty() )
<table style="border-top:10px solid #aaa;">
<tr bgcolor="#fff">
<td>Activity</td>
<td>Points Earned</td>
</tr>
#foreach( $badge in $BadgesEarned_cList )
#set( $rowColor = $rowColors.getNext() )
<tr bgcolor="${rowColor}" style="color:#333;">
<td>${badge.description}</td>
<td>${badge.points}</td>
</tr>
#end
</table>
#end

 

16 lines instead of 17: yay! And a little easier to read, maybe.

 

Create an Alternator object by passing a List to $alternator.manual. Velocity then handles the modulo-based loop internally, whenever you call getNext().

 

(If you’re confused about the difference between auto and manual, I don’t blame you, but trust me that manual + getNext() is what you always want, especially because of the more advanced application we’re going to do next.)

 

Alternating between complex objects

Alternating between Strings (single hex colors like "#ff4400") is the simplest task.

 

But let’s say you want to vary the background and foreground (text) colors for optimal contrast:

 


Now, you’ve got a set of 3 “color schemes” and each scheme has 2 characteristics (background and foreground).  You should already be thinking: an array of objects!

 

And that’s exactly what I do here, passing an [] of {}s – an ArrayList of LinkedHashMaps, technically – to AlternatorTool:

 

#set( $rowColorSchemes = $alternator.manual([
{
"bg" : "#ff4400",
"fg" : "#fee"
},
{
"bg" : "#ccff00",
"fg" : "#333"
},
{
"bg" : "#0099cc",
"fg" : "#ccff00"
}
]) )
#if( !$BadgesEarned_cList.isEmpty() )
<table style="border-top:10px solid #aaa;">
<tr bgcolor="#fff">
<td>Activity</td>
<td>Points Earned</td>
</tr>
#foreach( $badge in $BadgesEarned_cList )
#set( $rowColorScheme = $rowColorSchemes.getNext() )
<tr bgcolor="${rowColorScheme.bg}" style="color:${rowColorScheme.fg};">
<td>${badge.description}</td>
<td>${badge.points}</td>
</tr>
#end
</table>
#end

 

Each HashMap has two keys, bg and fg. Notice I only call getNext() once per iteration to advance to the next object in the List.

 

The generated HTML:

 

<tablestyle="border-top:10px solid #aaa;">
<tr bgcolor="#fff">
<td>Activity</td>
<td>Points Earned</td>
</tr>
<tr bgcolor="#ff4400"style="color:#fee;">
<td>Onboarding</td>
<td>200</td>
</tr>
<tr bgcolor="#ccff00"style="color:#333;">
<td>Influencer</td>
<td>500</td>
</tr>
<tr bgcolor="#0099cc"style="color:#ccff00;">
<td>Helpful</td>
<td>200</td>
</tr>
<tr bgcolor="#ff4400"style="color:#fee;">
<td>Evangelist</td>
<td>350</td>
</tr>
</table>

 

 

That’s it for Alternators for today! But I have another post ready to go on how to combine Iterators and Alternators for some advanced fun. Stay tuned.

 

 

 

Notes

[1] Many template systems have macros for N = 2, like isOdd and isEven, built-in. Few offer anything as flexible as AlternatorTool.

 

[2] Indeed, Java operators like % are semi-supported in Velocity as well. But the VTL parser is much stricter than Java’s, leading to hard-to-debug problems. I always use the MathTool methods instead.

The post title is a bit of a mouthful, but if you've been bitten by a certain feature gap you'll know what I mean.

 

One of the first things you learn about Wait steps is they don't have a literal Add Choice option.

nelson pointing at marketo

 

This can be frustrating when you want to vary the Wait delay based on  runtime conditions (that is, conditions you can’t know until the person has qualified and entered the flow) importantly including no delay at all.

 

But with a tiny bit of work, you can simulate Wait step choices.

 

It’s a matter of managing a Date/DateTime field, earlier in the same flow, using Marketo’s simple plus/minus support.

 

Here’s such a field:

 

field mgmt

 

And here’s a flow that uses that field to manage a subsequent Wait step:

 

flow steps

 

This approach works because of 3 convenient truths:

 

  • Change Data Value is synchronous within a single flow[1]
  • Date tokens understand a few math operators
  • Wait steps using a Date token will be skipped if the Date token is empty

 

Truth be told, I don't always endorse this tack over multiple Smart Campaigns. Whatever’s making you want drastically variable Wait periods (other than implicitly variable periods like wait-until-anniversary) may mean the lifecycle is going to differ in other ways as well, in which case discrete SCs help you keep your sanity. But it's there if you want it.

 

P.S. Yes, you can also use a Number {{my.token}} for the delay itself, a setup that might be almost too cool to follow! Or a Text {{my.token}} (don’t know why you’d choose this over Number, though) as long as you don’t include the unit  (“days”, “hours”) in the value, keep that hard-coded in the Change Data Value box.

 




Notes
[1] “Synchronous” meaning the New Value is guaranteed to be readable in the next flow step. Contrast this with, for example, webhook-based updates, which are asynchronous (background) value changes.

Despite instructing a Community member to “search my posts” the other day, I ran a search myself and there wasn’t a one-stop explanation of what Do Not Track (DNT) means in Marketo (on a deeper technical level than you get on the official doc page). So here goes.

 

As you probably know already, there are 2 DNT options, Ignore and Support:

 

 

We won’t worry about Ignore.

 

But what does it really mean to choose Support? On a technical level, it means one specific thing:

 

If a user’s browser sends the DNT: 1 HTTP request header along with a Munchkin-logged pageview or link click, Marketo will not save the activity to the Activity Log database.

 

So here are some things Do Not Track = Support does not do:

  • it does not stop gathering Clicked Email stats: email clicks are still tracked unless you separately turn off link tracking
  • it does not stop Munchkin JS libraries from loading
  • it does not stop Munchkin from initializing and setting its _mkto_trk cookie
  • it does not stop Munchkin from sending a Visit Web Page (assuming you're using the default configuration which always sends a VWP on startup)
  • it does not stop Munchkin from sending a Clicked Link for <a> links on the page

 

But again, here's the very important thing it does do:

  • it stops the Marketo platform from storing the Visit Web Page and Clicked Link hits sent by Munchkin

 

 

Why not stop Munchkin completely?

It's not that Marketo would not like to be more proactive on the browser side, I'm sure. But the weirdest thing about DNT is there's no programmatic (let alone cross-browser) way to know if the user has set a preference! Ergo, you cannot know if the person would've wanted you to turn off Munchkin downloading/initialization/hit logging. You have to dumbly send the hit in all cases, then the server will discard it if it's accompanied by the “please ignore me” header.

 

The privacy appeal of having the DNT setting be unreadable in the browser is clear — it's the equivalent of an HTTP-only cookie that can't be seen from JavaScript — but it certainly creates confusion. For example, someone with DNT enabled who’s also running Ghostery or similar will still see that the Munchkin tracking JS was blocked, which is suboptimal: ideally, it wouldn’t show up at all. You might seem like you’re being worse corporate citizens than you actually are. (A link on your Privacy Policy confirming that you honor Do Not Track is useful.)

 

 

The browser's-eye view

The browser sending the DNT: 1 header is a prerequisite, of course. Privacy-oriented browsers do this by default; other browsers do it in Private/Incognito/InPrivate mode only; the the rest do it for all pages/tabs/windows when selected. Here's the setting in an older version of Chrome, for one of a zillion examples, which will send DNT: 1 for all pages viewed in this user profile:

 

 

And here’s a screenshot of the HTTP request for the main document, showing the header:

 

 

And Munchkin’s Visit Web Page XMLHttpRequest, showing the same HTTP request header and its acknowledgment in the response:

 

Update 2019-11-07: This tip must be paired with the HTML and JS hints in this newer post to fully avoid accidental blocking by Tracking Protection.

 

Buried in a bunch of my Nation responses is this ginormously important guideline: whenever possible, use your Marketo LP domain in your form embed code instead of the default //app-something.marketo.com.

 

That is, if the embed code in the Marketo UI is:

 

<script src="//app-sj01.marketo.com/js/forms2/js/forms2.min.js"></script>
<form id="mktoForm_999"></form>
<script>MktoForms2.loadForm("//app-sj01.marketo.com", "123-ABC-456", 999);</script>

 

and your primary Marketo LP Domain (or a Domain Alias) is:

 

https://pages.example.com

 

then edit the embed code (after pasting on your external site) to be:

 

<script src="//pages.example.com/js/forms2/js/forms2.min.js"></script>
<form id="mktoForm_999"></form>
<script>MktoForms2.loadForm("//pages.example.com", "123-ABC-456", 999);</script>

 

The only and I mean only reason to avoid this change is if your external site requires SSL (https:) but your Marketo subscription does not yet include Marketo's SSL add-on. (Built-in browser security won't allow forms to load if you're in this unfortunate situation.)

 

In all other cases, you can and should switch over. Like, yesterday. That includes (1) when your external domain and Marketo LP domain both run over SSL (best practice in 2019); (2) when neither uses SSL (eh, it works); and (3) when the external site doesn't, but the Marketo domain does use SSL (strange but possible).

 

What's this about?

It's about tracking protection. If someone browses your site using Firefox with TP turned on, or with Ghostery or a similar plugin, they will not be able to load forms from app-*.marketo.com, because they can't load anything from domains matching *.marketo.*.

 

It makes sense that Munchkin (from munchkin.marketo.net) would be blocked, of course. That's what anti-tracking features/plugins are designed to do. But forms can be thrown out with the bathwater, if you will.

 

Yes, it's not really fair for all that matters! because form submissions require deliberate user action, and they don't inherently “track” anything but the Filled Out Form activity itself (assuming Munchkin cookies are blocked and all existing cookies were deleted).

 

But it's something we have to live with: Munchkin is fairly described as a tracker, Munchkin comes from the domain marketo.something and the major marketo.{tld} domains are of course all owned by Marketo. So, fair or not, privacy wins out... even if that means forms leave a blank space on your page for some end users.

 

By loading from your Marketo LP domain instead, you fully comply with the anti-tracking plugin (since you aren't dropping any new cookies or logging any more pageviews/clicks with Munchkin blocked) but also allow forms to be seen and be filled out. So do it!

 

 

Why isn't it the default?

Because of the SSL exception described above. Apparently, the Embed Code textbox in the UI and the underlying domain setup can't communicate. So the LP domain can't safely be the default, as not everyone can use it.

 

 

We're in the process of getting SSL on our Marketo LPs, but not sure if it's ready

If you have a pending order with Marketo, you can quick-check the state of affairs (for the purposes of this blog post) by loading the Forms 2.0 forms2.min.js in your browser. Here's Firefox's way of saying your custom cert isn't installed yet:

 

A clear symptom of broken custom JS is when you see form fields, in URL-encoded format, in the location bar after clicking Submit:

 

5c886b174c52a500bf5f9c28_fields_in_url_url.png

 

You should get to know the cause on sight: an uncaught error has been thrown in a custom Forms API onSubmit function.

 

To replicate this, just set up an otherwise empty page with your form embed and this purposely bad JS:

 

MktoForms2.whenReady(function(form){
form.onSubmit(function(form){
form.oohlala(); // the property `form.oohlala` will never exist
});
});

 

This code won't throw any early errors on page load, because it doesn't have a syntax error. There's no way for the JS engine to know that form.oohlala will end up being a nonexistent method at runtime, it just adds the onSubmit listener happily onto the form.

 

However, when the form is actually being submitted, Marketo runs your custom onSubmit function. Then the browser gets serious and throws a hard TypeError, as you can see in the Dev Tools console (make sure to check off Preserve Log or you'll miss it people don't realize this behavior is always, without exception, a JavaScript error because they fail to prepare their debugging environment ahead of time).

 

Here's the runtime error:

 

5c886b174c52a500bf5f9c28_fields_in_url_console_error.png

 

But why does a JS error result in fields in the URL?

It's simple. Marketo forms use a true HTML <form> tag (this is A Very Good Thing™) and standard <input>/<select>/etc. elements under the hood.

 

The Forms API, among many other things, is responsible for transforming the standard W3C submit event into an elegant cross-domain POST to your Marketo instance.

 

In an error-free environment, the standard submit is swallowed by the Forms API's robust set of handlers. The event is triggered, but its default action that is, sending fields to the form's action (destination URL) using its method (HTTP method) is turned off. The API's non-default actions take over.

 

But when there's an error thrown within the Forms API listener stack, the form reverts to standard HTML form behavior as the default action doesn't get a chance to be turned off.

 

The Marketo <form> element doesn't have a specific action or method[1], i.e. it looks basically like so:

 

<form>
<input name="Something">
<input name="SomethingElse">
<button type="submit">Submit</button>
</form>

 

The default value for method is GET, and the default value for action is the empty string which represents the current URL. So the effective markup is:

 

<form
method="GET"
action="https://whatever.example.com/the_current_page.html">
<input name="Something">
<input name="SomethingElse">
<button type="submit">Submit</button>
</form>

 

When the button is clicked with this markup, and there's a JS error, the browser reverts to a standard GET of the current URL with form fields in the query string:

 

https://whatever.example.com/the_current_page.html?Something=a%20value&SomethingElse=another%20value

 

Whenever you see this behavior in your browser, realize your forms aren't working at all (even though the form seemingly "posts" somewhere). So fix em!

 

 

 


Notes

[1] Nor does the <button> have the newfangled formaction or formmethod attributes, which would be used if present.

Printing fallback content when a Lead field is blank is a basic Velocity task. You can do it in a few lines of clunky code... but that's a few lines too many!

 

Seeking a one-liner, you might reach for Velocity's built-in $display.alt. But that won't fill the bill in Marketo-land. You see, $display.alt($field, $fallback) outputs the fallback if the field is null. But null isn't the same as the empty String that Marketo uses for unfilled Lead fields.

 

Therefore, the code

 

Dear ${display.alt($lead.FirstName,"Friend")},

 

will only ever output

 

Dear Joe,

 

or

 

Dear ,

 

It will never fall back to

 

Dear Friend,

 

because Marketo ensures $lead.FirstName is always a String of some kind, never null.[1]

 

So without any other tools at your disposal, you're left with a typically wordy #if block:

 

Dear ##

#if( $lead.FirstName.isEmpty() )

Friend,##

#else

$lead.FirstName,##

#end

 

This will work fine, but as your scripts get cluttered with repeats of this same structure, you start to go a little crazy.[2]

 

The good news is there's a short Velocimacro you can include globally that greatly lightens the load. Once you set up the #displayIfFilled macro, you can reduce the logic to one easy-to-read line:

 

Dear #displayIfFilled($lead.FirstName, "Friend"),

 

Building #displayIfFilled

Here's the macro definition:

 

#macro ( displayIfFilled $checkValue $fallbackValue )
#if( !($checkValue.isEmpty()) && !($checkValue == $display.get("0")) )
$!checkValue##
#else
$!fallbackValue##
#end
#end

 

As you can see, #displayIfFilled takes 2 self-explanatory arguments: the field to check for filled-ness, and the fallback value.

 

#displayIfFilled is designed to treat null the same as the empty String. It thus covers a superset of the cases covered by $display.alt, so you may never be tempted by the latter function again.

 

Step further out: #displayIf

We can abstract the functionality of #displayIfFilled into a more general #displayIf:

 

#macro ( displayIf $truePredicate $trueValue $falseValue )
#if( $truePredicate )
$!trueValue##
#else
$!falseValue##
#end
#end

 

#displayIf takes 3 arguments: a Boolean, the value to output if the Boolean is true, and the output if the Boolean is false.

 

To emulate #displayIfFilled, pass an isEmpty() check as the first arg (the $truePredicate):

 

Dear #displayIf($lead.FirstName.isEmpty(), "Friend", $lead.FirstName),

 

Lots of tricks up your sleeve with #displayIf. Say you want to switch output based on a specific non-empty value

 

Your #displayIf($lead.trialType.equals("Other"), "VIP", ${lead.trialType}) trial is almost over!

 

or output based on a date/time property

 

Good #displayIf($calNow.get($calFields.AM_PM).equals($calFields.AM), "mornin'", "aft'noon")!

 

or anything that can be expressed as if-then-else!

 

Of course, #displayIf can be overused; past a certain point of complexity, you should be using #if-#else on separate lines. But where a one-liner doesn't hurt readability, I say use it. #displayIf is critical to my sanity (if I have any left) as an avid Velocity coder.

 

Stay functional

There's another detail that you'd eventually learn on your own, but I'll spoil it to save you time.

 

When passing macro arguments in parentheses, you can include any chain of function calls but not syntactical expressions. For example, though Boolean operators and expressions are valid in other parts of Velocity, you can't do:

 

#displayIf(!$lead.FirstName.isEmpty(), "${lead.FirstName}'s", "Your") special offer is ready!

 

That won't compile because of the !. Velocity's parser doesn't accept operators in that place (nor would it accept <, > or == operators there).

 

Instead, either chain with the equals() function:

 

#displayIf($lead.FirstName.isEmpty().equals(false), "${lead.FirstName}'s", "Your") special offer is ready!

 

or as some programming style guides suggest anyway don't rely on negated Booleans and instead put your true case first:

 

#displayIf($lead.FirstName.isEmpty(), "Your", "${lead.FirstName}'s") special offer is ready!

 

To be clear, this doesn't mean you can't use all manner of operators and expressions to construct Boolean values, you just can't use them directly inside the parentheses when calling a macro. This will work fine:

 

#set( $hasCompany = !$lead.Company.isEmpty() && !$lead.Company.equals("N/A") )

Is #displayIf($hasCompany, $lead.Company, "your family") in the mood for pizza?

 

Disrupts the dream of a one-line solution, though.

 

What's with $display.get("0")?

Ah, yes. I don't want to overwhelm you earlier with the details of null checking in Velocity.

 

$display.get("0") (up above in the first #displayIfFilled macro) gets the value of a reference that is guaranteed to not exist, i.e. guaranteed to be null, in any Velocity context.

 

Why is it guaranteed to not exist? Because neither Java nor Velocity variable names are allowed to begin with a number.

 

Why not compare to the literal null? Because and this reality sneaks up in other important places in Velocity there is no four-letter keyword null! The null-ability of injected data is honored, even favored, in Velocity, like by $display.alt as noted above. But it doesn't have a keyword to create new null values easily.

 

So you have to find a roundabout way of getting a reference to a null value. Elsewhere on the net, people say "just use a variable you didn't #set anywhere else, like $abcdefg, as that will naturally be null." The flaw in this reasoning is nothing actually stops someone else (either a future coder or a current collaborator) from using $abcdefg for something else. So I prefer to use a reference that cannot exist even by coincidence.

 

 


 

 

Notes

[1] In the Marketo Lead/Person world, you aren't gonna run into literal null values, but rather empty strings. This is true despite non-filled fields being represented as [null] or NULL in parts of the Marketo UI. You can and will encounter null with Custom Objects, though. That's the stuff of another post.

 

[2] Yes, you could smush the VTL into one line. But if you think Dear #if($lead.FirstName.isEmpty())Friend#else${lead.FirstName}#end, is sufficiently readable, you're made of stronger stuff than me.

This JS string-splitting approach is a sure code smell, but I see it all the time on LPs:

 

var partsOfString = stringifiedLeadInfo.split("|");
var firstName = partsOfString[0];
var lastName = partsOfString[1];
var companyName = partsOfString[2];
var phoneNumber = partsOfString[3];
/* ... and so on and so on... */

 

 

Presumably stringifiedLeadInfo when the code was first written was a string like

 

Sandy|Whiteman|FigureOne, Inc.|212-222-2222

 

But this code is clearly fragile: there's no guarantee that the “magic numbers” 0, 1, 2, and 3 will continue to represent the same data (business-wise) inside the string.  If order shifts around at the source, or if a new data point is added in the middle, all these lines may need to change. That leads to bugs.

 

Instead, use what I call a header string. It's nothing more than a sample string containing the variable names in the currently expected order

 

var delim = "|",
    stringifiedLeadHeaders = "firstName|lastName|companyName|phoneNumber",    
    leadHeaders = stringifiedLeadHeaders.split(delim);

var leadInfo = stringifiedLeadInfo
                 .split(delim)
                 .reduce(function(acc,next,idx){
                   acc[leadHeaders[idx] || "Unknown_Property_" + idx] = next;
                   return acc;
                 },{});

 

 

Now, leadInfo is a simple object:

 

{
  firstName: "Sandy",
  lastName: "Whiteman",
  companyName: "FigureOne, Inc.",
  phoneNumber: "212-222-2222"
}

 

 

And you only need to change the header string if the data starts coming in differently. No other lines need to be added or changed. 

 

(I also made the delimiter a variable, ’cuz that could change too. And if new data points appear in the data before you add them to the header, they're given automatic names like Unknown_Property_5 to help signal the change.)

 

Please use this — or something along these lines, there are other methods with the same effect — in your code. It makes it less painful to read (scrolling through 25 variable assignments ain’t fun) and because of my curious specialty I spend a lot of time reading other people's stuff.

 

Do it in Velocity, too

The equivalent can be done in any language. Always better than magic numbers, IMNSHO. Here's the comparable VTL:

 

#set( $delim = "\|" )
#set( $stringifiedLeadHeaders = "firstName|lastName|companyName|phoneNumber" )
#set( $leadHeaders = $stringifiedLeadHeaders.split($delim) )
#set( $leadHeadersCount = $leadHeaders.size() )
#set( $leadInfo = {} )
#foreach( $next in $stringifiedLeadInfo.split($delim) )
#if( $foreach.index < $leadHeadersCount )
#set( $void = $leadInfo.put($leadHeaders[$foreach.index], $next) )
#else
#set( $void = $leadInfo.put("Unknown_Property_${foreach.index}", $next ) )
#end
#end

 

 

The main difference here (Velocity's verbosity aside) is that Java's String.split always treats the delimiter as a regular expression, not a simple string. Since the pipe symbol "|" has special meaning in regex-land, I escaped it as "\|" to treat it non-specially. Character class "[|]" would also implicitly escape the pipe.

 

(JavaScript's split(delim) also supports regexes, but the language can tell the difference between a "string" and a /regex/ so you don't need to escape strings.)

 

Better yet, don't give yourself the need to split

It could be argued that all string splitting is smelly, and this improvement is just code cologne.

 

Indeed, the best string-splitting code is the code you don't have to write, because you store multivalued fields as JSON or some other well-known, self-describing  format. Private formats with pipes, semicolons, or commas are to be avoided when possible. We'll never completely get away from them, though, and they’re admittedly efficient storage-wise.

Or not so much “beware” as ignore an Email Bounced Soft that doesn't have an associated Category.

 

Far too many posts and practices imply that grouping Email Bounced Soft-s together with a simple filter is harmless. The thought is that you may want to separate Category 3, Category 4, and Category 9 but you don't have to.

 

The approach is implicitly encouraged by the official docs:

 

 

But this doc is misleading, because there's one type unlisted: call it the No-Category (NC) Soft Bounce.

 

Guess what falls into NC? If you read my posts avidly, you might suspect it's something code-related. Yep: a Velocity token parsing error results in an NC Soft Bounce.

 

Here's one I triggered at a client just now by accidentally deleting the close parenthesis ) from a #set directive:

 

 

I quickly fixed the error before sending again in this case.

 

But imagine if it took more troubleshooting, and each send triggered an operational campaign (or qualified for the equivalent daily batch) that counted Email Bounced Soft activities without a Category constraint, setting Marketing Suspended = true when people exceeded a threshold. There are well-known recipes for Marketo database cleaning out there that do just that!

 

Don't make that mistake. Constrain Email Bounced Soft by Category [is not empty] to catch only the bounce types which may call the lead's info into question. Velocity coding errors must not be held against the lead, as they're completely in your court.

 

You can view just the Velocity parsing errors like this:

 

 

 

P.S. and N.B.: the Velocity errors that are surfaced as NC Soft Bounces are Velocity Template Language (VTL) parsing errors. That is, forgetting the #end of an #if, missing parentheses or brackets, or other broken syntax. That doesn't include Java language errors thrown by syntactically valid VTL, like trying to get() a nonexistent index in an ArrayList of Custom Objects. The latter type of error shows up verbosely in the Preview UI, but if it makes it to send time, it's swallowed by the system. You will only see the Sent Email without a corresponding Delivered Email in this case. Obviously that's pretty ambiguous. So test, test, test your Velocity!

Filter Blog

By date: By tag: