Web form API Sync into Marketo - Not Marketo Form - Help!

[SanfordWhiteman]

With (7) you open your Marketo instance -- and this form post implementation in particular -- to an easy Denial of Service (DoS) attack. Unless you're paying $ for more API calls, you only have 10,000 API calls per day to be shared among all of your integrations.  The daily limit can be just fine as long as you always batch + bulkify your calls. If you have multiple disparate integrations running, set a hard daily max for each or, much better, run them all through a central integration server that acts as your resource governor.

But if you let every form post consume an API call, then a malicious user with a simple home connection can quench your entire daily limit in less than an hour (the API itself limits you to 100 calls every 20s, so it would take all of 33 minutes to send 10,000 calls).  Even if s/he decided to be more leisurely about it, 10,000 is a really low number when you're talking about web hits.  By running what amounts to an API proxy on your public server you're inviting people to deny service to your other API apps -- and, even in the absence of other apps, they can deny service to your forms.

With (4) there's no additional rate limiting beyond 30 posts every 1m per IP address.  Marketo may have some unpublished anti-abuse measures at a much higher level, but those are likely not going to punish your whole instance.

[Justin_Norris1]

Makes sense about (7).

Regarding (4): so you mean the client-side form post (basically a hidden Marketo form) DOES have the same 30 post per minute per IP restriction that applies to server-side form post?

Does this actually mean that all embedded forms in general have this limitation? Since technically I would see little distinction between the hidden form submitting and a visible one.

If so that's kind of a big deal, since it's not hard to envision scenarios where you'd get 30+ form submits in a minute (e.g., sending out a big email blast with a form CTA) in which case you would have data loss.

[SanfordWhiteman]

30 submits in a minute from behind a single IP, though.  Which is still possible if a large corporation shares a NAT IP, but presumably worth the risk (one does wonder, though).

[Anonymous]

Does this meant that I can POST to Marketo web servers to create/update a lead record without having to count against my REST API call limit?
Does this work for things other than forms? (activities, custom objects, ...)

[SanfordWhiteman]

You can use the form submission endpoint from a server application, yes. But as you'll be connecting from the same IP, you can only use it once every 2 seconds.  It's not meant as a server API.  (That's why server-side form posts are not a real solution to the dangers of an unmanaged REST API, they just shift the problem around.  Instead of losing posts after the 10,000th hit, you can lose form posts during any period of high traffic.)

For occasional use, it can definitely be useful, but if you're concerned about 10,000 API calls a day and you're already properly bulkifying your calls, you're well beyond the range of "occasional"!  What kind of stuff are you doing with REST right now?

P.S. The form endpoint only processes data you can post from a true Marketo form. You can't access those other objects from forms.

[Anonymous]

You can't access those other objects from forms.

Thanks for the answer!

We're just in the beginning of integration. We're going the "Marketo-facing integration server cluster" part where anything aimed in that direction goes to a particular pool of workers, for batching and such. We have > 10 years of pre-existing systems, and want Marketo more as a GUI front-end for our marketers, than as a total e-mail/messaging platform, so our integration is somewhat different. Also, we don't have a user's e-mail until they've decided to register for our application (an online game) so we don't as much follow the "browse, whitepaper, newsletter, convert" path -- instead, we already have the user "converted" but now we want to monetize. (And our main channel to users isn't even e-mail -- it's in-game messaging, somewhat like Facebook messaging or a game Guild forum. Fun times!)

We're actually paying extra for more than 10k API calls per day, because we can't keep Marketo as our system of record for user data -- that has to be in our existing player/user database. Thus, even with batching, there will be significant churn. We also can't really use the public forms API, because the way un-authenticated users can update leads in the Marketo database would be found and gamed by our customers in very little time (they are a creative and tenacious bunch, ferreting out any chink in the armor to get a leg up.)

We actually have a subnet of our own IPs, so I guess if we worked hard, we could post once every two seconds per public IP address in our server cluster, but that sounds like a little more duct tape and chewing gum than I think would really be healthy. It's good to understand more about this option, though!

[SanfordWhiteman]

Actually, you can block people from updating existing fields via form fillouts. Set every field to have a default value and set Block Updates in Field Management.  You can also, to some degree, authenticate form fillouts by including a passphrase and checking equality using a webhook (discarding any updates that don't match) -- though you still use your instance's resources to throw away those malicious posts.

Regardless of the above hacks, it sounds like your volume demands batch interaction at every turn, not the singleton updates you get with forms. The multiplier on purchasing more API calls (300 x # of calls) is a better deal.

And good to hear that you're looking at this project with real EIPs in mind.  That's a relative rarity in my experience: most Marketo integrations are quite fragile. Typically nobody is aware of the fragility due to separation of the risk/security and marketing ops teams. As you point out the, let's say, "friendly threat" (as opposed to totally outside threat) is exaggerated in your industry so you have to pay attention, though really everyone should!