Boolean Count

[SanfordWhiteman]

We have nineteen boolean fields and only scoring off of three took roughly two hours to completely update the new score field.

So I don't think this is going to work out for us.

Are you sure that the performance is actually linear (i.e. with 6 does it take 4 hours)? How many leads did you run against in those 2 hours?

[Anonymous]

Not sure if it's linear but two hours is still too much time.  I believe in those two hours roughly 27k leads were updated

[SanfordWhiteman]

I'll run some benchmarks on my primary instance to see how FlowBoost fares in comparison, but I'm confused about why 2 hours is too much time for a weekly batch.

[SanfordWhiteman]

Benchmarked FB and was able to get 125ms/call, so that'd be about 60min as opposed to your 120min.  Not instantaneous, but I think I'm impressed rather than disappointed! I'm wondering what your specific objective is (or what leads you to think Marketo can be made to meet it ).

[Diederik_Marte4]

Hi Josh,

It is probably due to Marketo's webhooks. They are NOT designed for high volumes (anything over 1000). Marketo strongly discourages webhooks for this. They recommend a ReST API instead. Though that also has a limit of 300 leads max per call.

The only native ReST API integrated formula suite is CalcItNow.com (also on Marketo launchpoint: CALCITNOW - Field formula calculations – Marketo LaunchPoint ).

with a complex scoring formula with multiple fields involved I was able to update over 3000 leads in less than 1 minute (please note the 5 minute sync interval before changes are picked up though).

Just PM me if you have any questions.

Best,

[SanfordWhiteman]

The claim of a webhook max of 1000 (per day? per batch?) is mythical FUD. It is not supported by real production experience.

That REST can be batchified and thus much faster in toto (unless you run out of API calls) than outbound integrations was never in dispute. I hope you can find selling points that are based on empirical evidence: it should be enough to prove substantively higher daily capacity with real-world numbers, not imaginary ones.

[Diederik_Marte4]

Hi Sanford,

You are right. I don't have data to back up the webhook maximum load. I'm merely quoting what Marketo is saying during their API office hour calls. I just suggested this might explain the performance that is being experienced (27K in 2 hours). I'm not trying to attack anyone here. I don't know enough about FB. I do know formulas is just one of the many things it can do right? But as Josh mentioned he's looking for higher performance on formulas specifically, I just mentioned our solution as an alternative. (P.s. FB looks really powerfull from the screenshots I could find on your blog). Do you have moe documentation? I might have some customers who might be interested in it.

---

Hi Josh,

I just ran a test on our uswest server (at AWS) with 25K records, each with 10 booleans, to store the outcome in the lead score field. All records were loaded, calculated, and stored back within 15 minutes. This in batches of 300. So about 10 seconds for each batch.

This could be faster, but capacity is intentially limited to 1 batch per formula per customer at the same time. The only exception is for the first batch of 300. So if 10 formulas, and less than 300 leads per formula need to be calculated. All 10 could run at the same time. But anything after 300 leads (so probably for bigger manual data changes you do), the next batches are sequential. This translates to the 300 per 10 seconds. I hope this helps. You can do a free trial by the way!

[SanfordWhiteman]

Thanks for clarifying, Diederik. On-the-ground tests show webhooks are fine with 50,000+ calls per day. This isn't specific to FB (though the fact that we autoscale helps, since we never run out of juice on our side). I'll run some more benchmarks this week and post about it.

With webhooks, the faster you can consistently return results (as opposed to point-in-time/synthetic benchmarks) and the smaller you keep your payloads (both in and out) the better. Building for webhook traffic involves specific challenges that would look like microoptimizations in any other case, like worrying about 100ms differences in process (re)start times, maintaining burstable performance, things like that.

I wasn't clear from this thread if Josh ended up calling a webhook at all, or whether he was talking about Change Data Value performance (fully within Marketo); he didn't try FlowBoost. But without making those optimizations, a service that works great for a handful of hits will eventually crumble.

With the above optimizations in mind, though, you can get "wire speed" performance from webhooks, that is, if your 'hook maintains a roundtrip time of 120ms from a barebones HTTP client, Marketo isn't adding substantive overhead to that. So we can get ~8 results/sec for a synchronous webhook and maintain that rate through ~500/min and ~30,000/hr. (Async style, for when you just fire-and-forget to a remote service/database, is about 25% faster.)

But for sheer performance, you can rip way more out of REST, no doubt whatsoever.

I can see why Marketo support is championing the REST API because, well, it looks good to have your own API be the integration of choice! Also, as far as maintaining control, what we see as the amazing power of webhooks might look like a scary "Wild West" situation from another angle. After all, the story goes that Munctions were taken away because they were perceived as too dangerous, or at least too dangerous to be a native function! I think that's weird given that you can wipe out data in many ways in Marketo (including a REST app, if it has the right permissions) but I guess offering one more way directly to admins was seen as bad karma, and webhooks seem like they occupy the same space.

I think CalcItNow is making perfect use of Marketo REST, since field updates are totally bulkified and won't run into the bizarre confusions of the Activity API. My principal problem with the Marketo API is that the microscopic number of calls must be shared across all your integrations, and I'm continually horrified that it's advertised for non-bulk applications (a.k.a. DoS fests). In a controlled context, there's nothing better.

Finally, like you said, FlowBoost is indeed aiming beyond string & number manipulation, but we're darn good at that, too, so we can't deny that FB and CalcItNow are going head-to-head.  I imagine we should come up with some official differentiators if we're both going to live in this town.

[Diederik_Marte4]

Thank you too for clariying. I can imagine Marketo pushing for the API of course.

CalcItNow does also have two webhooks. One as stand-alone webhook. The other specifically for Marketo. you can the webhook, but it actyally uses the API to store process the data. The webhook merely takes the leadid as input.

It seems there is indeed some competition. Though one of the other solutions that are being developed right now, likely will also have some competition (not with you). I'm not particularly interested in going head-to-head. I just want to provide people a solution to their challenges, the same way I do with contracting in marketing ops.

I think FB allows you to do more advanced raw coding. CalcItNow is more wysiswyg wizard style. Making it easy for the less techy people to create something. Maybe it's like Eloqua vs Marketo? Eloqua has the ability to do more complex flow structures. The flipside is that it's harder to understand what that workflow does, simply by looking at it. I created CalcItnow 1.0 back in 2014, which was only used by people I know (not openly purchasable). Now 2.0 is publicly available.

P.s. We also have Hoosh in our space...

The other solutions planned for 2017 (hopefully befoe the summit) are in a different area than CalcItNow and FB. So no worries 😉

[Diederik_Marte4]

Don't forget: CALCITNOW - Field formula calculations – Marketo LaunchPoint