Webhook related issue.

[milanjain2208]

Can we retrieve comprehensive information about updated Marketo fields within our backend?

Approach I've taken:
I've established a Marketo webhook and integrated the endpoint URL.

Additionally, I've configured a trigger to initiate a webhook request to our endpoint whenever there's a "data value change" in the first name field.
The webhook setup is added to the campaign flow.

Although this is working, I'm encountering a couple of challenges:

1. I'm aiming to collect data for all updated fields, not solely the first name. Is it necessary to manually set up individual triggers for each field?
2. Presently, my endpoint is triggered multiple times for various field changes. Is there a way to ensure that my endpoint is only contacted once even if there are updates to multiple fields of a lead at once?

[Darshil_Shah1 · Accepted solution]

What is your exact business requirement for this? And would your systems really benefit from it? are some questions I usually like to get started before jumping in.

If you want to send update notifications to your backend stack by making a webhook call to your service, then this particular method you’ve outlined might put a strain on your Marketo instance, especially if you do this for all the fields, as a typical instance has a lot of updates being done daily, which would all lead to making an outbound webhook call to your service posting the data change. Doing so would likely hamper the performance and introduce delays in your campaign processing. FYR- This is an ideal use case for Marketo's real-time lead activity data stream given that receiving real-time updates is critical for your backend system. You could subscribe to receive the following data via streams. 

1. Lead Changes – all changes on all fields and new Leads
2. Lead Activities
3. Deleted Leads
4. All Custom Objects on the Lead (if requested). It’s all or nothing at this time.

FYI, the last time I checked this with Marketo for a client regarding streams a couple of months back, they let me know that the beta for this had ended, and they would roll this for GA in the next few months (you'd still need to contact Marketo to enable streams in your instance).

Also, with your as-is approach, I'd not recommend setting individual triggers for each field as that'd certainly bog down your instance with too many live trigger campaigns. I don't see it being too scalable. If your backend stack isn't expecting real-time updates, you can always use Bulk Activity Extract API to pull the Data Value Change activities with New+Old values from Marketo and push them to your backend regularly (e.g., nightly basis). Most users go via this route of pushing the CDV activities/Person updates to their systems via Bulk Activity Extract API.

[SanfordWhiteman]

Like Darshil says, a webhook-based approach isn’t appropriate for detecting any change to any field and triggering immediately.

You may still be able to use a webhook, but by calling it as a daily batch as opposed to immediately.

There isn’t a trigger that applies to all Data Value Changes at once, but you can use an Updated At filter in a batch Smart Campaign’s Smart List to qualify anybody who had any changes today. Then use Request Campaign to call a trigger SC that runs Call Webhook. This’ll be fine for up to ~100,000 calls per day as long as your receiving server responds quickly.

The streaming feature is indeed better for this but requires a robust infrastructure.