A recent project required mass bulk data updates between Dynamics and Marketo. The large amount of updates were expected to cause backlogs and disrupt the production environment integration. In exploring ways to mitigate the effects, we consulted with Adobe Marketo Support.
We also referenced this excellent post on Marketo / Salesforce Sync Limits & Priority Info, but found that several processes work differently in the Dynamics sync. In this post I will share a few findings.
- Unlike the Salesforce-Marketo sync, the Dynamics sync works on a first in, first out (FIFO) basis. Updates in the Dynamics sync do not batch by entity – they will be made in the order in which they occurred, based on the system mod timestamp.
- For example, if 50K Lead records are updated, and then 50K Contact records are updated, there is no throttling by object as described for an SFDC integration. The 50K Lead records will sync first.
- When a new Dynamics custom entity is added to the sync, all entity records will enter the sync (unless a custom sync filter is in place) and will be processed FIFO before ongoing operations continue in the sync.
- Two related points that are well known and documented:
- While the sync between Marketo and Dynamics is disabled, updates that are recorded in the Marketo Log are retained. When sync is re-enabled, the sync picks up, continuing through the updates by sysmod timestamp.
- The Marketo Log does not detect and record updates that are made outside the CRM UI. For the update to be synced, it cannot be performed directly in the backed database (e.g. via Azure Data Factory).
- There is no means of increasing the throughput (i.e. temporary increase in bandwidth) of the Dynamics sync. There is no multi-batch queueing (MBQ) for Dynamics integrations.
- Once the Account entity is in sync, account records are synced to Marketo regardless of whether a Contact is associated. Marketo keeps a lookup table of accounts in the background, which is also used for ABM. (This is probably the same in the SFDC sync.)
- Updates to an Account happen in one data table. The account update does not have to be populated to each associated “dependent” Contact – it is pulled in via the Account tab.
- Note that a custom sync filter can be applied to Account, Opportunity, and custom entities. Sync filters on multiple related objects can be tricky to manage. (I was not able to confirm whether the custom sync filter is available for objects other than Lead and Contact in the SFDC integration.)
- If using a Custom Sync Filter on Lead and Contact records, while the sync filter is set to FALSE associations (to an Account or Opportunity) or conversions (from Lead to Contact) will not be recorded in the Marketo Log. When the sync filter is set to TRUE for that record, the log will “catch up” on field updates, but associations and conversions may be impacted and not reflected in the sync.
- The usual methods to mitigate impacts of large data updates apply.
- Perform large data updates at times of low activity (nights, weekends).
- Break large data updates into batches.
- Adobe Support has seen anecdotal sync throughput rates of about 50K updates per hour. Assuming your sync is not backlogged or stressed, aim for batches of 50K records or fewer to throttle large data updates. (Previous documentation had indicated peak throughput rates of 10K updates per hour.)
- Employ a custom sync filter to control the sync of records.
- Avoid large data updates on the day of an Adobe Marketo Engage release. Wait a day or so until the release is complete.
.png)