In order to confidently sync our proprietary data with custom objects (organizaion info) we need an automated way to either:
- Truncate all custom objects
- Calculate the objects that need to be deleted by taking the difference of all current custom objects with a given batch.
Marketo's api does not currently allow users to get a complete list of current objects, without specifying some filter criteria making the latter option not feasible
I don't see an API to truncate a given custom which makes the first option no possible.
What is the suggested approach to sync custom data to marketo to ensure that after a given batch there is no outdated data?
Short Answer:
TL;DR
It's a best practice to use caching in all Marketo REST API integrations. This and other best practices are discussed here: https://developers.marketo.com/rest-api/marketo-integration-best-practices/. For custom objects to be removed using the API, it's more than best practice, it's practically a requirement.
You should keep a record of each custom object created by your integration with at least all dedupe fields and the Marketo GUID. By comparing the list of new and updated custom objects against the list of those existing in Marketo, you can determine which ones should be removed/deleted. The remaining custom objects can be both created and updated using the Bulk Custom Object Import methods which are discussed here: https://developers.marketo.com/rest-api/bulk-import/bulk-custom-object-import/
While importing all currently valid custom objects over to Marketo each time will work, it's not as efficient as it could be. It's best practice to exclude existing custom objects that haven't been updated. That implies caching the entire record or a hash of it (https://devopedia.org/fingerprinting-algorithms) for comparison. Alternately storing a modification timestamp which can be compared to the last batch's timestamp could also work though timestamp hysteresis is a potential problem, so a margin of error may also be needed with this method.