Syncing Marketo Data to Snowflake via Fivetran

[Fmuir1]

Hi!

We recently purchased FIVETRAN to sync our Marketo Data to Snowflake and looking to get insights from the community on:

1) Best practices on data load (in particular historical data loads)

2) How did you manage Bulk API MB Limits and API Limits

Thanks in advance!!!

[DGunnEtumos · Accepted solution]

Hi @Fmuir1,

This is absolutely my favorite way to get data out of Marketo! 
I'm sure by now you are aware of how easy the connection is so I will make sure to just stick to the questions you have answered. 

Best Practices:
- Since the first two weeks of a new connection are free in FiveTran, you will want to get as much information into your DB as possible at the start. In order to do this you can (and should) ask Marketo support to increase the API limit of your instance for a couple of weeks while everything gets pushed over. This will allow you to get more data within the two week timeframe without having to worry about your limits.

- Focus only on what you need. After the first two weeks if you are not careful things can get expensive. So you will want to limit the data coming into FT by really only selecting the fields & objects that you need for reporting. You can always add something if it is missing later down the road! Are you doing email reporting? Then focus on the logs ONLY associated to email activities. Do you care about someone's birthday? Maybe don't include it because you're probably not reporting on birthdays anyway. One thing you can do is limit the person table to literally just the ID and greatly reduce the costs in FT.

Managing Limits:
- So the best thing i can do here is basically state what i already have. Only take out what you know that you need, add something later if you are missing it. 
-  Use the API call limits in FT. Within the connector in FiveTran there is a daily call limit that you can set. I would highly recommend looking at all of the api calls you have coming into marketo and then base the FT limit off of that. Usually when I'm setting it up I will leave about 10k calls worth of wiggle room within Marketo just incase something weird happens. You could probably do with even less.
- Reduce the sync amount in FT. More than likely once every 24 hours is totally acceptable for a sync if you are doing reporting from your DB. Marketing teams don't need to know something down to the minute, or even hour.

Hope this helps! Happy to chat more if you would like!

-Derek