Here (Marketo Replication to Data Warehouse ) you mention the option to export the db snapshot for customers.
How can we get hold of that snapshot for our account?
To give you some background: we are building a data pipeline and building models for varied business questions. Marketo is one of the many source systems. Going forward we have setup a service to work with the bulk api and extract ongoing events across leads and activities in a periodic manner. For backfilling historical data we need that one large export.
All help in this regard will be appreciated!
You can export your Marketo database by going to Lead Database>System Smart Lists>All Leads>Click the "Leads" tab then select the export button at the bottom.
Thanks for such a prompt response! The activities data export is the one I am more concerned about (due to its size). Anyway to self-serve that entire table/db?
So with bulk Activity Extract, that's 76 CSVs (19 per year), half of which will be very tiny (the leftover days in a 31-day month). I don't know about your volume per month but this could take less than a workday to run.
I think we are not talking about the same thing here. When I extract Activity Data for ALL Leads in our database using bulk export job creation api: marketo_base_url/bulk/v1/activities/export/create.json I get files via export endpoint; atleast 100MB in size for only one day of activity. The API restricts me to only a 500 MB download per day. And hence asked for a master snapshot of the historical data. Am I assuming something wrong about bulk exports here?
OK, if you're generating an unfiltered dump with that much volume you'd need an offline export. I'm not sure if this is still available without a Professional Services engagement.
Best way is an export via a API. If you have a decent API limit it shouldn't take too long - a few weeks. We bring everything down nightly into warehouses for analysis. This includes Leads and all Lead Activity for the last 4 years.
On a regular basis, sure. For four years of this much data I'd incline toward an offline export because you're basically going to paralyze your all your other API-consuming apps for longer than I'd be comfortable with. (10K calls per day is still the official standard.)