Automatically pull data into a data lake?

Kevin_Wick
Level 1

Automatically pull data into a data lake?

We have a data lake in Azure and are trying to pull data from the Leads and Activity tables in Marketo into this data lake.  We are able to pull the data using calls from Postman collection. However, we have yet to find a good way to automatically pull the data into a SQL database. In reading online, the best option should be to use Logic Apps in Azure since Data Factory does not allow the proper token credentials. I have also looked into the API Management tool. Has anyone found an effective way to automatically pull data into a data lake? What tools did you use in the Azure Portal? What steps did you take? Are you able to provide a link to documentation or a video that would help?

1 REPLY 1
Casey_Grimes
Level 10

Re: Automatically pull data into a data lake?

So, what you're describing is essentially an ETL tool's responsibility, and there are plenty that work with both Marketo and Azure SQL DWH/Azure Data Lake; both on-prem (Boomi, Alteryx, et al.) and SaaS (Fivetran, Stitch, Alooma, et al.) ETLs can handle this sort of work on a scheduled, automatic basis.