API Reporting Discrepancies against Smart Lists?

[Nav_Singh]

I seem to be hitting a roadblock with trying to scale our Marketo instance. We tried to automate some of our analytics through REST apis (in lieu of smart lists) and have noticed some fascinating discrepancies that make the APIs seem less reliable.

For context-
Due to the volume of records we have and ongoing activities it's imperative that we implement our processes for scale. I'm concerned because from an analytics perspective I don't think it was clear that we would not be able to use APIs to augment or supplement our reporting without seeing stark discrepancies. I was curious if anyone else has found a way to address this or understand why we would need to purchase something like RTP to get accurate data from the REST api on web page visits.

I was comparing data pulled via API for the visited web page activity vs. a smartlist for visited web page data from Marketo. I had compared this smart list for 'visited web page' on Dec. 5th and against api results in that same range and wanted to understand why the numbers do not match up. Has anyone else run into this issue or come across a solution?

We used GET /rest/v1/activities.json with the following parameters: activityTypeIds=['1', '3', '7', '8', '10', '11'],

sinceDatetime='2016-12-05', untilDatetime='2016-12-06',

    #              batchSize=None, listId=None, leadIds=None

Appreciate any advice or inputs.

Thanks,
NS

[Nav_Singh]

Sanford Whiteman, was curious if you had any experience with discrepancies between API results and smart list results.

[SanfordWhiteman]

Broadly speaking, yes, there are various ways the two can diverge. Both Smart Lists and the Activity Log are "living" records but to different degrees (that is, it isn't as simple as the ActLog being totally static/append-only while the SL is live).

But I'd need to see the actual query results, and look at the disjunction(s) between the two, to be able to diagnose.

[Wyatt_Bales4]

Hi Nav,

Note if you're using the Web Page Activity report, it won't be using the same constraints as your API get. In general, API calls are much more specific and will return rows of activity that you then have to tie back to the individual lead vs a smart list brings back a list of leads meeting your criteria.

Also know that internally, Marketo can evaluate anonymous views which can skew the results dependent on which table the smart list is querying. And lastly, a lot of smart lists use BlackCat to index data which isn't the same freshness as what's available through the REST API.

All not great answers but constantly using the REST API to parse through the activity is going to get your the best results even over the smart lists.

[Nav_Singh]

Hey Wyatt,

Thanks for the response. A little confused. So, what I was finding actually was that the REST API was less comprehensive when it came to reporting on the visits web page activity type. I compared this against a smart list which logged any web page visits on day "x"  to a REST api call to get all activity (type 3 or 10 I think, can't recall the number off hand)  and the Reporting API data was missing about 300 results when compared to the data I exported from a smart list.  That's why I'm a lot more concerned since the REST API is certainly more scalable but it does seem to be missing a significant chunk of data.

Nav

[SanfordWhiteman]

Yes, I would not in any way say that the REST API is more comprehensive. In fact it is distinctly not so.  Scalable?  I don't know if I'd use that term, either.  But extracting using REST will give you a gateway to certain kinds of BI slicin'-and-dicin' you can't do any other way.