Bulk Lead Extract API - updatedAt filter vs updatedAt lead field

[Kurt_Koller]

If I send a request to get Leads with a filter of updatedAt in a date range:

{"fields":["id","email","company","createdAt","updatedAt","unsubscribed","emailInvalid","originalSourceType","leadSource"],"format":"CSV","filter":{"updatedAt":{"startAt":"2017-

09-01T00:00:00Z","endAt":"2017-09-15T00:00:00Z"}}}

In the updatedAt column that's returned, I get dates that fall well outside the range, like 2017-02-14 or even other years. When I look at the lead activity in marketo, the date I get for the updatedAt is the same date as the "Updated" in the UI.

Via the Bulk Lead Extract API, we are trying to update fields in our synced lead records only for leads that have had an update on the actual lead record. When we ask for updatedAt in the filter, it seems that it is based on the last activity that happens with that lead, not on the updatedAt field that we get in the data. It's also confusing that if we ask for an updatedAt filter range we get back records with updatedAt outside of this range. I guess these are both named the same but mean different things? I would expect that if I ask for a range of updatedAt I'd get updatedAt in that range.

Is this working correctly?

We're also not 100% positive on what the updatedAt date being returned is, see this post by Bill Lundy: What does the "Updated" field on a lead/person reflect? 

Is there any way to get information on just leads that have just had their lead record updated in some way with a filter, vs whatever it's doing now?

Thanks

[Kenny_Elkington · Accepted solution]

Hey Kurt,

Your confusion is justified, and we haven't done a great job of communicating what the updatedAt field actually reflects, versus what our APIs use to determine if a record has been changed within a date range.  The updatedAt field only reflects changes made to the specific primary Lead/Contact table, but not all of the fields on lead records reside there, including some system managed fields, and all Custom Lead fields.  The updatedAt filter for Bulk Export instead looks for Data Value Change activities within that range, and returns the associated lead records which have changes in the range.

I'll make sure to update the docs to reflect this.

[Kurt_Koller]

Kenny Elkington, thanks for that information.

Based on your reply, we did some tests. We see that if a lead has its three most recent Data Value Changes on 2/12, 2/13 and 9/9 (an actual lead we tested) if we do three bulk requests and use an updatedAt filter for each of the days, the 9/9 export has the lead and the 2/12 and 2/13 exports do not.  So if you're updating the documentation, and if that's how it works (and it seems to work that way) I'd say something like "The updatedAt filter for Bulk Export returns the associated lead records in which the last Data Value Change for that lead is in the range specified" since it doesn't return any lead that had a change during that time.

Some people might also want to know which fields are in the primary table and therefor trigger that field to be updated, I personally am not currently one of those people. We've been storing the updatedAt field thinking it was something else, so now I know that that's pretty useless for our use case and therefore we won't be retrieving it/storing it any longer. So thank you for that!

Thanks again, it's very helpful to know what's going on behind the scenes in cases like this.

[Kurt_Koller]

Hi Kenny Elkington, one last related question, will every lead record created have a valid updatedAt for the filter so we can use that filter to get all leads? Or is there a way for a lead to get created that doesn't have that updatedAt set? It seems like a timestamp on the base table so I think it'll be there for every lead record, just making sure.

[Kenny_Elkington]

Yes, it should be valid for all leads.

[fivetrandev]

Thank you for your answer, Kenny. I recently checked the CDV activity returned through the activities endpoint after modifying a standard field and once after modifying a custom field and I received CDV activity for both. So the question is, shouldn't the updatedAt field in lead be the exact value the `updatedAt` filter will be working on even though it doesn't directly use updatedAt field? And which fields don't updatedAt to change?