Kurt_Koller
Level 4
Santa Monica, CA
Chief Software Architect
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
3 weeks ago
|
54
Posts
20
Likes
1
Solution
05-05-2020
03:42 PM
We are starting to use this solution due to the large amount of tracking protection showing up these days, Ghostery, the new Edge default settings etc. Any idea how often Marketo updates their code library? I maybe need to set an alert on this on the instances where we deploy. Thanks for the replies Sanford, really appreciated.
... View more
05-05-2020
03:40 PM
Ok, it was given to me and I guess they got it from the CodePen. Will switch to the newest version from your blog post and will make a note to do that in the future. Sorry about that.
... View more
05-05-2020
02:04 PM
@SanfordWhiteman One quick follow-up question if you don't mind. In the code that comes back from marketo (the default frame): if(message && message.mktoRequest && message.mktoRequest.ajaxParams){ var p = message.mktoRequest.ajaxParams; var params = { type:"POST", dataType:"json", url:p.url, data:p.data, crossDomain:false, jsonp:false }; params.success = function (data){ In the code you have posted to use in place of that frame: if(message && message.mktoRequest && message.mktoRequest.ajaxParams){ var params = message.mktoRequest.ajaxParams; params.success = function (data){ Do you think there is any possibility that by copying the whole original params object vs setting them individually like Marketo does that we could get different information in the referrer or querystring? I find this is a very difficult architecture to debug in and this isn't an area I normally stumble into. Thanks for entertaining my questions.
... View more
05-05-2020
01:17 PM
@SanfordWhiteman Okay I'll look into it again. It seemed that we were losing Referrer and Querystring/utms. Thanks for the reply.
... View more
05-05-2020
11:27 AM
Hi Sanford, This is great, thanks so much. When this approach is used, the Referrer is lost as well as the Querystring. Any ideas on preserving this data? I was going to work on this but thought I'd ask your thoughts first, since you really know this stuff inside an out. I'm thinking Querystring could be read from the parent frame or passed in to the child fame as a querystring on the html file, but the Referrer I think is not programmatically changeable so that seems like it would have to be passed down somehow and passed in as another field if we wanted it. Again, just looking for your thoughts on this. Thanks!
... View more
10-02-2018
02:48 PM
Release Notes: Fall '18 - Marketo Docs - Product Documentation In the release notes liked above, it has the following: API Lead Map REST API Marketo now supports the use of email-disposition fields over the Lead APIs (includes Black Listed, Marketing Suspended, Email Suspended, and Relative Urgency). What does this mean exactly? I think it might mean we will be able to get these fields when querying a lead. They don't show up in lead/describe.json (release date isn't until October 12 so that makes sense) so I'm not sure what they mean exactly. Any info on this would be appreciated Thanks Kurt Koller Digital Pi
... View more
- Tags:
- lead api
Labels:
06-15-2018
10:43 AM
Sanford, Thanks for this. We are definitely seeing things vanish, I assumed there would always be a delete lead or merge lead, I think that was a bad assumption. I'm going to verify today or Monday via the deleted lead endpoint, and then I guess I need to build a rotating/drip sync-check to determine if leads are still in the system. Let me know if you have any other info
... View more
06-13-2018
01:16 AM
Yes we have. That endpoint doesn't give merges, just delete lead (37) activities which we also get through bulk already. There are many leads which are no longer in the system which never had a delete lead (37) or merge lead (32) activity, so I'm looking for other ways a lead could be deleted from marketo and not throw a delete lead or merge lead activity. We need to solve this, after time goes by we aren't mirroring data any more.
... View more
06-12-2018
06:39 PM
1 Like
We are trying to keep Leads in our external database in sync with leads in the Marketo database. Nonwithstanding the inablility just get all leads in once call, instead having to go through in monthly gets using a filter like created or updated, once we have all the leads in our own database we figure we could get updates and keep them in sync. Leads are removed in a variety of ways. Activities have two main activities that we see (we are testing on instance with a single partition) when they are removed: 37, Delete Lead 32, Merge Leads We have been using Bulk Export API to get all activities for Leads. In this data we get activity data for both of these activities. The question is, are all leads which are Deleted included in these activities? I know you can only get them for a limited about of time, but will every deleted lead also return a Delete Lead activity or a Merge Leads activity? I ask because we are trying to keep leads in sync, and using these types and we are off by a large percentage in some instances, where the leads just have a smattering of activities and then are gone, with no 37 or 32 or anything else indicating that they've been deleted, but indeed they are deleted, so we never see that the lead is removed and our database is completely wrong very quickly.. I'd love to know all of the possible types of activities that could signal a Lead has been removed, if there are more than these two. If not, are there cases where the 37/32 wouldn't be pulled down via Bulk API? Or should every lead that has been deleted/merged show in this data? Alternatively we would have to check the lead API endpoint in batches etc on a constantly rotating basis to figure out if leads are still existing, which is both wasteful and uses many API calls and takes days. Thanks, Kurt
... View more
Labels:
03-23-2018
09:53 AM
1 Like
Well I'm going to maybe blow your mind some more here because those show up I believe as a lead merge activity. We then take the targetid of the merge and go back with non-bulk API for any leads that have been made known and get their activities. It's not great. Also speaking of merge, we also need to keep activities sorted and mapped to the proper leads for aggregate reporting, and we need to keep leads sorted, so we have to look at all delete lead and merge lead activities. If we get a delete lead, we mark the lead as deleted. If we get an n-way merge for leads and they were merged into another person, we then change the leadid for all old activities for merged leads, and mark the old leads as merged/deleted. This also means we need to regenerate summaries related to lead/activity counts since the activities can be remapped to different leads later in merges. There are a lot of gotchas if you're trying to keep data synced with marketo.
... View more
03-22-2018
02:25 PM
1 Like
I've asked about the whole asynchronous thing and I was told that that isn't an issue and that the only things would be people moving in and out of a partition and anonymous leads being converted to non anonymous. You won't see anonymous activity in bulk but if a lead is converted later to known then if you got the same bulk data later you would see their activity
... View more
02-16-2018
12:57 PM
Well, this was a transient error and I have no way of generating this on command that I can think of, so I can't tell you what the headers looked like. I just had a file that downloaded correctly, wouldn't import, and it had json in it. I can put in a check for the file length (wasn't in there because if the file downloaded correctly with no errors and a 200 status code I'd assume that was a success). Ideally a REST API would set the HTTP result code so you can logic accordingly (i.e. 5xx is a server side problem, you can try that one again later, 4xx is a client problem so you can log that and figure out what you (the client) did wrong, etc) before getting into results / file contents. They return a 4xx code when you try to get something without credentials, so this type of code would work for instance (we have something like it in production), however the last bit doesn't catch the error in question because they return a success HTTP code: ``` if (response.IsSuccessStatusCode) { // read file return true; } else { if (response.StatusCode == HttpStatusCode.Forbidden) throw new AuthenticationException("Invalid username/password combination."); else throw new Exception($"Bulk download error, API returned http status {response.StatusCode}"); } ``` I could check json and content types sure but this is the correct way for a REST API to report errors, and the Marketo API does return a forbidden when you haven't authenticated or your token has expired, so I think they should continue that through to the rest of the API error codes, particularly on this url where we aren't needing/wanting to parse the result at the time of download.
... View more
02-16-2018
12:34 PM
Currently, when you download a file on the Bulk API at and there is a server problem on the Marketo env, instead of the CSV file you get {"requestId":"3fd2#161992bc7e1","success":false,"errors":[{"code":"611","message":"System error"}]} It would be ideal if this call would return a 500 or 503 status code, since this is a file retrieval operation and not an API call like the others. We can check length of the download as a bandaid, but if we are getting leads and we get just a header row it could easily end up being the same length as the specified file download length. And it's a little awkward to check what's supposed to be a CSV export file for json. ALSO When you're out of quota you get an error 1029. http://developers.marketo.com/rest-api/bulk-extract/ " The daily quota is a maximum of 500MB per day, which is shared between leads and activities. When the quota is exceeded, you cannot Create or Enqueue another job until the daily quota resets at midnight Central Time . Until that time, an error “1029, Export daily quota exceeded” is returned." The error message in the table of errors http://developers.marketo.com/rest-api/error-codes/ says: 1029 Too many jobs in queue Subscriptions are allowed a maximum of 10 bulk extract jobs in the queue at any given time. Is this error doubled up on or is the documentation incorrect? Thanks, Kurt
... View more
Labels:
12-07-2017
02:17 PM
1 Like
When we extract activities from the bulk API, we don't see any of type 28, "Sync Lead Updates to SFDC." We do continue to get them through SOAP for the same dates. The expectation I have is that Bulk extract of activities would have all activities for a time range. Does anyone know if these activities being excluded from the Bulk extract API? Thanks!
... View more
11-13-2017
10:24 AM
2 Likes
Do you know if it's near-future or far-future? Also what are the types of subscriptions that don't have this feature? We want to have something in our documentation about how they know and what the ramifications are. We would also love an approximate date for our own planning, if that kind of thing is available. Even things like "first half 2018, maybe" is a good answer as is "it's not currently on our internal roadmap but we will be planning this out over the next few months so feel free to ask again in 3 months." Very many thank yous in advance Thank you David Everly
... View more
11-10-2017
10:54 PM
1 Like
Hi there David Everly, regarding: 4. You can determine if the instance supports updatedAt by calling Create Export Lead Job endpoint. If not supported, t ou will receive an error 1035, "Unsupported filter type for target subscription". We do this, it's fine, but do you know, will the servers/APIs that currently do not support the updatedAt filter be getting an update to support that in the future?
... View more
10-18-2017
11:46 AM
Hi Kenny Elkington, one last related question, will every lead record created have a valid updatedAt for the filter so we can use that filter to get all leads? Or is there a way for a lead to get created that doesn't have that updatedAt set? It seems like a timestamp on the base table so I think it'll be there for every lead record, just making sure.
... View more
10-05-2017
08:21 PM
1 Like
Kenny Elkington, thanks for that information. Based on your reply, we did some tests. We see that if a lead has its three most recent Data Value Changes on 2/12, 2/13 and 9/9 (an actual lead we tested) if we do three bulk requests and use an updatedAt filter for each of the days, the 9/9 export has the lead and the 2/12 and 2/13 exports do not. So if you're updating the documentation, and if that's how it works (and it seems to work that way) I'd say something like "The updatedAt filter for Bulk Export returns the associated lead records in which the last Data Value Change for that lead is in the range specified" since it doesn't return any lead that had a change during that time. Some people might also want to know which fields are in the primary table and therefor trigger that field to be updated, I personally am not currently one of those people. We've been storing the updatedAt field thinking it was something else, so now I know that that's pretty useless for our use case and therefore we won't be retrieving it/storing it any longer. So thank you for that! Thanks again, it's very helpful to know what's going on behind the scenes in cases like this.
... View more
10-05-2017
10:04 AM
1 Like
If I send a request to get Leads with a filter of updatedAt in a date range: {"fields":["id","email","company","createdAt","updatedAt","unsubscribed","emailInvalid","originalSourceType","leadSource"],"format":"CSV","filter":{"updatedAt":{"startAt":"2017- 09-01T00:00:00Z","endAt":"2017-09-15T00:00:00Z"}}} In the updatedAt column that's returned, I get dates that fall well outside the range, like 2017-02-14 or even other years. When I look at the lead activity in marketo, the date I get for the updatedAt is the same date as the "Updated" in the UI. Via the Bulk Lead Extract API, we are trying to update fields in our synced lead records only for leads that have had an update on the actual lead record. When we ask for updatedAt in the filter, it seems that it is based on the last activity that happens with that lead, not on the updatedAt field that we get in the data. It's also confusing that if we ask for an updatedAt filter range we get back records with updatedAt outside of this range. I guess these are both named the same but mean different things? I would expect that if I ask for a range of updatedAt I'd get updatedAt in that range. Is this working correctly? We're also not 100% positive on what the updatedAt date being returned is, see this post by Bill Lundy: What does the "Updated" field on a lead/person reflect? Is there any way to get information on just leads that have just had their lead record updated in some way with a filter, vs whatever it's doing now? Thanks
... View more
- Tags:
- bulk api
Labels:
08-10-2017
12:32 AM
Sanford, This is the last post I'll make on this, I'm not sure what your motive is for arguing, but to me this is pretty clear. The CSV export of the _same data_ from the UI is, in my opinion, correct. The reason given for including ,null, in my opinion, is not a good one, beccause CSV != JSON. There is indeed a precedent in Marketo, and that precedent for CSV is that ,null, is not used, and that ,, is used. I believe they made a mistake here, and the mistake is that they applied the JSON null representation into CSV when they had a perfectly reasonable CSV implementation for the same data already. Thanks for your counter opinion, but I personally believe it's incorrect, since the existing CSV export that they have represents it the way I'm expecting it here.
... View more
08-10-2017
12:11 AM
I won't belabor this, but CSV != SQL dump, and again, the Marketo UI CSV dump of the lead data from the UI (same data) does the (correct in my opinion) ,, behaviour and doesn't include the word null for missing data.
... View more
08-09-2017
10:45 PM
Sanford, The CSV export from the Marketo UI does exactly what I suggest.
... View more
08-09-2017
06:08 PM
I will add logic to our parsers to look for a null string and replace it with an actual null in our database. One thing that gets lost is that sometimes people end up with the text "null" in a field in Marketo because of import issues etc, and encoding nulls the way they are currently encoded in bulk export means we can't detect if it's a real null or a text "null"
... View more
08-09-2017
06:02 PM
OK. Here's my $0.02 on this. I'd say that CSV is a loose standard that existed well before JSON became a thing, and I've never seen a CSV implementation where null is returned. The JSON spec says you can have a string, numeric, or null for a value. The parsers can look for this easily, because a string is always quoted. CSV has never worked like this. CSV has unquoted values and quotes anything with a comma or a quote in it. i.e. sometext,123,"Last Try, The" If I try to parse a CSV I have no way to know if null is a string or a null. Sure, that's a rare occurrence but CSV parsers generally will treat null as a string because the loose standard has always had a missing value for a null. The worst parsers will return an empty string instead of a null in a typed language.
... View more
08-08-2017
06:14 PM
My other response didn't post for some reason. Thanks for #1 request, and thanks for #4. Appreciate all your answers.
... View more
08-08-2017
06:11 PM
David Everly thanks for the response. With #2, I just pulled an export of leads from the UI with the same fields, and none of them contain the word null in them. Sample line from the CSV for my account in our system, note the ,,, and not ,null,null,null: 1013663, kk@digitalpi.com ,Digital Pi,2016-04-26 15:13:15,2017-07-27 23:44:17,,,1,Omega,List import
... View more
08-08-2017
06:01 PM
Nish Vel I haven't seen that, I've seen missing marketoGUID in bulk export and I've raised a support ticket but don't have resolution yet. I've been away from the project a little while I'm waiting for resolution. I haven't seen duplicates but honestly I was stymied by the lack of a key and didn't notice one way or the other. I'll follow up if I see similar.
... View more
08-03-2017
10:51 AM
thanks, the accept header wasn't working for us at launch, I will try again. Appreciate all your responses over the last few days.
... View more
08-03-2017
10:50 AM
Thanks Dave! It would be great if we could manage the files in daily quota/delete them for more quota. There are some instances where it's 250M-500M a day in activities and getting any kind of history is a very long and arduous process. Thanks again
... View more
08-03-2017
10:46 AM
You could look at the Bulk Extract API for Leads. Once you have the initial leads pulled, you can pull just updates every night via the Bulk Extract. The initial pull isn't straightforward either using the regular API that you specify or the bulk extract, you'll have to do some kind of loop with either.
... View more
Public Statistics
| Date Registered | 08-09-2016 02:34 PM |
| Date Last Visited | 3 weeks ago |
| Total Messages Posted | 54 |
| Likes Received | 24 |
Likes from
| User | Count |
|---|---|
Anonymous
| 5 |
| 1 | |
| 3 | |
| 1 | |
| 2 |
.png)