Setting up Twilio and Marketo webhook

[Social_Garden6]

Hi there 

Sorry I am having issues setting up Twilio and the Marketo webhook.

Should the URL of the webhook be...

• https://[ACCOUNT_SID]:[AUTH_TOKEN]@api.twilio.com/2010-04-01/Accounts/[ACCOUNT_SID] /SMS/Messages.xml
  
  OR
  
  
• https://[ACCOUNT_SID]:[AUTH_TOKEN]@api.twilio.com/2010-04-01/Accounts/[ACCOUNT_SID] /Messages.json

Should the "Response format" be...

• XML 
  OR 
• JSON

I am using details from the post that was written in 2015 see below, (see link below) although its is not working.  
The From number starts with my country number for Australia it is +61.

Sorry can you pls advise what I have done incorrectly in the webhook? I cant get it to send an SMS.

I have included a screen grab of the information I have placed into the webhook, if anyone can shed some advice that would be much appreciated.

Thank you so much  

/blogs/marketowhisperer/2015/07/08/sending-an-sms-using-twilio 

fyi Sanford Whiteman‌, Josh Perry Karin Edmondson‌ @johnm

Latest test

[Tyron_Pretorius]

Just to provide people with another vendor option for SMS I recently published a blog on the Marketo forum that shows how to use webhooks to integrate with the Telnyx SMS API: 

https://theworkflowpro.com/marketo-text-messaging/ 

@Victor_Herrero I saw your later comments about wanting to do number lookup, response mapping, and use tokens to populate the SMS message.

• This blog post shows how to do a number lookup (for free!) with webhooks and map the response variables to Marketo fields: https://theworkflowpro.com/marketo-data-enrichment/  
• This blog post shows how to use the {{campaign.description}} token in the SMS webhook so that you can change the SMS message from smart campaign to smart campaign and still use a single webhook: https://theworkflowpro.com/marketo-sms-integration/

I hope these posts are helpful. If there are any questions let me know.

Full disclosure I am a Telnyx employee. I have been using Marketo for close to 2 years now and got tired of seeing all these Twilio posts so I wanted to balance the scales and let people know that there are other options out there.

[Social_Garden6]

Hi there everyone 

Thanks so much for all your feedback its really appreciated.

Special thanks to Sanford Whiteman‌ and Josh Mateer‌   Their answers were correct.

Just incase anyone else gets stuck I've posted the webhook that worked for me.

Thanks  

[Victor_Herrero]

Hi @Social_Garden6 

So in your experience the phone format should be with country code but no '+' (i.e. +44 --> 0044) or with no country code at all? 

I seem to e having a strange problem, where twilio will add '+1' to my 'To' phone number, regardless of which country it belongs to or if it has country code already. 

[SanfordWhiteman]

It does need to have a + when sent to Twilio, but that + doesn't need to exist in the field itself if it's hard-coded in the webhook definition (as URL-encoded %2B).

[Victor_Herrero]

Right! 

I didn't think about what '%2B' actually translated to. I guess the posts and documents I have been checking are centered around the US. 

This is how I am thinking of setting it up (and we may end up doing it from Salesforce anyway, with more serious development so we can see message history and merge it with an sms functionality we already have on the lead record page): 

(SMS as parent folder, AP and HF represent our two types of clients)

The webhook:

https://{{my.twilioSid}}:{{my.twilioToken}}@api.twilio.com/2010-04-01/Accounts/{{my.twilioSid}}/Messages.json

From=%2B1{{my.twilioFromNumber}}&To=%2B1{{lead.Phone Number:default=edit me}}&Body={{my.twilioMessageBody}}

I am using folder and program tokens. The parent folder includes the ID and token for the twilio account, I intended the country folders to include the 'From' numbers and the programs to include the message body token.

I was hoping to just create one webhook by doing this, instead of one per 'From' country or one per message (I really hope not :S )

Now I see that my template/payload is hard-coded to the US and I need to work on that. 

The next problem I see is that our phone numbers aren't always in the right format. What would you suggest we do for this? 

Of course validation is the right answer, but if that were not an option I was thinking of a script token that transforms numbers into the right format before the call is executed - I think you said script tokens don't work outside of the email context in some other post. 

In that case perhaps another webhook that has been previously called and normalizes our phone numbers from the start or maybe even a Salesforce solution that kicks in on sync? 

Do you see any other problems with a setup like this? 

I have also been trying to map the response from twilio to a field in the template, but I haven't managed to access the right element in the response as explained here. In any case, I have been able to have a look at the logs in twilio and I think the first element in the response I would be interested in would most of the time always say 'queued', which is not what I want to see. I want so see 'sent' or an error message. 

Even if I got that right, we couldn't see the history, only the most recent log... I don't know, this all feels like an MVP at best. 

[SanfordWhiteman]

---

Of course validation is the right answer, but if that were not an option I was thinking of a script token that transforms numbers into the right format before the call is executed - I think you said script tokens don't work outside of the email context in some other post. 

---

Yeah, you can't use Velocity here. You can use Twilio's own Lookup API as a a webhook, though: https://nation.marketo.com/t5/Product-Blogs/Calling-the-Twilio-Lookup-API-as-a-Marketo-webhook/ba-p/...

---

I have also been trying to map the response from twilio to a field in the template, but I haven't managed to access the right element in the response as explained here. In any case, I have been able to have a look at the logs in twilio and I think the first element in the response I would be interested in would most of the time always say 'queued', which is not what I want to see. I want so see 'sent' or an error message. 

---

Well, the SMS isn't sent at the time the webhook finishes technically, that's why it says "queued."  You can however use the Webhook is Called trigger to look for "queued" in the response and then set (Change Date Value) the field to "sent:{{system.datetime}}" instead. (It's actually a great idea to use Webhook is Called for such things because just resetting the field to "queued" over and over again won't register in the ActLog.)

[Victor_Herrero]

Hi @SanfordWhiteman , 

Thanks for your reply. 

I did eventually see their lookup product, of course paid 🙂 . But you do say it is reasoably priced... so we will keep it in mind. I bookmarked your article about setting it up as webhook.

I also found something on github a long time ago that does exactly that: input a phone number, outputs the number in different formats and other metadata like landline or not... not sure if carrier though. Probably unsafe to use directly and probably would get out of date without maintenance... ?

Eventually I also figured out how to map response attributes, only I'm mapping single ones... is there a way to map the whole response to a field? 

I tried things like 

to try to emulate what is explained here. But the response from Twilio doesn't follow that structure... 

{
    "sid": "SM####################################",
    "date_created": "Tue, 16 Jun 2020 12:56:47 +0000",
    "date_updated": "Tue, 16 Jun 2020 12:56:47 +0000",
    "date_sent": null,
    "account_sid": "AC################################",
    "to": "+41#########",
    "from": "+41#########",
    "messaging_service_sid": null,
    "body": "subscription settings",
    "status": "queued",
    "num_segments": "1",
    "num_media": "0",
    "direction": "outbound-api",
    "api_version": "2010-04-01",
    "price": null,
    "price_unit": "USD",
    "error_code": null,
    "error_message": null,
    "uri": "/2010-04-01/Accounts/AC##############################/Messages/SM###################################.json",
    "subresource_uris": {
        "media": "/2010-04-01/Accounts/AC#########################/Messages/SM###############################/Media.json"
    }
}

Also thanks for the webhook was called tip i was using that to create a history of 'sent' and 'failed' events. 

The change data value token looks like this: 

[{{system.dateTime}}] Delivery failed | {{lead.SMS History (Mkto):default=edit me}}

Which results in a story field with concatenated entries like these:

I'm wondering what happens if you keep doing this and reach the field's character limit... 

The hardest part is the 'delivered' status. For that it seems a whole web service is needed (you mention this in one of your posts), which doesn't seem to be too complex, but definitely too complex for me at the moment. 

[SanfordWhiteman]

is there a way to map the whole response to a field? 

Only if you use a webhook aggregator.  A couple of my recent blog posts discuss that approach.

---

The change data value token looks like this: 

[{{system.dateTime}}] Delivery failed | {{lead.SMS History (Mkto):default=edit me}}

Which results in a story field with concatenated entries like these:

I'm wondering what happens if you keep doing this and reach the field's character limit... 

---

It just wouldn't update anymore (the text would be truncated).

[Josh_Mateer2]

Our twilio webhook URL is set up as

• https://[ACCOUNT_SID]:[AUTH_TOKEN]@api.twilio.com/2010-04-01/Accounts/[ACCOUNT_SID] /Messages.json
• With the response format in JSON

. The only thing different in ours is the payload. We added %2B1 in front of the from and to (From=%2B118008008000&To=%2B1{{lead.Phone Number}}) and have gotten succesfull responses back. Not sure if that will help you to troubleshoot!

[SanfordWhiteman]

Yeah, JSON is the default response format even if the XML endpoint still works. (For the clear reason that not all client libraries have a native XML parser, NodeJS, for example, doesn't so it would mean more dependencies.)

Prepending the encoded plus sign as you have here is OK if you know the phone number doesn't already have it. Depends on how much normalization you're doing ahead of time, I recommend storing in E.164 in your database (in which case the + is already there).

[Joke_Van_Essche]

I've checked our configuration and we use: https://[ACCOUNT_SID]:[AUTH_TOKEN]@api.twilio.com/2010-04-01/Accounts/[ACCOUNT_SID]/Messages.xml (without the SMS in between) and response format XML and it works. 

And yes, the number you showed is the correct format. 

Have you tried testing with just a message in the body instead of the token? 

[Social_Garden6]

Hi there I have updated  many thanks Sanford   

[SanfordWhiteman · Accepted solution]

https://{SID}:{TOKEN}@api.twilio.com/2010-04-01/Accounts/{SID}/Messages.json‍

Response type is JSON, as you already have set up.

[SanfordWhiteman]

Please remove the attachment (I can't see them, they don't show up for all Community users) and paste the image right in the thread, thanks.