Google robots.txt Tester

[Pete_Jones]

We have several pages on our website that use Marketo forms. Google's robot.txt testers are not seeing the Marketo form data and have treated the blocked resource to impact indexing of the pages. They say all we need to do is verify ownership of the blocked resource, our Marketo forms. I'm curious if there are any best practices for uploading the HTML file to our Marketo instance and displaying it for Google to see. The domain where we display our files loaded to Marketo and the Marketo domain we use to link to the forms are two separate domains so Google cannot verify them.

[SanfordWhiteman]

I'm curious if there are any best practices for uploading the HTML file to our Marketo instance and displaying it for Google to see.

That question doesn't make sense.

You can manage your Marketo LP domain's robots.txt like so: https://blog.teknkl.com/customizing-the-robots-txt-for-marketo-lps/

[Pete_Jones]

I am not interested in managing the Marketo LP domains. I am interested in providing verification of ownership of the Marketo forms I have embedded on my website. Google is not seeing the forms and as a result, it is blocking the robot.txt file from completing the crawl. One way to verify ownership in Google Search Console is to upload the HTML file to Marketo and display so Google can see it and complete the crawl. I am curious if that has been done and if there are best practices available.

[SanfordWhiteman]

You create an LP with the verification HTML as the content.

There's no other practice for creating a page at the root of your Marketo LP domain.

[Pete_Jones]

Thanks, but Google is seeing the Marketo form app and our domain we use for our LP differently. So, to verify the HTML we need to domain structure of the uploaded HTML file in Marketo to mimic that of the Marketo app domain. I'll just have to pursue another means of verification.

[SanfordWhiteman]

You're not providing a lot of detail (not sure why).

Are you loading your forms, and the forms2.min.js library, from app-nnnn.marketo.com or from your Marketo LP domain?  You should be using the latter.

[Pete_Jones]

Well, that may be the issue. Our Marketo forms use app-xxxx.marketo.com as the domain in the embedded form on our website. Should that be the same domain that we use for our LP's?

[SanfordWhiteman]

You should load from one of your Marketo LP domains, yes.

Note if your website runs over SSL then your Marketo LP domain needs to support SSL as well (that's why this setup is not the default in the embed code).

[Pete_Jones]

Thank you Sanford. We're working with Marketo to secure our ssl cert. I'll focus on this once we've confirmed that has been completed.

[Pete_Jones]

Sanford. Okay, we have the SSL cert added to the Marketo domain now but still seeing this issue. For example, the domain below. When Google indexes the page with their robots they do not show the form and actally, the form is Blocked in the Google Search Console. Thoughts?

Parachurch Organization Program | GuideOne Insurance