Googlebots Robots.txt Parser Update and Your Marketo Landing Pages

[#1]

Issue

Issue Description
Google is changing production of their robots.txt parser to an open source format and removing support for some widely used but unsupported  code:

In the interest of maintaining a healthy ecosystem and preparing for potential future open source releases, we're retiring all code that handles unsupported and unpublished rules (such as noindex) on September 1, 2019. For those of you who relied on the noindex indexing directive in the robots.txt file, which controls crawling, there are a number of alternative options:

-Link to Blog Post from Google: https://webmasters.googleblog.com/2019/07/a-note-on-unsupported-rules-in-robotstxt.html?m=1

Solution

Issue Resolution
This should not affect any Marketo Landing pages if the Noindex, NoFollow code has been put in the Robots meta tags as per the documentation: https://developers.marketo.com/blog/block-crawling-and-search-indexing-of-a-marketo-landing-page/

This will only affect your landing pages if NoIndex, NoFollow are in the robots.txt file and not in the header meta tags.