1 Reply Latest reply on Nov 28, 2017 8:49 AM by Sanford Whiteman

    Whitelisting Crawl Tools without removing robots.txt?

    Kelly Ayres

      Hi there,

       

      We use a tool called SiteImprove for our main site, and want to add our Marketo landing pages for improvements. But since our pages are non-crawlable to search engines, the tool doesn't recognize them as a verified group of pages.

       

      Has anyone been able to whitelist certain crawling tools for Marketo landing pages?