Preventing Search Crawlers on a Page-Level Basis

A document by KCS Integration on behalf of Chris Coel on Nov 27, 2017Last modified by Vishal Sharma on Jun 26, 2018
Version 3Show Document
  • View in full screen mode

Issue Description
Search engines index all pages, but you want to exclude some pages from the index and allow others.


Issue Resolution
To prevent most search engine web crawlers from indexing a page on your site, place the following meta tag into the <head> section of your page (copy the bolded print):

<meta name="robots" content="noindex">


To prevent only Google web crawlers from indexing a page:

<meta name="googlebot" content="noindex">


You can also add the following after the content="noindex to stop links from being crawled on a page:

, nofollow"

Example use:


Using "nofollow" means the crawler will not crawl the links on the page while the "noindex" will only block the page from being indexed.


Within Marketo this can be applied on a template basis and then on any pages using said template will not be indexed. You can also ask support to set your Images and Files Directory in Design Studio to not be indexed so documents such as whitepapers will not show up in search engine results.

Is this article helpful ?