amazing..thank you!
Learned something new once again - thanks Sandy!
Wow, of course.
How should the robots.txt be formatted to Disallow just pdf file types? Would this article help?
apparently if you set the PDF headers correctly, you can make the PDF itself disallow crawling.
How should the robots.txt be formatted to Disallow just pdf file types? Would this article help?
I'd go with
User-agent: *
Disallow: /rs/
apparently if you set the PDF headers correctly, you can make the PDF itself disallow crawling.
Yep, that would be cool, but we don't have access at that level.