Hello Friends!!!

How to disallow a particular directory from being crawled by google? [Important| Updated May 2019]

Use this simple code to disallow a particular directory from being crawled by Google [Latest 2019]

• Login to your cPanel
• Go inside File Manager
• Click on public_html
• Create a robots.txt file

Use this code:

User-agent: Googlebot

Disallow: /protected-directory/

This code will disallow the protected directory from being crawled by Google.

Just for an example i have searched https://facebook.com/robots.txt and found all the directories which has been disallowed to be crawled by Google.

Hope this helps you!!!

Thanks for reading!!!