Page 1 of 1

Robots index Demo page

Posted: Tue Nov 01, 2011 12:10 am
by tonylees
Hi,

How can I stop the search engines indexing the demo page... this is happening on sites where I have purchased a license.

Cheers
Tony

Re: Robots index Demo page

Posted: Tue Nov 01, 2011 7:23 pm
by marcus
You can disable indexing of arbitrary pages by placing a plain text file named robots.txt in your root folder.

A very simple robots.txt which prohibits indexing of anything inside /webyep-system could look like this:

Code: Select all

User-agent: *
Disallow: /webyep-system

Please note, that already indexed pages will remain in the search index for several days.
To prevent this in the future you have to add the robots.txt before you install WebYep.

For detailed information please visit www.robotstxt.org or ask the search engine of your choice.

Re: Robots index Demo page

Posted: Tue Nov 01, 2011 9:07 pm
by tonylees
Thanks Marcus it is appreciated