Robots index Demo page

General discussions about WebYep
Post Reply
tonylees
Rank 2
Rank 2
Posts: 48
Joined: Sat Feb 07, 2009 11:40 pm

Robots index Demo page

Post by tonylees » Tue Nov 01, 2011 12:10 am

Hi,

How can I stop the search engines indexing the demo page... this is happening on sites where I have purchased a license.

Cheers
Tony

marcus
Objective Development
Objective Development
Posts: 170
Joined: Tue Nov 30, 2010 1:25 pm

Re: Robots index Demo page

Post by marcus » Tue Nov 01, 2011 7:23 pm

You can disable indexing of arbitrary pages by placing a plain text file named robots.txt in your root folder.

A very simple robots.txt which prohibits indexing of anything inside /webyep-system could look like this:

Code: Select all

User-agent: *
Disallow: /webyep-system

Please note, that already indexed pages will remain in the search index for several days.
To prevent this in the future you have to add the robots.txt before you install WebYep.

For detailed information please visit www.robotstxt.org or ask the search engine of your choice.

tonylees
Rank 2
Rank 2
Posts: 48
Joined: Sat Feb 07, 2009 11:40 pm

Re: Robots index Demo page

Post by tonylees » Tue Nov 01, 2011 9:07 pm

Thanks Marcus it is appreciated

Post Reply