The new forums will be named Coin Return (based on the most recent vote)! You can check on the status and timeline of the transition to the new forums here.
The Guiding Principles and New Rules document is now in effect.
exclude a PDF from search engine crawl LOCK
The_Glad_HatterOne Sly FoxUnderneath a Groovy HatRegistered Userregular
* User-agent: the robot the following rule applies to
* Disallow: the URL you want to block
These two lines are considered a single entry in the file. You can include as many entries as you want. You can include multiple Disallow lines and multiple user-agents in one entry.
Each section in the robots.txt file is separate and does not build upon previous sections. For example:
User-agent: *
Disallow: /folder1/
User-Agent: Googlebot
Disallow: /folder2/
In this example only the URLs matching /folder2/ would be disallowed for Googlebot.
Are you able to upload a file called "robots.txt" to the root directory of your website? i.e. so that yourdomain.com/robots.txt is accessible by search engines?
This file can be used to tell well behaving search engines to ignore specific files.
Posts
http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
Edit: Quoted in case it's not evident:
* User-agent: the robot the following rule applies to
* Disallow: the URL you want to block
These two lines are considered a single entry in the file. You can include as many entries as you want. You can include multiple Disallow lines and multiple user-agents in one entry.
Each section in the robots.txt file is separate and does not build upon previous sections. For example:
User-agent: *
Disallow: /folder1/
User-Agent: Googlebot
Disallow: /folder2/
In this example only the URLs matching /folder2/ would be disallowed for Googlebot.
This file can be used to tell well behaving search engines to ignore specific files.
See example in Wikipedia.
Edit: Outplayed sir! Well played zeeny, well played. =P
Play with me on Steam
Thanks guys!
put a fork in it, this thread's done!
and by fork, i mean lock.
/lamejoke