The W
Views: 100266071
Main | FAQ | Search: Y! / G | Calendar | Color chart | Log in for more!
31.10.14 0556
The W - Internet & Computers - Robots.txt? Register and log in to post!
Thread rated: 6.76
Pages: 1
(763 newer) Next thread | Previous thread
User
Post (2 total)
Pool-Boy
Lap cheong








Since: 1.8.02
From: Huntington Beach, CA

Since last post: 1323 days
Last activity: 89 days
AIM:  
ICQ:  
Y!:
#1 Posted on | Instant Rating: 1.00
I have noticed when I run statistics on my site that this filename gets a huge number hits - and since it doesn't exist on my site, it only generates an error message.

I understand it is supposed to have something to do with search engines - what exactly is is supposed to do, and what is it supposed to have in it?

Thanks...



Still on the Shelf.com

Updated Weekly
Promote this thread!
thecubsfan
Scrapple
Moderator








Since: 10.12.01
From: Aurora, IL

Since last post: 10 hours
Last activity: 7 hours
#2 Posted on | Instant Rating: 10.00
Robots.txt files are used to request search engines (or any web spidering program) not include parts of your website in their database.

You can tell engines to skip over a certian directory, file, or even the whole website, depending on what you put there. (And if you don't have one, or just put a blank one in, it'll just do anything in can find.) People can still can get into those pages you've filtered out, but they just won't turn up as results on Google.

For more information, visit your local internet.



thecubsfan.com - CMLLBlog
Thread rated: 6.76
Pages: 1
Thread ahead: Windows Media Player possible bug
Next thread: AIM cloning for Macs
Previous thread: CD Burning Question
(763 newer) Next thread | Previous thread
Uh....copy them. A USB thumb drive is probably the easiest way. If you're still worried about DRM, you should be able to log into iTunes on your new computer and add it to your list of authorized devices.
- CRZ, i-Tunes help (2011)
The W - Internet & Computers - Robots.txt?Register and log in to post!

The W™ message board

ZimBoard
©2001-2014 Brothers Zim

This old hunk of junk rendered your page in 0.06 seconds.