Robots.txt files are used to request search engines (or any web spidering program) not include parts of your website in their database.
You can tell engines to skip over a certian directory, file, or even the whole website, depending on what you put there. (And if you don't have one, or just put a blank one in, it'll just do anything in can find.) People can still can get into those pages you've filtered out, but they just won't turn up as results on Google.
Just wanted to make sure everyone was aware what may go down tomorrow. If you have getting microsoft updates or virus protection updates you should not be affected. http://www.cnn.com/2009/TECH/03/31/april.fools.computer.virus/index.html microsoft.