I got a basic spider from the book I've been reading to compile and run against my site. In less than five minutes, it had downloaded 316 pages to my hard drive. I didn't even realize I *had* that many pages. Sheesh. The bot is currently downloading the files individually and storing them. However, with a little tweaking it will store them in a database. I'm pretty excited, because this means a day of running and I'll either a) have a pretty sweet data set to experiment with, or b) have a full harddrive.