I set up a new handwritten traffic monitor and was surprised at how many page views the message boards get until I noticed that the vast majority of them are bots (programs that crawl across the Internet indexing web pages for search engines). Since there are so many messages here and in the archives, and the bots generally repeat their crawl once a month -- I suspect that that is where the band width the bandwidth usage has been coming from. I want people to be able to find the site, but I don’t think the message in the message boards or archives are necessarily the places they find. So I set up a file which will block the bots from accessing the message board and archives. They can still index the main page and the other pages, but not the messages in the message boards or archives.

In a couple of days I’ll be able to tell how well this will reduce the bandwidth.