apache apachecon app apple asf atom atomprotocol atompub barcamprdu blogapps blogging businessblogging conferences family feeds foss general glassfish google humor ibm java javaone links linux mac microsoft movies music netbeans opensocial opensource photos politics rest roller rome rss socialnetworking socialsite socialsoftware sun triangle trianglebloggers vacation webdev webservices wiki


Main | Next day (Feb 1, 2003) »

Friday Jan 31, 2003

CNET: Blogs open doors for developers.

Comments[0] by Dave in Blogging

Excellent article with quotes from software pioneers and bloggers Dan Bricklin and Mitch Kapor.

CNET: "Web logs (commonly known as "blogs"), message boards and other online forums are becoming increasingly important vehicles for developers to attract customers -- and development talent -- well before an application even enters the beta stage."

Moved to another Roller.

Comments[0] by Dave in Roller

It's nice that there are multiple competing Roller sites now.

Carlos Villela: I had to move this blog to another Roller. Sorry guys and gals, but freeroller.net is just too slow, and I don't know if it's a problem in their bandwidth, mine, or if they're just too loaded.

Scooter.

Comments[0] by Dave in General

You probably did not notice this, but my site was getting a tremendous number of hits this week. The reason is Scooter. Scooter is the AltaVista search robot. Scooter is persistent but not very smart. Scooter is not content to index a page, follow the links, and then index the linked pages. Scooter tries to guess the URLs to pages on your site and does a terrible job at the guessing game. If Scooter gets thousands of 404 PAGE NOT FOUND errors in one day, Scooter does not care. He just keep on guessing. According to Seach Engine World, he may be here for several months:

Search Engine World The Altavista search engine starts by spidering your entire site with its spider Scooter. Although lately Scooter hasn't been scooting too well. Scooter may take up to three months to spider and index your entire site (if it is going to crawl your site at all). It normally spiders about 2-10 pages per site in any week. Sometimes Scooter needs a good swift kick to get it to index certain pages.

Yep, I would like to give Scooter a good swift kick. Scooter obeys a robots.txt, if you have one. I'm hoping that the following addition to my robots.txt file will make him go away.

User-agent: Scooter/3.3
Disallow: /

User-agent: Scooter/3.2.SF0
Disallow: /

This work is licensed under a Creative Commons License.
Copyright 2002-2007, David M Johnson (dave.johnson at rollerweblogger.org)

This is a personal weblog, I do not speak for my employer.