WmW Access Restored
I’m happy to report I can once again access WebmasterWorld. Brett announced today in his new blog that I was one of 30 people (another was DaveN) who got caught in his auto rogue bot filter because I surfed his site with a search engine UA.
Let that be a warning to all of you who have the Firefox UA switcher plugin installed. If you screw up and leave it set on Googlebot, you will be banned.
Banned from WebmasterWorld
This morning I started getting an XML Parse Error on my WmW feed. I did a quick header check and found:
——————————-
(Response for http://www.webmasterworld.com/)
HTTP/1.1 403 Forbidden
Date: Fri, 16 Dec 2005 19:13:44 GMT
Server: Apache/2.0.52
Accept-Ranges: bytes
Content-Length: 3985
Connection: close
Content-Type: text/html
(Repeat request)
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
——————————–
I guess Brett didn’t like me posting about the fact that he’s cloaking is robots.txt file?
The funny thing about it is the fact that the 403 is serving a linux test page. Come on BT, if you are going to ban me, at least have the balls to print "403 Forbidden" on the page.
Network Solutions to Acquire MonsterCommerce
Network Solutions to Acquire MonsterCommerce
Congratulations to Stephanie Leffler and everyone else at MonsterCommerce. I can’t think of a more deserving group of people.
Welcome Back Brett
I was doing some test surfing this morning using a new user agent/header checking tool Dax just built. Just for fun, I loaded up WebmasterWorld with a Slurp UA. Suprisingly, I was able to navigate through the site. I was also able to surf the site as Googlebot and MSNbot.
I quick check of the robots.txt with several different UA’s showed that MSN and Yahoo are now given a robots.txt that allows them to crawl. However, Google is still banned, and humans still must login in order to view content.
Apparently, it’s been this way for awhile because both engines already show a dramatic increase in page counts.
MSN 57,000
Yahoo 160,000
I’m not sure why he’s still banning Google. But at least there are a couple places where you can once again search WmW.
IMO, it’s a smart move. But I’m sure there will be those who will complain about not allowing humans to see the page they click on from a SERP without registering.
Update: It appears the BT is allowing visitors coming from Yahoo to view the page listed in the SERP. You aren’t forced to login/register until you try and click on a second page. That seems like a good compromise to me.
Evil Jeremy Part 2
One of the things about blogging I haven’t yet come to grips with is the idea that anyone other than my 10 drinking buddies will be reading what I write. When discussing search with the brew crew, I always have the luxury of knowing everyone present has a good understanding of where I stand on all the major issues. That being the case, I never have had any problems with someone leaving the table completely clueless about a point I was trying to make.
I now understand (thanks john) that I need to be a bit more obvious here on the blog, so let me lay out in great detail where I stand on the subject of paid links.