AdSense Bot Working Overtime
During last Tuesday’s Rockstar show, I mentioned that I had been working on a project that got a bit messed up due to the fact that Google’s Mediapartner bot (aka Mediabot) was being used to index content for Google’s database. We had setup some 301’s for Googlebot, but had neglected to redirect the Mediabot. The end result was a whole bunch of duplicate content due to the fact that we were serving Mediabot the old url, and Googlebot the new one. Both were getting indexed and added to the cache.
Matt was in the chat room when I made the original comments, and he said that he’d like to see some examples. So I thought I’d post one from this site.
The content of that post got indexed in a template that we only serve to AdSense. It has no navigation and no comments; just the actual post. We built this template to experiment with getting better ads to display. The idea being that it might be possible to get ads other than blog related products to show if we removed all the content that wasn’t part of the actual post.
The interesting thing to note about this page is that the post was originally made in January. And for quite sometime it had a cached page that was a representation of what Googlebot was given. But then Mediabot visited on April 7th. And the page it was served on that date ended up replacing the Googlebot version in the cache.