Thursday, January 25, 2007

Link-based search algorithms lose followers

After earlier in the week Wikipedia's Jimmy Wales decided to reinstate the rule that all outbound links in Wikipedia would have the rel=nofollow attribute, meaning that search engines would not count them for the purposes of calculating a page's popularity, Dave Winer has announced that he wants to do the same sort of thing with Techmeme for his blog.

Actually, what Dave announced was that he was blocking Techmeme's crawler script, called Wazzup, via's robots.txt file. However, it came out eventually at the Wordpress annex that he didn't want to block himself from appearing at Techmeme at all, which is what the robots.txt change would have done, but wanted to stop his links being used as part of Techmeme's algorithm. He was quite fine with being indexed and his stories appearing as headlines, he just didn't want to be a secondary discussion link.

From a technical standpoint, there is no way to get exactly what Dave wants at the moment, because the rel=nofollow attribute is not search-engine-specific, as robots.txt can be. Maybe, if Dave can get some support for the idea, there will be a new "version" of robots.txt where the site can still be indexed but you can specify that links are to be discarded for the purposes of TechMeme or Google et al.

It all adds up to a trend of political and epistemological discontent for the power structure that link-based search algorithms have accreted. Andy Beal decided to champion a crusade to reduce Wikipedia's PageRank to zero by encouraging every blogger to add rel=nofollow to their links to Wikipedia. Wales' stated reasoning was to prevent spam by cutting the power source for SEO experts who were filling Wikipedia with decreasingly relevant links, while Dave's agenda appears to be more to subvert the hierarchy of patronage that Techmeme has become. I wonder what the next step will be in this evolving power struggle.


Post a Comment

Links to this post:

Create a Link

<< Home