Rank is the new hits
I sincerely hope the little snippet entitled
Digg is (almost) as big as Slashdot posted at Techcrunch (attributed by Mike Arrington to Brian Benziger, though Brian didn't blog about it himself) and linked by O'Reilly Radar with only an "Uh, wow" does not start a trend. Web site metrics are slippery suckers at the best of times, and they will bite you in the arse if you don't watch them.
Specifically in this instance, the graph Mike links to is highly misleading. Alexa's data is spurious to begin with due to its methodology, as is detailed in the comments on TechCrunch following the post. Ignoring the Linux/Windows and IE/Firefox battles for a minute, the whole concept of a ranking graph has big problems.
To put it in language that Web 2.0 adherents understand, think of the Alexa ranking as a Long Tail graph. Slashdot is already way up there on the rump of "hit" sites, whereas Digg is gradually making its way up the tail from obscurity. How much more of an achievement is it for Slashdot to rise one place in the Alexa rankings? If you believe in the Long Tail, then you know that Digg's achievement in creeping closer to Slashdot is not impressive enough to justify a claim that "Digg is (almost) as big as Slashdot". A more revealing graph can be found on Alexa itself: the daily reach. Well looky here, it seems Slashdot is matching Digg pound for pound in audience growth! However could that be? The answer lies in the page views, which shows the two sites' lines converging, to Digg's credit. So Digg is not catching up to Slashdot in numbers of users, but instead is merely getting more sticky? That's a story worth telling, but it's not the story Mike wrote.
One other trap for new players is revealed when going back and looking at the vertical axis of the rank graph. Ruh-roh, why are those markers spaced differently? Why are the markers near the Slashdot line bunched up together while the markers near the Digg line are spaced out? Anyone who has a cursory knowledge of statistical analysis would call bullshit on that, because it has the effect of making the lines look closer than would otherwise be the case for a straight graph. Give us the raw numbers in a consistent format please Alexa, don't come the raw prawn. And why do you distort the rank graph but leave the other two pristine?
We went though this in the first dot com boom, people. I'm old enough to have been working in the industry back when PR for your company's shovelware site consisted of blaring out how many million hits you had per month. Never mind that with all the images on each page, including infinitely-repeated single-pixel-GIF hacks, that each page view generated hundreds of hits. Oh no. How long did it take for page views to take over as the key metric? Years, wasn't it? Let's not go through that again. Ban ranking from any and all future metric discussions, please, or at least put a big-arse disclaimer wherever it is mentioned.
Note: it seems like I'm picking on Mike here, but there have been many pieces written about the Digg vs Slashdot comparison in the blogosphere lately, none of which have drilled down into the numbers to find the real story.
Digg is (almost) as big as Slashdot posted at Techcrunch (attributed by Mike Arrington to Brian Benziger, though Brian didn't blog about it himself) and linked by O'Reilly Radar with only an "Uh, wow" does not start a trend. Web site metrics are slippery suckers at the best of times, and they will bite you in the arse if you don't watch them.
Specifically in this instance, the graph Mike links to is highly misleading. Alexa's data is spurious to begin with due to its methodology, as is detailed in the comments on TechCrunch following the post. Ignoring the Linux/Windows and IE/Firefox battles for a minute, the whole concept of a ranking graph has big problems.
To put it in language that Web 2.0 adherents understand, think of the Alexa ranking as a Long Tail graph. Slashdot is already way up there on the rump of "hit" sites, whereas Digg is gradually making its way up the tail from obscurity. How much more of an achievement is it for Slashdot to rise one place in the Alexa rankings? If you believe in the Long Tail, then you know that Digg's achievement in creeping closer to Slashdot is not impressive enough to justify a claim that "Digg is (almost) as big as Slashdot". A more revealing graph can be found on Alexa itself: the daily reach. Well looky here, it seems Slashdot is matching Digg pound for pound in audience growth! However could that be? The answer lies in the page views, which shows the two sites' lines converging, to Digg's credit. So Digg is not catching up to Slashdot in numbers of users, but instead is merely getting more sticky? That's a story worth telling, but it's not the story Mike wrote.
One other trap for new players is revealed when going back and looking at the vertical axis of the rank graph. Ruh-roh, why are those markers spaced differently? Why are the markers near the Slashdot line bunched up together while the markers near the Digg line are spaced out? Anyone who has a cursory knowledge of statistical analysis would call bullshit on that, because it has the effect of making the lines look closer than would otherwise be the case for a straight graph. Give us the raw numbers in a consistent format please Alexa, don't come the raw prawn. And why do you distort the rank graph but leave the other two pristine?
We went though this in the first dot com boom, people. I'm old enough to have been working in the industry back when PR for your company's shovelware site consisted of blaring out how many million hits you had per month. Never mind that with all the images on each page, including infinitely-repeated single-pixel-GIF hacks, that each page view generated hundreds of hits. Oh no. How long did it take for page views to take over as the key metric? Years, wasn't it? Let's not go through that again. Ban ranking from any and all future metric discussions, please, or at least put a big-arse disclaimer wherever it is mentioned.
Note: it seems like I'm picking on Mike here, but there have been many pieces written about the Digg vs Slashdot comparison in the blogosphere lately, none of which have drilled down into the numbers to find the real story.
posted by Paul Montgomery | 10:07 am
1 Comments:
Yeah, I think you're right - the numbers are only suggestive, rather than authoritative. Digg is certainly gaining popularity, but the Alexa rankings don't tell the whole story. Luckily, plenty of people pointed this out in the comments of the TechCrunch post.
Post a Comment
<< Home