Understanding the complex network that drives stories to the front page
Last Wednesday at about 11 a.m., tech journalist David Cohn was perusing his RSS reader when he came across an article titled “Measuring the ‘Colbert Bump.” The piece summarized the analysis conducted by a political scientist who wanted to quantitatively measure the effects a politician would see if he were to appear on Comedy Central’s The Colbert Report. The study found that Democrats who go on the show “enjoy a significant increase in the number and total amount of donations they receive over the next 30 days when compared to similar candidates who do not appear on the show.”
After Cohn finished reading the article he did what he often does when he comes across an online item he finds particularly interesting; he submitted it to the popular social news site Digg. Several hours later, after hundreds of other Digg users voiced their support for the story by clicking on the “digg it” button, his link reached the front page. From there, it would go on to drive tens of thousands of readers to the article in a server-crushing wave of traffic. With over a million registered users and many more casual readers and RSS subscribers, there are few sites that can provide more online exposure than Digg.
But despite proclamations of its democratic community — one where a group of users works in tandem to pluck out an important news item to push to the masses — not every vote is created equal. Cohn’s link, for instance, didn’t make it to the front page simply because it led to an interesting story; the tech journalist is part of a band of elite Diggers able to consistently catapult its submissions over the threshold that separates the “upcoming” stories from the “popular.”
In a Thursday phone conversation, Cohn told me that he opened his account in 2005, when Digg was mostly a tech community.
“How the site used to work is completely different than how it works now,” he said. “Now there’s sort of — and this is a credit to Digg, from my perspective — it’s harder and harder to predict what stories will go to the front page … It used to be based on the numbers alone, like if a story has 150 diggs it’d go to the front page. In the past it did feel like you were able to — not control, necessarily — you could leverage power over your submission to get to the front page, and that’s harder and harder now.”
As the site grew increasingly popular, it became the target of more and more users who wanted to game the system; entire companies sprouted up offering to promote stories for money and homogenized groups worked together to promote their own agendas and content. The site administrators eventually rolled out new algorithms that necessitated a “diversity” of votes to make it harder for a group of 50 or so malicious Diggers to hijack the site for its own purposes.
But despite these new hurdles, the elite group still enjoys an enormous amount of power when it comes to promoting its own links. And it wasn’t luck that awarded this power; almost all of Digg’s top users engage in regular behind-the-scenes networking in a you-scratch-my-back strategy, a series of instant messages, shouts and emails that come together and thrust hordes of stories past other competitors.
As many relayed to me over the past few days, it’s a game that takes a good bit of persistence and an abundance of spare time. And, because of a lack of transparency from Digg employees, there has been a growing tide of conspiracy theories about this group and others, claims of “bury brigades” and secret cabals of editors that comb through upcoming categories, bashing down any links that don’t promote a particular viewpoint or bias.
In the past week I’ve conducted phone and email interviews with over 50 of Digg’s top users in an attempt to piece together the cohesive links and strategies that make them so successful. It was during these discussions that I discovered that these Diggers are much more than a team of link pushers and self-marketers; many of their conversations have later developed into real-world friendships.
“It’s mostly through IM,” Michael Abehsera told me earlier today. “I speak to most of them and know them personally. I talk to them on the phone. It’s a network, we all speak to each other. All the people who get five or six stories a month on the front page, we see each other on IM and say hello to one another.”
Abehsera’s current account is about three weeks old, though he’s been a user for much longer; his last account was deleted for what he openly admits were questionable (blackhat, even) tactics in promoting his submissions (“I don’t do that stuff anymore though,” he told me). I asked him to explain how much more influence a Digger who takes the time to market one of his links has compared to one who simply submits the story and then backs away.
“You have to have networking,” Abehsera asserted. “If you have good content and not the networking, if you don’t know how to optimize content, then there’s no way to get to the front page. Unless you happen to be really lucky.”
When I expressed a certain amount of skepticism of this — after all, shouldn’t an important breaking New York Times story become popular regardless of who submits it? — he waved off my democratic idealism, repeating that great content, though important, only makes up a fraction of the criteria needed.
“I don’t think it’s a bad thing,” he said. “While you need to first find an interesting article, Digg gives you that power to sell it. If you don’t have the power to sell it, then it’s not going to make it to the front page. I’m not saying that only power users are successful, because there must be 5,000 to 10,000 people who can get a front page article. It’s just that you have to sell the article, and if you don’t it’s not going to make it.”
Any modicum of doubt I still had was dispelled as I spoke to more of these top Diggers. Each one outlined a honed strategy to maximize his chance of success while continuing to look for any signs of inefficiency in his system. In what can almost be described as blatant politicization, several of the Diggers told of analyzing their friends lists and trimming away those least likely to aid them in their quests for the ultimate goal: front page promotion. This is precisely why you’ll see thousands of user profiles stating things like ( taken from an actual profile), “I digg friends articles. I do think it works both ways: If you find yourself removed it is because you didn’t digg me or weren’t around for awhile. Digg me and I will add you back.”
Brian Cuban — brother of billionaire Mark Cuban — told me that many of the Digg users split off into their own respective niches, developing almost what a traditional journalist would refer to as a “beat.”
“My brother Mark Cuban is attempting to purchase the Chicago Cubs so I have been gravitating to stories about that,” he said in an email. ” I generally come across those links in the Chicago Tribune since those will be the freshest and most relevant on the subject. I feel stories on the Cubs pending sale are of national interest to the Digg sports community. The Digg community has confirmed this. All but 1 of my submissions on this subject have gone [front page].”
Like most the people I interviewed for this article, I asked Cuban whether he bought into any of the more radical conspiracy theories about Digg, including the existence of “bury brigades” that constantly monitor the site to exterminate any stories that fall on the other side of some political divide.
“I think a lot of it is wives tales and it is certainly all speculation,” he wrote. “The Digg community is so large that I think it would take a very concerted effort of lots of diggers to bury a story that would otherwise have national interest. I think a lot of stories that you would not think would appeal to the masses make it though artificially because people just do not care about the subject matter enough to take a positive or negative interest in it and it slides through. This happens a lot with videos. If the subject matter touches a nerve on some level people will respond one way or the other. As along as there are algorithms, there will be MIT math geeks who crack them and figure out how to beat them. I think it is a good thing that people probe for weakness. That leads to improvement and innovation.”
This sentiment was echoed by most the others I spoke to. Many seemed to have a fair amount of confidence that the Digg gods, in their omniscient wisdom, were managing to stay ahead of the game, tweaking their system to fend off any organized effort to undermine Digg’s chaotic-yet-pure body of links that it manages to churn out every day. Though most admitted that yes, there was still a decent bit of manipulation, they claimed that the Digg community managed to weed most of it out, with only a very small number of questionable stories maintaining their perches atop the front page balcony.
Of course it was inevitable, however, that I would come across long-time users who were entrenched in nostalgia for better times — those who exclaim daily in the comments section of stories that Digg has lost its way, that it has become subject to lowest-common-denominator stories in its attempt to appeal to the broadest base of users.
I happened to catch a user named Kelly, who Diggs under the account tomboy501, when she was admittedly buzzed on two glasses of wine and therefore susceptible to writing the stream-of-conscious email that I received from her late Friday night.
“Here’s some reality: digg is gamed beyond belief. Period,” she wrote. “Here’s the secret: Want to your stories to be popular and go to the front page? Blind digg. Blind digg everything the power users submit. You’ll get there soon. It’s a gross and disturbing display; digg is a mere shadow of it’s former glory.”
She went on to bemoan the running memes that have become staples of Digg– “It’s 4chan. It’s lolcats. It’s Goat-Fucking stories. It’s 12-year old bathroom humor that will make you king of the frontpage.” — and her desire for the community to devolve back to its tech roots.
“I love digg in many ways,” she said. “Some great obscure content there if you know where to look…but it’s buried deep in the site. I keep waiting for digg to right itself. And the community to realign. Maybe one day. One day I won’t care much ..and will give up…and will fade away with the rest of them. For now…it’s a bad habit I can’t seem to break yet.”