February 2nd, 2006
Interview with Digg's Kevin Rose, Pt 2: On Personalization and Fighting Spam
In Part 1 of my interview with digg founder Kevin Rose, we discussed digg’s popularity, reputation systems and recent issues with GroupThink. In the final part of this interview, Kevin and I talk about digg’s battle with spammers and the upcoming release of new personalization features, plus an API.
On Fighting Spam
Richard: Now to the question of spam. I know you’ve had a number of issues fighting spam recently, so I wondered if you would explain some of the technical ways that you are fighting this spam and making the system better in the future - in that respect.
Kevin: The way that we’re fighting it the most is allowing the community to handle it. Right now approximately 1,200 news stories are submitted per day. When you have a site that gets as much traffic as digg is now starting to receive, you have users that will try and game the system. It’s natural for them to try and create false accounts or send personal blog posts. But, the digg community quickly sees through a lot of this - and users are marking these stories, and removing and flagging them from the queue itself.
One of the ways they do this is through The digg community quickly sees through a lot of this a feature we have called Digg Spy. It allows you to see what people are digging, commenting on and burying - in real time. So there’s a community of users that will sit there and select which modifiers to monitor. For example they may choose to see only those stories being buried and how they’re being buried (as spam, as duplicate post, etc). And they’ll actually work together as a community to filter out these types of stories that are being posted.
Richard: Do the algorithms you use in digg detect spam automatically - or is it all human intervention?
Kevin: There are systems behind the scenes that notify our moderators, who are watching the site, whether there’s any activity that we can detect. It’s probably 95% user-driven There are a few different methods that I really can’t get into, that we have set up to keep an eye on things. We can tell based on what the average of incoming stories is, stories with certain types of links, and we basically look at all different types of things across the site to get a good index of what’s going on - there are certain flags that are set up that notify us of certain types of fraudulent activity.
But for the most part I’d say it’s probably 95% user-driven, so the users handle most of that for us - which is really nice from the admin side, in that we don’t really have to have a large staff that’s policing the site. It’s actually only one person who watches the site in general - it’s not really that big a task because the community handles most of it.
On Personalization
Richard: Onto the plans for personalization, I read a fantastic article in BusinessWeek at the end of last year in which you said that you’re hoping to expand digg into more of a personalized system in which people can create their own communities, e.g. for non-technical topics. How’s that coming along, when will we see the first signs of that?
Kevin: Well that’s something we’re working on right now. One of digg’s top priorities is to expand outside of tech news It’s one of our top priorities to expand outside of just technology-related news. We want to open digg up to just about anyone and everyone that wants to express their interest in any type of news story or Web content. So we’re in the process of working on that right now and we hope to introduce this in the next few months.
Richard: Just before I got on Skype with you, I sent an email out to the Web 2.0 Workgroup (which I’m a founding member of) and asked if they had any questions they’d like to ask. One of them is from Dion Almear from ajaxian.com, who asked: "At the moment digg doesn’t work for me as I am not a 17 yr old Xbox playing PHP hacker." He’s obviously being a bit generic there, but will there be a variety of social networks in digg in the future? It’s kind of different to the personalization question I asked earlier, because this is about whether there will be more formal digg social networking communities that you’d set up?
Kevin: Right now the features that we have for connecting users are primarily digg will enable users to create cliques of users the friends functionality that allow users to monitor what friends are digging, commenting on, and submitting. But we’re very much focused on expanding and allowing users to create cliques of users that get along and want to discuss things on digg. So if there’s any type of content, whether it be news or no matter what we get into later on - we want digg to be the location where users can come and share that information easily.
APIs
Richard: John Musser from programmableweb.com wants to know about your plans for an API. I think you mentioned that in the BusinessWeek article, so what’s the latest on that?
Kevin: Shortly after we roll out into different areas outside of tech, we’ll be finalizing the spec for the API and releasing it - and pretty much giving anyone who wants to create an application to manipulate the digg data the ability to do so. Hopefully we’ll see this out right after we launch the new sections of digg.
That concludes this two-part interview. Part 1 is here.
