Spolsky: Hello, OK so the technology is working now, and the bamboo for our lobby has arrived.
Atwood: Ooo neat.
Spolsky: We're in the new office and it's going to be really noisy: There's a guy doing some drilling to install the shelf for the AV gear where we're going to install all the electronics that we need to play Rock Band in the new office so there'll be some noise from that. None of the walls are here or the doors to any of the offices - there was this plan was that the front of every office would be a glass wall with a sliding glass door. But those are nowhere to be seen and apparently they'll be coming in mid to late october.
Atwood: Do you guys have the computer setup so you can actually work? Do you have the internet and stuff like that?
Spolsky: Yeah, and in fact, our systems administrators did just a a fantastic, heroic effort - came in on the weekend while everything was being moved and pretty much had everything up and running for us for us by the morning. Uh oh! There's a guy that's bringing - this is not good...
Atwood: Is a one man band approaching?
Spolsky: There's a guy delivering a shelf.
Hey Liz? ... Looks like they are delivering this couch here and it needs to go where the dumpster is... the dumpster is ... Although actually, I guess they're gonna set it up tomorrow so it doesn't matter. ... Alright. Probably safe. They're moving the dumpster.
Oh, [someone] just glided by on a scooter. We've got people bringing in bicycles and scooters since the office is much bigger than it used to be. So it just takes longer to get all the way down to the other end, it's just a long straight shot from Rector Street to Broadway Avenue. You can check it out on Google maps. And we basically go from one street to the next.
Atwood: Thats very cool. You guys don't have Segways yet though?
Spolsky: That would be good, I'm all for Segways.
Atwood: Yeah, that's fair. So, the big news this week, in addition to you guys getting moved in which is cool. Is, of course we now have a public website.
Spolsky: Congratulations! We launched.
Atwood: So you wrote a blog post and I wrote a blog post and I think the amazing thing to me is that the site is still up because I really wasn't sure - I really didn't know.
Atwood: It's just hard to do that level of load testing but the site has survived and just to give you an idea of the dramatic increase in load our CPU graph used to be at 2-3% most of the time. Minimal, even under pretty heavy beta load and now it's pretty regularly at above 50% sustained.
Spolsky: Wait a minute.... When I publish something - on my blog at least - there's always like a day or two of heavy traffic that it generates but then it ends, I mean it subsides. I'm sure the same is true for you. And I also send out an email to 52,000 people who subscribe to my email newsletter. What a blast from the past that is! Email newsletter - what's that?
Atwood: Right. You'd assume this is some kind of worst-case scenario, right? This is probably more attention than we are going to get later which is good because we gotta have some room to scale.
Spolsky: There'll be a lot of attention now but that will sort of be covering up an underlying growth curve so once the noise goes away there'll be some pretty heavy growth there.
Atwood: Right. I actually got an email from Jeff. We did hook up Google Analytics and I do have some actual numbers if you want to hear them for day one.
Spolsky: Oh yeah, totally! Totally!
Atwood: On day one - just random numbers here - 1,500 questions were asked and in context during the entire length of the beta I think we had 8,500 questions and that was a month and a half so in one day we generated a pretty sizable percentage of everything that was done during the beta. Almost 6,000 answers, 1,700 comments were left, 62,000 unique visitors, and almost 700,000 page views.
Spolsky: 700,000 page views. In a day?! Wow!
Atwood: Our site is kind of amenable to people looking at pages over and over, right? I mean, you want to see the new questions. It's the kind of site you hit refresh a lot.
Spolsky: One of the things that struck me as - you know how we talked last week about how we're all stupid? So here's a stupid thing that people said. [laughs] You know a lot of people had been commenting saying that StackOverflow is great with this elite super-dooper beta with very few people coming in but what's going to happen when a million people come flooding in and start asking questions?
Spolsky: And your questions only stay on the home page for two minutes instead of 15 minutes or something.
Atwood: The lowest I saw it get during the beta was about 20 minutes. Now it gets down to two minutes so it's ten times higher.
Spolsky: But the point is that we also have ten times as many viewers - as much traffic - and so a post is still seen by about the same number of people even though it survives for less time on the home page. I think the only problem is if the unanswered questions count starts climbing uncontrollably. That might be a sign that somehow that we've lost the balance. We did have a balance in the beta where things were getting answers - they really were. Only very obscure questions weren't getting answers but the only way to fix that is just to have gigantic scale where we've got a couple of hundred thousand regular visitors so there's likely to be somebody who knows about that Quickbooks XML integration with Visual Basic 4.0 bug. So for that you need huge scale, you need Google and that's something were never going to get good answers to those questions until we had much more critical mass. But in terms of a person asking how to do X with some technology getting answers then their question is going to appear to about the same number of people as it would have before the beta - even if it doesn't live as long on the home page.
Spolsky: Assuming that the mix of people looking to answer and looking to question is roughly the same and the demographic mix of the people looking to come into our real site are the same mix of demographics as those that came into our beta - and I really think they are, I think they are pretty close.
Atwood: Well I do think, as we've said, many times before one thing that works in our favor is that it's kind of a selective audience. I mean, it should really be programmers. I can't imagine any sort of random internet user coming to our page and ...
Atwood: ... wanting to stay there for any length of time. Because it's just much of technical jibba-jabba about really arcane things ...
Spolsky: Jibber Jabber.
Atwood: ... but that's the strength. That's why we think it's going to work. And yeah, we were lacking a long tail of programmers in the private beta. I did get one email from Jason Cohen of Smart Bear Software and I guess he works with Justin Standard. And Justin was the person that asked you at the Business of Software Conference about the community-owned questions. That was actually Justin.
[Correction: Jason Cohen and Justin Standard do not work together, though they were both at the Business of Software conference. Justin is the one who approached Joel at the conference. It was Jason, not Justin, who sent Jeff the email. Whew.]
Spolsky: Mm hmm.
Atwood: So they're heavily involved with the site and I got some passioned email from him (Jason and Justin) urging us not to take the site public. They were really deeply, deeply concerned about the eternal September thing of just, you know, bunches and hordes or random users coming in and really diluting the quality of the site. But I don't actually think it works that way. I don't agree that's really a risk. I think the bigger risk is having an insular community where you don't have the long tail so these really obscure questions can never really get answered. So I think there's a bigger downside to being a closed community than an open one in our case. Now, if we were something like Yahoo! Answers maybe. And actually this brings up a good point and a transition to something I want to talk about. We're going to have a special guest in this call. So MetaFilter - there was a nice topic on StackOverflow on MetaFilter and it got quite a bit of discussion and that was really a big honor because I've been a big fan of MetaFilter. But MetaFilter is a closed community. It costs $5 to sign up for MetaFilter. I'm looking at Wikipedia - that's something institutied in late 2004 - but it's really kind of a selective community. You can read it of course but you can't really do anything unless you pay the $5 lifetime membership fee. It's really just a little barrier to get people in there, and the quality of the community is, in fact, very high.
Spolsky: We have our own five dollar barrier.
Atwood: What's that?
Spolsky: It's called OpenID. [laughs]
Atwood: That's a pretty minimal barrier.
Spolsky: It's about five dollars. It's about five dollars of headache.
Atwood: I disagree. One of my favorite threads and I think I had to close this one because it was beta stuff was somebody was complaining about OpenID and then the highest voted answer was - I don't remember who this was so I apologize - I think I'm going to make up a name. "My name is Jim. I was a VB programmer for 16 years. I was able to sign up for this site." Even if you're a Visual Basic programmer you can figure it out and make it work. I think there's some complains about it, but the people that are complaining about it - I don't get it. I think the status quo is so... evil. Having just thousands of logins across thousands of sites.
Spolsky: I think if we can do something to promote OpenID then it's worth it. Speaking of VisualBasic programmers: Another frequent gripe that we've been hearing is - there's the vacuum cleaner, I was telling Jeff earlier that they're vacuuming in the office and he said: "Can we not just do the podcast after they've finished vacuuming?" and I said: "Yeah, that will be in November sometime." [laughs] There's sort of an asymmetric long tail to construction projects, they just go slower and slower and slower as they get to the end of the project.
Spolsky: Can you hear the vacuuming or is my excellent microphone...
Atwood: A little tiny bit, it's not in any way obnoxious.
Spolsky: Well, yay Seinheiser. But the real point was: You mentioned VB and another - I don't want to say criticism - observation that we got from a few people was that the site seems heavily .Net oriented: C# and .Net are the biggest categories of questions. I actually didn't think that we were disproportionately .NET oriented. I don't have statistics for this but I actually believe that our distribution of tags is probably pretty close to the actual distribution of working programmers in the world. I don't think people realize just how common it is to be a C# .Net programmer because it's mostly the internal and the enterprisey applications that do that. The people that make a lot of noise are the bloggers with their web [me-]too-point-oh startups using the Ruby on the Rails but I just think that actually our distribution of tags feels to me to be pretty close to the actual distribution of programmers working today - the actual technologies people are using. I don't know that that's true, maybe a listener can suggest something we could compare ourselves to - maybe books or job listings or something.
Atwood: Right, I was actually surprised and pleased. This is one of the major goals when we launched the site, right, was to be agnostic. Right we are not saying this is a place to discuss Windows programming, this is just a place to discuss programming, period.
Atwood: I think really the challenge for us is there's still lots of people even though like I've tried to make it really obvious like a little sidebar that says, OK, this is what a question should be and actually made it bright yellow for new users now, by the way-- it really pops off the page. It's, like, ask programming questions. And we still get questions that are just not remotely programming questions. I mean, they're computer related, they're not programming questions. And I feel bad, like, voting them down and stuff, but I think we have to, and that's more the risk than not having... There was a huge question about bash tricks the other day, which I thought was very cool. So we're actually getting... There Python questions, I've seen a handful of Ruby questions. There's quite a bit of activity around non-Microsofty technology, which was an explicit goal. And I've been pleased with that. I think the bigger risk is people not asking programming questions-- It's frankly a bigger problem to me than "Are we Windows-centric?" I don't think we are.
Spolsky: Well what do you mean by... If it's a question that's just "What's the best programming chair?" - that's kind of a programming question. Or...
Atwood: That's to me is fine. It's questions like "I'm having this problem in Windows XP."
Spolsky: Oh. Yeah. If there was a real way to tag those out of view... You know what I mean?... If they came in appropriately tagged and I never saw them I wouldn't care personally. And that goes back to the question of tags which I want to bring up. The first time I tried to do a Q&A site it was really just discussion groups, and there's still one of them which is the .NET questions on my home page - if you click on .NET Questions there's a discussion group there (which I'll probably close down now we have Stack Overflow) where people ask questions and answer them about .Net. Of course for some reason in the past I used to think that the right way to go about building something like Stack Overflow would be to start small with something that you can easily accomplish and to go narrow so that you at least have a chance - I would generally tell people to pick a narrow niche because it's easier to reach the people that are in your niche. Maybe I would have said: "You know what? Let's just make a site for Ruby programmers trying to learn Python." And if that's really successful then you can expand it. You see all those Q&A sites out there for individual technologies - there's a lot of Microsoft SQL sites, four or five of them - in fact those are notorious for being behind paywalls. Obviously there are great sites for other technologies, Ruby - what was the one I keep forgetting - Delphi; there are great Delphi sites - but the idea of tags allows you to make one site, make it super broad and not worry about it and not have to reach anybody, just tag it and it's really kind of working in a way. I mean think of the original UseNet which had - I'm sorry, I'm being distracted because they are rearranging the couch that was just delivered.
Atwood: There was a hierarchy of newsgroups early on. I don't know the history of it. But that's effectively like tagging when you have, you know, this explicit hierarchy. Although we do not have a hierarchy. We just have random tags that are applied to questions. And some people actually want that rigid hierarchy, but I don't know of a way to do it that's not, like, a giant pain in the butt.
Spolsky: It's so much better not to have to find the right place in the hierarchy to randomly tag things, to make a new tag on the fly even though yesterday I was the first person to use the tag for the DHTML edit control - an obsolete piece of technology I'm still working with, to my chagrin - but that tag is correct, and in fact it's also tagged VB6 because that's VB6 code. And you know what? I got a great answer to that question - I asked how to implement smart quotes using VB6 and the DHTML edit controller and I got code that did it!
Atwood: Wow. That's impressive.
Spolsky: Can you believe it?
Atwood: I think it's the ultimate measure. Is it actually working in practice? In other words, if you ask a question-- a real question, not a BS question-- with some problem that you're having, do you in fact get an answer? And I've repeatedly got email-- I would say at least ten-- from people who are like, "Wow! That's what I did. I asked that was hard and obscure and I got a really good answer."
Spolsky: Another thing that I think is funny - I guess this is a third point of criticism that we're getting which is: "Well, look at this question! It has two bad answers and one even worse answer and the even worse answer has more votes!" So: OK, go vote it up! [laughs] That's why there's a little vote button right there! You can provide a new answer. In other words, a lot of the problems are reminding me very much of the way the mainstream media responded to Wikipedia, which is when Wikipedia was first coming out you would see all these mainstream media articles saying: "Well, I went to Wikipedia and I looked up this, that and the other thing and this was wrong and that was incorrect and the other thing didn't even have an article at all, and literally by the time this thing made it into print and was in people's hands, in the black and white newspaper printed on newsprint paper and delivered to somebody's front lawn with a rubber band around it for the birds to poop on [huh, huh - he said "poop"] by the time this had arrived there the Wikipedia entries had all been fixed.
I think that's where people miss the point about the editing and the answering that you can say: "Here's an example of something that's not right." But when you have a malleable site, it just gets better and better and better. Which ever argument you try to use, it's just going to get weaker and weaker and weaker.
Atwood: Right. Yeah, that's totally fair. Well let's go ahead and add Josh to the call. OK? This is one of the MetaFilter, one of the core staff there-- there's one, two, three, four, five people total, so he's one of the main team. He's on standby so I'm gonna go ahead and attempt to add him. Let us see how this goes. ...to our call, add caller...Josh. OK.
Spolsky: Oh! It's working!
Atwood: It's ringing. It's ringing. Let's see what happens here.
Spolsky: You dingy?
Spolsky: Uh you know? It just ended the call recording and started a new one. Thats ok, because I can merge them if its really...
Atwood: Merge them. OK, that's good. Hi, Josh. Welcome to the call.
Josh Millard: Thanks, am I coming in OK here?
Millard: I've got kind of a fiddly setup, OK.
Atwood: Yeah, you sound great, you sound wonderful.
Spolsky: Josh, are you on an airplane using the WIFI on American Airlines with a laptop?
Millard: Actually I using the built in mic on my Macbook Air was my main concern, I didn't know how good it was gonna sound, but if its coming out OK, then we're in good shape. I'm actually going over a Spring EVO[?] card which has been rock solid since I got it, so... so yeah!
Atwood: Very cool!
Atwood: Thank you for taking the time to do this. Let's open... Let's talk about MetaFilter for people who may not know what MetaFilter is, and who you are, and what that' all about. Could you maybe just talk about that for a little bit?
Millard: Yeah! What is MetaFilter is one of those weird unanswerable questions, but the stock line at this point is that it's a community weblog and its essentially a collaborative blogging site on the front page MetaFilter.com where anyone in the user base can make a post on whatever subject, its a generalist sort of anything-goes place. Kind of in the same circles of what you think of as like BoingBoing or like Reddit if you took only the 20 or 30 top posts and didn't make it quite so tech centric.
Millard: But everyone in the community has a right to post, everybody in the community has a right to comment in threads, so somebody makes a post, everybody kind of jumps in and talks about it, some get a whole lot of discussion, some not so much. Thats sort of the classic MetaFilter function. A few years ago we started "ask MetaFilter" which is probably the most heavily visited part of the site where folks can go to ask questions to get answers. You had mentioned that you saw the thread yesterday on MetaFilter where from MetaFilter users side you could see some of the discussion in terms of StackOverflow vs ask MetaFilter. Which I thought was kind of interesting seeing those two going against each other, but you can see kind of the identification people have on the site with that function as well. So thats basically the two big things the site does, it also has some other subsites that are less well trafficked. I'm one of the three main moderators there. Matt Howie of course started the site back in I think July of 99 and he's been running it since day one, Jessamyn West has been moderating on the site since I think the beginning of 2005, and I came on officially as moderator early last year, so I've been there about a year and a half now.
Atwood: But I've been looking at your profile on MetaFilter, and it says...
Millard: Oh, yeah: Been there forever.
Millard: I think April 2001 is when I joined - spare time in college and memepool didn't have enough posts on any given day so I've been hanging around there forever and grown up a little bit in the mean time. I definitely wouldn't want 2001 me moderating the site.
Atwood: Yeah, 2001 feels like so many years ago. It really wasn't that long ago.
Millard: Yeah, it's strange - the time dilation, especially with the Internet.
Atwood: Yeah, that's very true. I think a lot of people on the Internet view MetaFilter as really a huge success story in terms of-- and when I say success story I want to qualify. So before you were on the call we were talking about how do you measure if things are working on StackOverflow.
Atwood: And what I said was, "Well, are people asking programming questions and getting good answers?" Right? That's the ultimate measure of any site. Does the unit of work actually produce anything useful?
Atwood: That's what's amazing about MetaFilter-- that people ask questions that are actually very interesting questions, that are very well researched on just any topic you can really think of and get this really amazing, really, like useful discussion on it. Not like your usual, "how is Babby formed?" type of discussion that tends to occur. And in that sense I looked at MetaFilter as a site that we sort of wanted to emulate in some ways when building StackOverflow. I was very, very gratified to see all the discussion. And it was quite an extended discussion. I really enjoyed reading it on, you know, what MetaFilterites thought of StackOverflow and what we're doing there.
So, yeah, that was very exciting for me and just having you on is very helpful. So I think the first question I have for you almost immediately is, How does one become a moderator? It's a pretty select club. We have the same issue at StackOverflow, although
Atwood: There's not a lot that moderators do that's unique, but one of the things that moderators can do is sort of force things over the delete threshold. And some other sort of high-level tasks that I wouldn't feel comfortable giving out probably ever to anyone, no matter how... How did that occur? How does someone get blessed into being a moderator?
Millard: On Metafilter it's only happened twice - or I should say two and a half times - but Jessamyn coming on, that was a big move for Matt because he'd been doing it by himself and just plain hadn't gone there previously(?) and it became, as much as anything, a workload issue. But she had also displayed really good sensibility towards Ask MetaFilter since it launched and he saw that she was someone who really got what was going on with the site, which was obviously important, but that she was also someone with a pretty even temperament, someone who did not so much wear her proclivities on her sleeve, she was able to disagree with someone without flipping out in classic Internet fashion. I think it was really that was what he ended up identifying in her was here was someone who was cheerful, someone with a good demeanor, who was good talking to people, someone who can actually state fairly why she made a decision and who really, really gets the site. I think having just glowingly described her and saying - gosh - the same thing for me feels a little self-serving. But, you know, that's how I got into - I was around, I spent a lot of time on the site, probably since about 2006 I became really involved and interested. I spent a lot of time in MetaTalk which is the back channel part of the site where people can actually talk about site policy, and moderation, and what should fly, and whether someone is misbehaving, etc., etc.
Atwood: Let me pause there because that's interesting to me. One of the issues we have on StackOverflow is it's sort of become like Fight Club where I don't really want people to talk about StackOverflow on StackOverflow. It's hard for us because we have aspects of wiki which is like community ownership. But we also have aspects of individual ownership - strong aspects of individual ownership. You have a reputation system, you can vote on stuff. These are kind of contradictory goals at some level. You can't service one without taking away from the other in some regard. I think one way this comes to a head is, like on Wikpedia for example, there's always the talk page. To me, the talk page on Wikipedia was just like a jumble. I would go there and be, like, I can't even follow this. I couldn't event follow what was on the page. So I ignored it for the longest time. But I see the need for it now because you have a topic and then you have metadiscussion about the topic. And those two are really very different audiences. But they're kind of demanding. So maybe you could talk a little bit about how you guys do that at MetaFilter. It sounds like there's a special place and it's somewhere off the site.
Millard: Yeah, it's not off the site, it's actually right there if you know to look for it. But a lot of people don't necessarily know to look for it. Matt set up MetaTalk sometime like 8 months after he started the site, it's right about the beginning of 2000 like March or so, because people were talking about MetaFilter on the front page. It's natural enough. People would say, hey what's with this, hey look at the post, hey this guy's a jerk. So he started up MetaTalk and started directing stuff that was metacommentary to that part of the site. Delete something and say, hey take it over there. If people wanted to have an extended argument that was derailing a thread, etc. And a lot of people when we talk about what make MetaFilter work, MetaTalk is one of the things they mention if you talk to a regular from the site as being key to the success of the site because it creates a sort of release valve. It's the same thing with talk pages on Wikipedia because I had the same experience the first time I checked those out. It's not necessarily comprehensible to the casual user, like, what is going on here? But for the people who are regulars, the people who develop a certain amount of passionate attachment to the sites, or really, really need to make their voice heard out of day one beyond just normal participation, you have this safe place you can let people ... let their free play fly, as it were, without damaging the core function of the site. You don't have big messes on the front page.
So there's a pretty strong culture of regulars who hang out on MetaTalk. Insofar as you have the big contributors and the serious regulars at any given site that makes up the core of the community, there's a strong correlation between that and who actually spends time on MetaTalk dealing with policy stuff and talking about user issues.
Atwood: Right. I totally get that. Because this is one of the things about designing social software - that you don't really understand the stuff until you've lived it. And I totally get that now. Because for the longest time I was like, just don't discuss this stuff here.
Spolsky: Hey, Jeff?
Atwood: Go ahead Joel.
Spolsky: I'm kind of with you. I still sort of really hate when discussion groups become about the discussion group and one of the reasons - I think my reason - I just want to hear, Jeff, why you were so viscerally against talking about StackOverflow on StackOverflow, because my reason was always that the questions were always newbie questions - "Why does this work this way?" - and they were very very very repetitive. What would happen is people would wander into the site for the first time and immediately start asking all the same questions about it. So discussion of the site, or of the discussion group, on the discussion group was inevitably extremely boring to the people that had been there for more than 25 minutes. And they could have read the FAQ but people wouldn't, and people on StackOverflow every single day from now until the end of the Universe you're going to have a person wandering on saying: "What is this OpenID? Can't we do this without OpenID?" And that's going to be the first question they ask. And you're gonna have one of those until the end of the Universe - officially. Guaranteed. And they'll be like: "Why are you down modding me to negative 364? I'm a newbie here! You guys are mean!" [laughs] Jeff: Did you have any other reasons why you don't like the site becoming a site about the site? Other than the sheer arrogance... [laughs] of the site that is about itself? [laughs]
Atwood: Well I think ultimately you're leaving bread crumbs for the next programmers who come along. And I do not feel that leaving bread crumbs about StackOverflow is ultimately helpful to programmers. You know, the average programmer doesn't really care exactly how StackOverflow works. They care about "If I ask a question will it be answered?" and that's kind of a solved problem at this point. I mean, that part actually works. So then it's just noise, I think, to the average programmer.
Atwood: But I do feel that for people who are highly engaged with the system, these are your star users. And I feel like if you alienate these users, you're kind of hurting yourself in the long run. And that's where I'm a little freaked out about it.
Spolsky: The question is...
Atwood: Go ahead.
Spolsky: Can you make a place for them to socialize?
Atwood: That's what I'm looking at now. I think what we need to... The thing that sucks is that we've almost made a system that's too good, at the risk of sounding incredibly arrogant. Because it's really easy, it's really easy to have a discussion there-- it's so easy.
Atwood: Right? It's completely frictionless. And that was a goal-- that was an explicit goal of the system. There's no login, right? You just go in and start typin', man, and it works! And people love that aspect of it. And I think if we've... We've got to reduce the friction. If we have a discussion forum, there has to be as low friction, almost as low friction as the site. I think there will slight intolerance for logins.
Millard: It's interesting 'cause the whole format of the site really presents itself as, "Hey, go crazy with the questions here." And there's nothing, glancing at the front page, that really-- I just set up an account all of 30 minutes ago, I think, so I'm sort of exploring it while we're talking-- but there's nothing that really shouts "Don't talk about this site here" even though my way of thinking would be, OK, there probably somewhere to talk about it. I found the uservoice.com subsite poking around which seems like sort of a place where people can more talk about the site in a metafashion. I don't know if that's...
Atwood: More in a bug/feature crust[?] way. How should we be moderating? Like that's one of the big threads that happened. And it's sort of catch-22 because I don't really think people should be moderating unless something really bad happens. Joel and I felt for a long time-- Joel more strongly than I, but I'm starting to come around to Joel's way of thinking on this-- that you don't really control the site, just let the voting do what it's supposed to do. If people don't like content that's on there, they will vote it down. And then below a certain threshold it just sort of stops appearing in any major area of the site. So someone asks sort of a dumb question, or something that's "Why can't I log into Windows XP?"-- it's not really a programming question-- it just gets voted down. Nobody has to step in and really do anything other than vote. And then for the things that do require moderation are really obvious, like really evil things. And we have, even in that case, you don't actually have to be a moderator-- we have that "Really Offensive" link. Maybe that's a good question for you guys. So you guys have a pretty closed community. It costs now five dollars to register which is just sort of like an arbitrary barrier that was instituted?
Millard: Yeah, it's a speed bump.
Atwood: Right. But I'm sure that helps tremendously. That's gotta help a lot.
Millard: Oh yeah.
Atwood: But what kind of [?] do you have with spam? How much moderation do you do on a regular basis? Let me put it to you that way.
Millard: At this point I would characterize my work at MetaFilter as being basically a part-time job. But it's spread out across a given day the way just Min, Matt, and I all sort of do bits and pieces as we go along to be kind of a full-time job nonetheless because there is sort of a lot of trust built into being a MetaFilter user. We mostly let people do their own thing. Mostly people are pretty good. There's a fair amount of community policing-- if someone does something stupid, people will tell them so. They'll call them out in the MetaTalk area of the site, for example.
But with spam we don't deal with too much. The biggest kind of spam we get is people dropping linkspam into AskMetaFilter questions, especially old ones. We'll see people google for something and they'll find a AskMetaFilter article in the first page results and then they'll sign up because they have a business selling widgets in that, and they'll say, "Oh, I found a really great site called widgets.com. They're great. You should check it out." You know, just linkspam because they know it's gonna get some traffic. So we probably see something like that every day or two. Some days we see a lot of it.
Every once in a while we'll get a honeypot question where a whole bunch of people show up. There's a World of Warcraft account management question months ago that we went back and looked at random and found that like five or six different wild gold[?] farmers had signed up over a period of a couple of months and pitched their own various sites. And there's this whole spammer subcommunity hiding in that one thread which actually inspired us to build a new tool to look for answers coming in very late to threads that are otherwise dormant.
So that's kind of how we've dealt with it. That's the sort of thing we deal with. It's mostly people coming in and try to take advantage of the pretty good page rank that AskMetaFilter threads get. Or people who sign up explicitly to spam on the front page. You know, they sign up, you have to wait a week, you have to make a few comments. It seems like a fair amount of people have figured out that this is the system because we'll see someone who waits a week, makes exactly the right number of comments to allow them to post, and then posts within hours of when they're first able to post. So at this point we get an email every time someone makes their first post. It tells us about it, we check it out. Sometimes it's a spit-take and we dam it immediately. Sometimes it's someone who's a legit new user. You know, to a degree there's a kind of that technologically-assisted profile on how someone behaves on a new site. It's a big assist for us in keeping the spam pretty much annihilated. Which is one of the things that I find very weird when I look at a site like Reddit or Digg or a pretty vast amount of the blogosphere, really, when you get down to it that is much less tenacious, much more friendly to linkspam and blogspam and astroturfing. And I realize to some extent it's a matter whether or not it's worth their time and energy to stop that, because you know, we're paying attention to the site every day, all day long, to make sure the stuff doesn't get out of hand. But some of the sites just seem to not care. You know, Digg and Reddit...
Spolsky: Josh - I disagree with you there. I have not seen spam on the home page of Digg or Reddit for years.
Millard: Oh, and that's... This is one of those things where I'm annoyed enough at enough of this stuff that I'm gonna probably be harsher in calling stuff spam. You know, I see people PR blogging, people hosting their own stuff, and really using Digg and Reddit as an attempt, and maybe a less successful attempt as time goes on here, to more self-promote than to actually do anything like actually organic link sharing. That sort of stuff. Maybe that's not spam but as far as I'm concerned, if the function of your site is to put interesting content out there, and what people are doing is using it as a platform to jam their stuff out there and hope that other people, or...
Spolsky: Well, if it's not good stuff it gets voted down early on. In the early days of Reddit there was some discussion and questioning about whether it was OK to post a link to something you had just written on your blog and they decided it was OK - that you can post your own thing and there was no reason to down vote it; just go read it and if it's good, vote it up and if it's bad, vote it down. In fact...
Atwood: Joel, you should explain your involvement with Reddit so he has some context.
Spolsky: Yeah. My involvement with Reddit is nothing really, I guess. I just met those guys really early on...
Atwood: You have your own sub-Reddit. Come on. You have the Joel Reddit.
Spolsky: That's true.
Atwood: I don't have a Jeff Reddit.
Spolsky: You could make one now.
Atwood: I could. You can now. They actually added that feature. I'm not saying you're biased. I'm just saying you have a certain amount of domain knowledge. I'm not in any way calling you [?].
Millard: That's fair. All three of you acknowledge that I coming obviously partly from the culture and annoyances on the MetaFilter that's very much-- there's very specific ways in specific parts of the site where you can do that sort of thing and otherwise it's really just verboten.
Spolsky: It seems to me - and I maybe missing something that's going on here - but it seems to me that just having the idea of a community vote is extremely effective at getting rid of spam very, very quickly...
Millard: Yeah, it's...
Spolsky: Because the only way to fight it is to make a bunch of fake accounts and have them all vote. You know spammers do do this so you need algorithmic ways to check for the people making a bunch of fake accounts and voting things up, but it's just been forever since I've ever seen something on the home page that I thought the community didn't want there. Now, what the community wants is a whole 'nother story - because Reddit, for example, changed very much from kind of a hacker startup site with a bit of a libertarian bent, into...
Atwood: Ron Paul!
Spolsky: Exactly! It just became, and for some reason, there was a particular political community that just really fed on that and drove out everybody else amid a certain amount of conspiracy theory and a certain amount of the Ron Paul business, and everything the mass media does is against us. Their favorite kind of article would be some injustice that was done to a ninth grader somewhere by their librarian - and that was the ideal Reddit article that would immediately zoom to the top of the home page. OK, that was fine and that's the reason they created a Joel Reddit because I told them: "Listen - one of the things that..." I mean, my suggestion to them when this site was barely even up yet, there was nobody outside of Y-combinator using it, that I thought that at some point that it would become a self fulfilling prophesy where a small clique of people started voting up a particular type of article driving away people that didn't agree with that type of politics, or just disagreed, and eventually you would wind up with a particular niche and you could never become a horizontal application everyone wants to use - you could never become a huge mass success in the way that Facebook is - because no matter what, you're eventually going to get very very like minded people...
Millard: Yeah, absolutely.
Spolsky: And so I said: "You gotta make it possible to make multiple Reddits." As long as you have these niches you want to have a lot of them. It's fine. You want to be the atheist conspiracy theorist site, that's terrific but let me also make one for people that are interested in the business of software and so that was what the Joel Reddit was. And I think that idea... The history it that they've then used that idea to build a site for Conde Nast called "Lipstick" - I think, I don't really know what the name of the site is, it has Lipstick in it - and it's just a version of Reddit that Conde Nast sponsors and that introduced them to Conde Nast who bought the company.
Atwood: They have rolled out that feature. You can make a custom sub-Reddit.
Spolsky: You can make as many as you want and there are a whole bunch of sub-Reddits. You can do this really nice thing on Reddit, which I wish you could do on StackOverflow, where you can say: "I'm interested in programming AND business and... UFOs" and you'll get a home page that shows you a bunch of all that stuff.
Atwood: Right. We're still working on that in terms of customization. We have some of the infrastructure but, yeah, I agree. So Josh, just to give you some of the background, Joel was pretty heavily... Joel knew Reddit really well. For example, not to be an elitist or anything, I don't really go to Reddit or Digg. I really don't. Like, I sort of have my blogs that I follow. And actually one of the things I follow is "The Best of MetaFilter" feed. And one of the things I like about MetaFilter is that it's truly all over the map. Like, I don't know if there's any one type of community on MetaFilter. I've viewed it as people who are really good at the Internet. If I had to characterize MetaFilter, that's my really rough characterization.
[garbled phrases and laughs from all three]
Millard: Yeah, that's probably my favorite part of what happens at MetaFilter. You know, it's a mix. There's also a fair amount of news that show up. We're trying to not be a news site. Not everybody gets that. It's sort of an ongoing challenge with different perceptions of what the front page is all about. But, yeah, you do see the same stuff that going to show up on Boing Boing, Digg, or Hot Key[?]. Yeah, but you also see wonderful posts about something. It's not just "Hey, here's this link," it's "Hey, here's this subject. Let me put together a nicely crafted collection of Internet resources about something that not everybody knows about." It's a great way to kill some time. If you got a rainy day, just go to MetaFilter and look at one of those posts.
Atwood: Right. And to clarify, when I talked about using MetaFilter as a reference for StackOverflow, that's exactly what I'm talking about. Or if somebody comes in on some really narrow programming topic, but they really know their craft, and they can write at some reasonably good level. So they write up the problem in a way that's like telling a story. Like, here's this problem and here's all the things I did to try to fix it. And then people are having this conversation about this problem and it ends up being this fairly definitive resource about this problem in a way that an average person can actually understand if you have some programming background.
Millard: Sure, yeah.
Atwood: So yeah. So that's one use of the system. Then there's sort of the drive-bys. Like if you're talking about news, about some quick and dirty thing that they need to get to. And I think that's fine too. But I guess my perspective is a little bit skewed because I'm only seeing the best of MetaFilter. And, actually, how is that selected? Like, how does that work?
Millard: What are you looking at? I know...
Atwood: I'm looking at the feed "The Best of MetaFilter" on Bloglines. I don't remember where I got it from actually.
Millard: This is maybe an embarrassing thing to say in my position, but I pretty much never touch RSS readers. Not because I don't like them, just because long ago I developed the habit of going to the place that I'm interested in when I have the time to go there. And I find a feed reader still a little bit weird because it can be too much or too little at any given time. We have a lot of RSS feeds built into the site but I don't really remember which ones are which. The "Best" may be driven by the popular favorites, which is something I think we created a feed for relatively recently, like in the last year or so. It basically takes sort of the equivalent to upvoting on the site. There's a "favorite" function where every post and every comment can be favorited by a user. So doing that same sort of thing... We don't really use it as an internal metric. It's just sort of a way for people to both say, "Hey, this is neat" or keep track of it because that can go back to it and check through their favorites-- sort of a bookmark function-- that's how it was born.
Atwood: Say I post a question which is really a bad question for a variety of reasons. So how does this get flagged? First of all, there's this discussion. Essentially the model is very much like collaborative blogging, right? So that's the model. And obviously I have a hugely successful blog and Joel doesn't like to call his thing a blog, but it is in fact a blog. So we know this model, right? And I allow comments on my blog, Joel doesn't. So I know how that works intimately and I believe it works really, really well. But so you're saying it would be sort of discussion in the commenting: "this post sucks."
Millard: Yes, there will be. We've got a flagging system built in. We don't really have up and down votes. We've got favorites which can be sort of taken as up votes but they don't do anything automatically except for stock like the popular favorites feed. But the flag system is kind of like a down vote if you wanna think of it that way, but it functions differently-- they're not visible, it sends a flag behind the scenes we can see from the admin site.
Atwood: Just to interrupt you real briefly. One mistake we made. We have a flag like that called "Offensive" and we made it visible, and that was a huge mistake. Huge. Do not make that flag visible, because people would get, like, for whatever reason people would tag things "Offensive". Who knows why people find things offensive. Like one thing I did want to do is have a drop-down, like, "Here's thirty reasons-- pick one of these thirty reasons that you're marking this." No, that's way too much thinking. It's like the evil...
Spolsky: 15 dollars for a piece of checked baggage. Which is that?! [laughs] Is it spam? Or whatever? I mean, it's obviously offensive! [laughs]
Atwood: Yeah, it's offensive. It's offensive to pay 15 dollars for checked bags. But who knows? And people would see this and, like, freak out and get kind of pissed that somebody found their thing offensive. So one of the earliest changes we made was like, OK, you can't see that anymore, only moderators can see that count.
Millard: Yeah, and I think that's really smart. Yeah, you have the problem where the person who is being flagged is going to be upset, you have the problem where you're going to end up with a metadiscussion in the thread by other people who agree with or disagree with that flagging-- you get this sort of, what we call the "king of the shitpile" function-- something we've tried to avoid on MetaFilter by not making the bad stuff visible. Yeah, so making that moderation-view-only is smart because that way if someone has a weird objection, you know, it's only going to be this one-off thing. It's not going to stack up. You're not really going to see much of it. You can ignore it. No one needs to know that it's being ignored. And then, yeah, if you get that real volume then you know, OK, we've got a problem. That's exactly how we tend to deal with things on the site as well. You know, we'll see something will get an odd flag here and there. We check out each flagged message to make sure there's nothing wrong. But if it's only one flag-- it's probably not a big deal. Whereas we'll see stuff that has five flags-- OK, there's something up. Every once in a while we'll see something that's got thirty flags because we're all at lunch and, you know, it's been a half-an-hour and it hasn't been deleted and it's something that really needs to go.
Atwood: That's still pretty low. Because you guys have a pretty active community. So you would say that thirty is like a colossally large number. And five would be enough to trigger, like serious attention.
Millard: Well, yeah. It's an opt-in system. You know, if you're a brand new user you may not even know about the flag system. We don't try to sell it particularly hard. We cover in the FAQ, but who reads the manual, right?
Millard: So the people who actually do flag-- the nice thing about that is that you don't have people just punching a shiny widget as soon as they join the site. They're probably going to learn about flagging in the process of becoming familiar with how the site works and becoming a more active member of the community. So when they start flagging they kind of know what's up and they aren't just flagging friviously.
Millard: And beyond that it's pretty much OK on the site to have some sort of civil statement of objection if someone's doing something problematic. It's definitely OK and sort of classic that people can have a conversation in a thread briefly on why something's not a good idea. Works pretty well if nobody freaks out. We've got that MetaTalk section that people are explicitly expected to take serious discussions about this stuff to. And then, in theory, nobody starts a big fight in the thread. In Ask MetaFilter, in theory, if there's a problem with the question people flag it and move on. In practice of course some people are going to be annoyed enough that they're gonna jump in with a comment, they're gonna say something nasty in the thread or just say, "Hey, this sucks," "You're a bad person for posting this."
Atwood: But some of that, you said, is expected, though.
Millard: To a degree it's gonna happen. And if it happens in small doses and it's done with a relatively civil tone, it's kind of OK. It really kind of depends.
Atwood: So what do you do in the case that-- we haven't gotten to this point yet, flirted with it, and it's obviously a very slippery slope-- but in terms of dealing with users who are just need to be excommunicated for some reason. Like, what do you do?
Millard: We do it very rarely, actually. As far as actually banning someone, it's really not common. Setting aside spammers, you know, anybody who's going to come up and just, like, abuse outright the whole fundamental idea of the site, yeah, they're going to go.
Atwood: This would cost them five dollars at some level, right?
Atwood: So that's already kind of a barrier.
Millard: Yeah, that's a kind of up-front disincentive to come up and act like a jerk.
Atwood: It costs you five dollars. Normally it's free, it's free to be a jerk. You can just be a jerk for nothing.
Millard: Yeah, so there there's many other places where they can get a bigger bang for their buck, if that's all they want to do. So, yeah, we'll give people time outs-- give them 24 hours that they can't log into the site. Or a week if they're really seeming like they're unhinged or need a break. And we do that occasionally. But it's really not common. Mostly what we do is deal with... Matt, and Jess, and I all write a fair amount of email every single day, including, you know, we'll drop someone a line and say, "Hey, this is kind of a problem. You need to cut it out. Talk to me about this if this is going to be an issue." And that works most of the time. We don't get a whole lot of crazy, raging cases, I think because of the paid signup as a barrier to keep a lot of really nutso, random drive-by people out. Yeah and beyond that, yeah, we try and work with people. As much as anything we may talk them to death in email if they really just want to fight. Once they've wasted our time and spent themselves, they go away on their own right. We don't usually have to just nuke someone very often, because most people will either chill out or just go away of their own accord.
Atwood: Well there's obviously some scaling issues there. Because you started with Matt, then Jessamyn, and you. So it looks like the staff has been growing over time, I assume to keep up with this administrative overhead.
Millard: That's definitely a big part of it. That's what brought Jessamyn on in the first place is when Matt realized the job was too big for him to keep doing as a one-man affair. And same thing bringing me on when it continued to grow.
Atwood: One thing we flirted with is having users sort of rate each other. But Joel quickly headed me off at the pass there and felt that would lead to what he called "bad high school days flashbacks."
Spolsky: Hey, hey, hey - we can have them rate each other's pictures!
Spolsky: I was thinking more like: We already got Wikipedia, Reddit and Sex Exchange - why not also have Hot-or-Not?
Millard: It could work, you've already got the voting architecture.
Atwood: [...] my life sort of ends, right?
Millard: The scale thing is obviously a real concern. With MetaFilter, we've been trying to keep it from going crazy with scale - I mean the site's obviously grown continually over the last nine years, and it's going to keep growing - but it's not zooming. We're not trying to turn this into something that's going to scale into a million users. I can't imagine what that MetaFilter would look like. That may be a little bit different from how people are approaching these type of things. You see someone who want to set up a community site, they may really want it to be a hit, they want it to be big, they want it to be huge. But to some extent, with that kind of scaling, [ringing] if what you're doing is primarily social, you're kind of trying to set up Yahoo Answers [ringing], and Yahoo Answers is not what I'd consider a success as something with communities.
Millard: Yeah, you can find that stuff.
Spolsky: Jeff, think about the whole scaling problem: If you have one person who's making stupid Gravatars that are problematic, on a huge site - even if you have ten people - you still have the same proportion of people that have this particular problem, that are anti-social in this particular way. If that proportion is low enough, who cares? So there's one or two people that are doing unacceptable things, the chances that anybody would even encounter them on a site are very slim. You know what I mean?
Millard: If you provide something like an offensive flag or a way for people to let you know there's a problem, then even when it does come up, you can deal with it as you go [beep]
Millard: So I don't think it's necessarily a huge issue, if you can keep on top of it. Obviously if it's scaled beyond your ability to control it you have to find another way.
Millard: Yeah. And that's the sort of thing: if you've got a user who's really causing problems to a point where people are flagging a lot of their content, behind the scenes are moderators you can see that easily enough. We certainly, as far as aggregate experience goes, Me an Matt and Jess can really tell if a user is a problem, we don't need anybody to be able to put a black mark on that user's record for the public to see. If we get repeated issues, yeah: we're going to talk to that person. Without it having to be a spectacle. You don't have this aspect of public shaming necessarily is how it's handled.
Which I think is a good thing, I really dislike that aspect of how things work on some sites I don't really like the idea of what you're going to do is that you're going to mock your users in public as a way to deal with them. That's certainly not a unanimous feeling; there's a certain social value to public shaming, but it's certainly not something that I think works well on MetaFilter, and I think in any site that's trying more for community than a certain function that happens to have people driving it. If you really want a community, you've got to have the mutual respect all around, which means you don't have the people in charge treating users - even jerky users - like crap. You gotta take the high road - period. Which is hard to do.
Spolsky: Not right now.
Spolsky: I'm going to think of it as soon as we get off the call. I'm gonna be like: "Wait a minute - what did you do about..."
Millard: We didn't really engineer it in on purpose, but the fear that new people feel about making a post to the front page I think is actually part of what makes it stay pretty good. A lot of people have expressed in MetaTalk that they don't post because they're not sure that they'll do a good job.
Spolsky: I've always been worried about that because actually I have the exact same problem which is that I keep discovering pockets of people that won't apply for a job as a programmer at Fog Creek because we have a reputation as being so strict about who we hire and, unfortunately the thing I'm afraid of is that a lot of the people who are really, really good don't think they're that good, and a lot of the people who are not that good think that they are that good. And so for people to self censor themselves and say I'm not going to apply and I'm not post to the home page of MetaFilter, I'm not going to ask my question. Maybe you get filtering but you don't know if it's necessarily quality based filtering.
Millard: Yeah, it's a mixed bag, I've seen the same thing and it's interesting: the front page has that fear aspect for some people who won't post, or at least are very hesitant and slow to finally post because they're worried about that. Then you look at the AskMetaFilter side of it and it's some much more obviously "let us help you out with something" function that I don't think we see that at all! We see people who sign up - and we make people wait a week before they can post their first question to avoid - again - a speed-bump [something] random spammy stuff, and we get email probably once a week from someone asking "Hey what do you mean I have to wait a week?" so there's a complete inversion of that, people are really enthusiastic to jump in, 'cause anybody can ask a question, it's not like writing a post about seventeenth century art
Spolsky: Or buying a handbag!
Millard: Yeah [laughing] So yeah, it's an interesting dynamic on the site itself in it's different parts.
Millard: No, thank you for having me.
Millard: Yeah, no , I'll definitely let you know. I'll be exploring the site some.
Atwood: It's very complicated, it's StackOverflow.comSpolsky: StackOverflow.com - thanks Jeff. And finally, there's a transcript of the show, a transcript wiki, it's a wiki where people contribute to a transcript of the show. Which is very helpful for finding things that we said that are embarrassing later, or for quoting the show from your blog. Right, you did that yesterday.
Atwood: That is true, but I find that it gets done a lot faster when you -
Spolsky: Disagree. If you are doing the transcript out of the goodness of your own heart transcribe things I'm saying. If you are doing it because you believe you are going to get some kind of a badge, or Jeff is going to owe you something in a future universe, or that you'll get some coupons in the mail good to buy songs on Rock Band II - transcribe things that Jeff said and we'll see who gets more of their Jibber-Jabber transcribed.
Spolsky: In the meantime, once again the phone number is 646-826-3879 which goes to a voicemail box. Leave us a message and we'll play it on the next show. Thanks very much and we'll see you all next week.
Atwood: And thanks Josh.
Millard: Yep, thank-you!
Spolsky: Thanks Josh.