The StackOverflow Podcast

Ads, intro.

Spolsky: Everything is going... falling apart this week. Like, I'm surprised my hair isn't literally falling out.

Atwood: What? Based on your move, I guess?

Spolsky: Yep, so the first problem was that we had to delay the move. So we scheduled Verizon to move the phone lines and that kind of stuff and we delayed the move and so we rescheduled that and then it really canceled the turning off of various lines so the entire office is now operating off one DSL line.

Atwood: (laughs) Awesome

Spolsky: So that is to explain why we may have lousy sound quality.

Atwood: It's ok at the moment.

Spolsky: They disconnected our fax line. We used to have like five phone lines at the office and now we have four, everybody's always using them and various machines are crashing and the aquarium is falling apart. It's just... The whole place is just gonna be a complete disaster area by the time we leave, it's gonna be totally trashed.

Atwood: Right, well moves can be pretty traumatic. I think that's somewhat normal for a move.

Spolsky: Have you ever done that? You're at your house or your office that you leave behind. When you leave it, you're like: "What a dumpster! How did we ever work there?" Because when you away all the stuff that you use, all that's left is this garbage that was hiding in the corner.

Atwood: Oh, yeah, sure!

Spolsky: And everything you take down,  all the little pictures off the wall and there's a rectangle where the paint's still clean behind the pictures. (laughs)

Atwood: Well if the leaving part is painful, what about the arriving part? Is the arriving part gonna be as traumatic as the leaving?

Spolsky: Well, we're supposed to have internet in the new place. In fact, we've had it in the new place on a different floor so they just have to move floors, which shouldn't be so hard and so far we haven't been able to get the people who installed the locks on the doors to agree to come in time. That means like bicycle locks on the doors or something. I don't really know how we're gonna deal with that.

Atwood: Yeah, not being able to get in might be sort of a show stopper.

Spolsky: We could get one of those super strong incredible New York bicycle locks. What are they, kryptonite New York bicycle locks? And then we could issue everyone one of these Bic pens they could use to get in. Do you remember that?

Atwood: Oh yeah, the pen thing was very controversial because somebody posted a video showing how to break in the lock using something like a standard pen.

Spolsky: You take the cap of a Bic pen and you put it in where the key goes and you turn and the lock pops open! That was all you had to do!

Atwood: I think that was an early example on the power of the internet that was changing the way people designed products. Because in the old days, let's say somebody knew that exploit, right? How would they get that information out to the world? They really couldn't...

Spolsky: Right, it would be in these, like, lock picking magazines and stuff, but that wouldn't be readily available.

Atwood: Right, but in the era of YouTube, it's in front of everybody, so...

Spolsky: Yeah, oh and even before YouTube, you could write about it, but nobody could read so they wouldn't care.

Atwood: Yeah, that's a good point, you know it's weird to think about, like, YouTube didn't exist until like, really 2005. I think it existed a little earlier but nobody knew about it. it's like you go back and I remember even going back through my blog entries and you'll say 'why didn't I put a video here', and it's because YouTube didn't really exist then.

Spolsky: (laughs)

Atwood: The landscape has changed so dramatically, it's hard to even imagine and maybe not even YouTube, but just the fact that video can be posted, right.. by a lot of people is really common so, yeah, I think that's a huge step in the evolution of the internet / tv. I mean, it's a big deal, like you said a lot of people don't like to read so the audience just went up by what, a factor of 10? Maybe..

Spolsky: You can tell that certainly the people that post comments on YouTube can't read or write.

Atwood: Yeah, very true. Well, I hope things smooth out for your move cause you know we're going to be there.. we did buy tickets and everything so the Stackoverflow team is coming to New York city, I believe the 20th. I'm arriving, I'm taking an overnight flight so I'll arrive really early on the 24th, which is a Wednesday, and we're staying until Sunday.

Spolsky: Well, if you get to the office before any of us do, just bring a Bic pen...

Atwood: (laughs)

Spolsky: And then you'll be able to get in.

Atwood: You know what I'm more concerned about? Is you know, I ordered Fog Creek a whole Rock Band set out of completely selfish interest of course, cause I want to play when I get there and you guys got to have that set up. I mean, even if the doors are locked, just set it up outside in the hallway or something.

Spolsky: Well, we have.. lets see, the projector screen is installed, but needs to be wired by the electrician so that you can cause it to descend. The projector itself has been ordered, the cabling for the projector is already in the ceiling, but might be discovered to be flawed which would be a catastrophe. The receiver has been ordered, and the speakers have been ordered.

Atwood: Good, all that stuff should probably arrive by the 24th, I would think. I mean, it's the 10th, so that would be two weeks.

Spolsky: I ordered an AV rack to rack it all up on, and you know the problem with like this AV gear, as I discovered, is the first time you install receivers and DVD players and stuff is that getting around to the back of them to hook up all eight speakers that you have to hook up so that there's like sixteen little screw terminals that you have to screw things on and if you've got this like up in a closet somewhere on a shelf, it's a real pain to reach back there and screw on all these little speaker things.

Atwood: Oh yeah, but it's a one time deal, theoretically.. if you get it set up right.

Spolsky: It is, but the only way to do it that's even half reasonable is to somehow like hold this 75 pound receiver on your knee while you're standing on a stepladder and screw on all the connections on the back and put them up and hope that nothing comes undone. So I ordered this rotating rack, which I hope works out well. It's this rack that you install and it spins around.

Atwood: It's like a lazy susan.

Spolsky: Yeah, in fact, I took a lazy susan, which I didn't like which had been used for cheese, like a marble lazy susan, and I used it for my beach house audio gear.

Atwood: Very cool. So hopefully the stars will align. We'll see how that goes. And big news for me as a big fake plastic rock enthusiast, Rock Band 2 does come out on Sunday, so I'm not really planning to get anything done except for launching the site I guess.

Spolsky: Ok, as a not-Rock Band enthusiast, what is Rock Band 2 and what...

Atwood: Well Rock Band 2 is mostly the enhancement of the existing system. They didn't rebuild it from scratch.. They just fixed a lot of the annoying edge conditions in the first one, they did add some new features. Probably the biggest new feature is that you can play, there's a band mode called 'World Tour', which is essentially the campaign, where you and one of your friends or yourself if you play two instruments, like if you sing and play drums or sing and play guitar, you sort of go on a world tour with your band and eventually you get into the hall of fame and you go through all these cities and events and venues, and it's actually kind of fun.

Spolsky: They had that in the first version, didn't they?

Atwood: They did, but you couldn't play online in that mode. That mode was strictly offline, so in Rock Band 2, that's an online mode so there's all these battle-of-the-bands type events that you can get into with other bands on the internet. You can also have people from the internet join your band as it goes on tour around the world. So that mode becomes a lot more robust I guess you could say..

Spolsky: As you're telling me about this thing, what's going through your head is fun fun fun and what's going through my head is.. do we have a Cat5 drop in the AV closet? (laughs)

Atwood: (laughs) Well there is a wireless adapter for the Xbox 360, I did not buy that, but we can get it if you want because it's kind of pricey. You know the console world is annoying because they nickel and dime you for every stupid thing an I don't want to get the wifi adapter because it's like ninety dollars for this stupid WI-FI adapter, but if we need it, absolutely I will get it when I'm there, so if we need to, that's not a problem. And for local play, you don't have to play with people from the internet so that's nice to have.

Spolsky: They always try to get you to plug your Blu-Ray player or your HDTV player into the internet and I'm like 'no way, you're gonna use that to break my HD-DVDs that already work, aren't you?'

Atwood: Right. Well yeah, 'cause of copy protection, sure. 

Spolsky: Well that's literally what it is, so they can revoke things and theoretically upgrade the firmware. But, you know.. the firmware better work the day it comes out, nobody's going to release a new Blu-Ray disc or HD-DVD disc. It's Blu-Ray, it's all Blu-Ray now, right?

Atwood: Yes, for high-definition playback I guess. One advantage to having things connected to the internet, though, is purely for games. There is a lot of patching that goes on the console now, surprising amount. It's almost like console games are becoming like PC games. They are still pretty stable because they target a fixed platform. So, they can only be so buggy. But there's been a few games that have been released that have been pretty buggy. And they actually patched them after the fact. So you would actually want to be connected in that case. It would be a benefit to you; e.g. that the game would stop being buggy and start working. And also they add features, like they patched Rock Band a number of times and actually added things that were significant.

Spolsky: So how did that work? You put in the old disk and it goes and downloads some fix for you?

Atwood: Yeah. There must be something in the loader that says hey, I'm going to check the version on the server. And if it is greater than what I have, and I'm gonna..you know. Sometime you enforce the update, sometime you literally can't play until you get the update because there are exploits or something in the game. So you have to actually get the update. Some of them are optional, but generally it's nice. I mean, consoles are becoming more like PCs, really..longer term. And I think PCs, actually for the purpose of gaming need to become a lot more like consoles anyway. They need to standardize more, have more control over the environment, stuff like that. I've been a long-time believer in the "theory of synthesis". Have you ever heard of it? Some story, and I'll have to look it up in Wikipedia. I remember in high school history class this always make a lot of sense to me. You have thesis and antithesis. You have two things that are completely opposite. Like console world where everything is locked down, there is no internet, the memory is very tiny on these cartridges. And then you have the PC, which is this totally open-ended platform, that can do anything, connect to the internet, the world of choice of any kind of platform. And then over time, they sort of evolve towards each other. You see this a lot in sort of competing things. That they sort of evolve towards each other and the ideas sort of merge together. At least good qualities of each idea merge together. And you can see this to a limited extent with web apps vs. desktop apps. Like they used to be very different, right.. The HTML used to be very limiting. You had that classic post about 5 things you can't do, which at the time was probably true. Now you can do so much in the browser. The browser apps are taking on aspects of desktop apps. And the good desktop apps are taking on aspects of web apps. So I'm a big believer in this evolutionary model where these previously competing ideas start to become more and more similar over time. They just evolve towards each other. And I think console gaming is kind of like that.

Spolsky: So now the game developers are going to get lazy and instead of really fighting to get a quality release. They'll just say, "OK, we can always patch it later". And they'll throw it out there on us. We'll have to learn to wait for x.03 version of every game.

Atwood: Absolutely! Absolutely it does. Because in the old school console mind set, I mean you had one chance. You were shipping on a cartridge, that was it, you were burned into the ROM. You don't get patches. You either get it right or you fail.

Spolsky: Not only that, but your whole business fails because no game has a 2.0. You know the game is done. They played it, and nobody wants to play again.

Spolsky: The difference is that we're getting paid for this! 

Atwood: Yes.

Spolsky: We're doing our jobs and they're having fun. (laughs)

Atwood: Yeah. Well, no, it's fun. I enjoy, ... I've said many times on previous podcasts I really enjoy working on StackOverflow and I really enjoy interacting with the community on it. And, by the way we've got another sign of success: there's another StackOverflow hating site that has just popped up. Somebody just told me about it today.

Spolsky: Oh, they just want free publicity; they just want me to mention their URL. So here's how you ... www.disney.com / dvorak.org/blog.

Atwood: Yes. (laughs) Well, this one even has a logo, though, which is pretty impressive. It's a "crap overflow", and they made our little springy thing into, like, a toilet, you know,  with water. (laughs)

Spolsky: How did they get the domain name?

Atwood: I don't know. I guess it was available, crapoverflow.com. But I'm reading this guy Ivan Hamilton who emailed me about it, and actually Ivan had a really negative interaction with the site. He's actually listing specifically - he's very excited to go on the site - and then he lists the questions that he went to, and he was very very dissatisfied with the answers in each one, so at the end of them he sort of concludes that there's too many incompetent developers, in a nutshell, and the problem is the answers that aren't correct enough for Ivan's taste are getting voted up ...

Spolsky: Yeah

Atwood: ..., and he feels if he puts in answers, they'll just be lost in the noise of the voting at that point.

Spolsky: That's nonsense, ... question after question that's not what happens. And it is true that we are all, as developers we're all hopelessly incompetent and the goal of a site like StackOverflow is to somehow share the correct knowledge wherever it may be as it is scattered throughout the universe, and to cause that to be voted up and to be spread amongst... You know, there's this big universe of dumb programmers, and I'm one of them, and we all have a little bit of knowledge, like I know how to do this thing in VB6 (laughs) which may be useful to somebody one day who's trying to maintain some ridiculously old piece of crap, and we all have these little tiny pieces of information and if we can just contribute a little bit, that information gets amplified, and maybe a thousand other dumb developers will benefit from my one little piece of good information. So, it's really an amplifier of the good information in the system, and any other way that anybody else has ever discovered of doing it, other than the StackOverflow way, does not have ... does not filter in any way the good from the bad. It doesn't have any kind of: this is good material, this is bad material. We're the only site that really has any attempt to in some way flag things. I mean, there's a little bit of "OK I accept this answer" on other sites, and there's a little bit of other sites where claiming that somebody is an expert, not by virtue on these other sites of whether this answer is good, but by virtue of the fact that we kind of like you, because you've had other good answers in the past, we somehow decide that your answer must be good, and I'm pretty sure StackOverflow doesn't have any kind of history like that, other than the fact that you have karma, that doesn't mean that your answers go to the top of the puddle. You have to re-earn your right to have the right answer to a question, every time.

Atwood: You know what we need, Joel, we need a place where experts can go online and exchange information that they're experts on. I think that's what we need. We just need a place for experts to come and exchange information.

Spolsky: You know, Dave Winer used to say, and I doubted him, but I now have discovered that this is relatively true, that whenever you have somebody in the audience who's just basically throwing darts at you, just sort of pissing on everything you do, and making these fake websites that talk about how stupid the whole concept is, and just generally getting angry at the whole thing, nine out of ten times, it's your competitor.  And I said, "What? That can't be, surely there are people who dislike what you're doing." But the truth is, people that dislike what you're doing, most normal people that just say "This is not going to work" or "I don't like this" or "This is dumb" or "This is not going to benefit me" will not spend any energy. They'll just leave.  They'll be like, "yeah, whatever, that's real funny" they won't spend any energy on it. The people that are actually actively spending time registering domain names, designing logos, in order to piss on you? Usually, nine out of ten times according to this Dave Winer theory, they're your competitors. They're the people that have some skin in the game and they need for you to fail. It's worth some of their effort for you to fail.

Atwood: Really, that's interesting. That makes sense, because it does take a lot of effort to really, you know... And this critique by Ivan is really quite long, this is a fairly sizable blog post.

Spolsky: Well I should say, there are also just insane people. It's not necessary that they...

Atwood: There are also just insane people.

Spolsky: Maybe they just have a stake, maybe he's got ridiculous karma at some other web site that does Q&A.

Atwood: I do appreciate, though, that he actually was very, very specific. Because so many times, people say that you suck. It's like, they don't really explain why. Ivan actually was very specific, and I do actually appreciate that. I don't entirely agree, because I think there's, like you mentioned, there's sort of the "expert bias" which I think is kind of an illusion. That you have all these experts waiting in the wings to do stuff. There's really just a bunch of amateurs trying to do things together. The people that are truly experts are too busy to even help, right? And if the experts are too busy to help, what difference does it really make if there are experts at all. Because the whole point of this endeavor is helping other developers, and whether you're an expert or not, if you have no time to help, you're not really contributing to the solution.

Spolsky: It's also, when you look at the questions, there are no experts on these particular questions. I'll give you a great example - I've been doing a lot of C# and ASP.NET. And so, I'm good at -I don't know- 5% of the functionality that that class library exposes to you in that framework, 5% that I'm pretty good at. I've got a young programmer here who's doing some development in ASP.NET, and I have to code review him. And he'll come to me with questions like: "I've got multiple forms on a page; how do I make the validator controls only affect the current form instead of all the forms on the page?" I don't know! I just don't know. Somebody does, lots of people do actually. I just don't know. Even if I was an expert, there will be things that I don't know. I remember there was a point in time, where I was really, deeply, the expert on the old Excel macro language programming model, because I was going through every single line of that and trying to find ways to replicate it in Visual Basic. And I never knew as much about it as some of the people who had made careers writing macros in the old Excel macro language thing. And there were areas that those so called experts may not have known about which I knew about, just because I had read and studied all the documentation. You can't be an expert in one particular field, whatever the field may be, because these things have so many facets to them, and they're so convoluted, there's so many little details. There's things that the designer, the person that wrote whatever it may be, the Document Object Model in the web browser. The person that wrote that very code, may not really know how it's going to interact under certain circumstances which happen all the time. And there may be somebody that has studied this on twenty-seven web browsers, and has a good answer for what you should do. So, it's not really about-- I mean, yes, you can be an expert, but your field of expertise is very, very narrow; and you don't have to go very far away from your field of expertise before you're as good as a newbie.

Atwood: I totally agree with that. And the way that I've characterized this in the past, it's kind of like a number in binary. Right? So you start off with a bit--

Spolsky: Mmm hmm

Atwood: --which is, you know, zero or one. And then as you go from left to right, you flip all those bits, and say you have six bits. By the time you've flipped six unique bits, think how many combinations there are of what you're doing. It was always amazing to me as a developer, it didn't take very long at all to get completely off the beaten path, to where, like, virtually nobody is doing what you are doing. And it's not like you're doing anything weird. It's just that you made 6 unique decisions that are all independent of each other, and software development is just kind of like that.

Spolsky: You wind up in a place where nobody's ever been.

Atwood: Yeah, exactly. Totally agree. And a good example of what you're talking about where even if you wrote the code, say you wrote the code: "Well I'm expert. I wrote the code" which actually does not turn out to be true a lot of the time (laughs). But you would think, the person who wrote the code must really know this. They must be the expert on this.

To go back to my Penny Arcade thing, so the Penny Arcade Expo, they had a panel with Harmonix. And Harmonix is the company that created Guitar Hero, Guitar Hero 2 and of course Rock Band, my favorite game. And the guy who wrote the code for scoring explained that after the game was out for a while people were coming up with ways to get scores that he didn't understand. There were these way to do, it's called squeezing, where you sort of trigger a star power a certain way. Like you get extra points if you sort of nudge the system a very specific way. He had no idea this was even possible and he wrote the code. So I think the community can end up, depending on who it is, can end up smarter than any one expert I think. It just takes an engaged community, and a lot of engaged community to collaboratively build the answers.

And I think what I would tell Ivan in this case, is if he had problems with the answers that people were giving, he should have left comments. We have this comment feature now where you can annotate answers. So if you feel like an answer is truly wrong, definitely leave an annotation on it indicating that. And I think annotations only take 50 rep, so you only need 50 rep to even do this. And other people would see the answer and see your caveat, "Oh wait a minute. There's this huge bug in this algorithm". So again, you're feeding back into the system. And of course, vote it down if you really dislike it. So I think there's ways for this to work and I can't really explain Ivan's reaction to it. I certainly haven't seen that much sort of, wrong or bad information. I think some of it's maybe interpretation, where if you feel there's: You shouldn't have to do any work, the top answer is always 100% correct and you do not have to look at it critically. Maybe that's a misunderstanding of how the system works. I think if you're really into a problem or a question, you'll read pretty much every response there. There's not many responses to most questions. I mean, 10 would be a lot. It doesn't take that much to read the question and look at all 10 answers and harvest the best information from each one. I think that's just something you have to do on the internet. There's nothing about our system where we're going to guarantee that every bit of information you're going to see on there is going to be perfect. That isn't true, certainly of Wikipedia or really any website you'd go to. That would be my advice.

Spolsky: Yeah, Ivan just needs a girlfriend.

Atwood: (laughs) Ivan probably has a girlfriend. No, I actually like this post. Because I like the specific criticism, it gives me specific things. I for one appreciate that.

But let's talk about something different. So last night Joel was chatting with me online and one thing we came up with was.. one thing we were thinking about implementing and you had some serious reservations around this, so I want to talk about it. One thing that's happening to us, we do have users that for whatever reason become negative users. In other words, they create accounts for the very purpose of just messing with your system. They have really nothing positive to add to this system. So we need a way to block these users and prevent them from.. I mean, we could just delete their accounts, certainly. But one thing I was envisioning, and I've got to be careful how I say this, is the ability for other users to flag your account as questionable. Sort of like the way "offensive" works now, but we wouldn't call it "offensive" obviously. But the way "offensive" works now in case you don't know, if you're not on StackOverflow, is: if 10 people mark something "offensive" that essentially removes it from the website, that's a soft delete. That's one of very few ways that content can be deleted from the system. And it's actually working. I did a query last night, I hadn't been looking at all the deletes, I had just known the ones I had done for various administrative reasons. There are a lot of deleted posts where people had hit this community threshold and they [the messages] had been deleted from the system. So we need something like that but applied to users.

Spolsky: Hah

Atwood: And I think you had some serious reservations about that.

Spolsky: Well, here's my reservation and where it comes from. You can either engage with people that want to have a fight, which is what they want. Or you can disengage, and they'll get bored and move on. I never even started thinking this way until a way long time ago when Phil Greenspun wrote that he was having trouble with people, probably on photo.net which is a very early version of Flickr that Phil Greenspun created in probably 1995, and - maybe it was LUSENET which is his version of usenet except it was web-based - and they had one visitor who was just a troll, just a problematic person in the quaint old problematic way before "problematic" meant that it was the russian mafia and they were trying to steal people's laptops and take their bank account numbers and so forth. This was quaint where he was trying to talk about recipes on a photography discussion group, and so that was considered bad behavior in those days. He was just a negative person and they wanted to make him go away. But they thought, "Oh gosh, if we try to ban him, he's just going to make another account or use a different name or get a different IP address or find some way around the system and he's going to be really angry, so he's going to have the motivation to do this. His anger will motivate him." So instead what they did was configured the web server so when anybody came in from his IP address or maybe his range of IP addresses, the web server would time out. It would basically just not respond. It would just run some 30-second loop and then time out. And the net result was that he just thought that the system sucked. He just decided that the system was slow and badly built and gave up on it and went away. And so Greenspun was bragging about this. Somebody can find the URL, I'm sure, to this story. All you gotta do is find the third thing ever posted on the Internet (laughs) in the entire history of the world, because this was really really early. And we adopted that method in the discussion groups that Joel On Software implements. We did something even simpler which is that when a moderator deletes a post on our discussion group software, which is commercially available, it's the discussion groups from FogBugz. When a moderator deletes a post, the post is still shown as if it were not deleted to the person who posted it, and to all the IP addresses in his range.

Atwood: Oh, that's just sneaky. That's really sneaky.

Spolsky: Yeah, so people think that this is cruel, but the net result is that it's a very pragmatic thing. What happens is if you delete someone's post and say, "We're deleting your post" and they know about it, then they are guaranteed to be angry, and if they're angry then they're going to try to post. They're going to try to find a way around this. This is just human nature. When you try to block somebody they get angry and they try to go around. They don't get angry and leave, they get angry and they try to fight and so they try to take down the system. They're going to start attacking your system in various ways and try to look for ways around your blocking of them, and they'll find them because the Internet is pretty anonymous and it's almost always possible to get a different IP address or to get a different user account name or something like that.

Atwood: You know what this is like, what you described, really briefly? FriendFeed has this feature where you can do a fake follow. The whole friending process is kind of a social landmine because people want you to follow them and you don't have time to follow them so you don't follow them. So with this fake follow, what you can do is like "Oh, I'll follow you" but it's a fake follow. To them, it looks like you followed them, but on your end you're not following them at all. So what you're describing where you delete it for everyone in the system except for the person who created it, it's very much like a fake follow. For this one person, everything looks normal: "Oh, nothing has happened." But to the rest of the world, nothing is actually going on. So it's interesting that you arrived at that similar technique.

Spolsky: Well, let me tell you, the end result of the similar technique is that sometimes people know that you're doing this and they're angry. But 9 out of 10 times they just don't, and they think that their little troll post that they posted is being ignored. They really feel like it's being ignored. And they'll continue to post and when they see that they're not provoking people, because nobody else is seeing their posts, they'll give up and they'll move on somewhere else because it's just no fun. They're looking for a fight or a conflict because it's fun; it's like a game. Their type of game playing is not the type of game playing that everybody else in the community wants to play. If you give them what they want, they're going to hang around. If you deny them what they want, they're not. I used to see this on IRC all the time in the old days. I'm sure it's still like this, but on IRC, the idea is that you chat, you make friends. The channels I went to, you used to go out on dates. (laughs)

Atwood: Where were you hanging out, man? (laughs)

Spolsky: I think I got like 5 dates in one week at once on a certain channel which will remain nameless (laughs) because that would give it away on IRC. (laughs) You're hanging out to socialize. And then there's a group of people who log on and they're addicted, so they're spending 12 hours a day on IRC. And they log on and they see people that are there they don't like and they're kind of bored and they don't feel like chatting anymore. But they're addicted so they need to do something with the system. So they kind of try to raise the ante, they're like "Hmm, maybe I'll try to take over that other channel." The Melbourne people trying to take over the Sydney channel. And you soon discover that you have to find and exploit various bugs in the system to actually take over a channel, but there's almost always a way to do it. Or you can try to take over somebody else's nickname so that their friends can't find them, because you couldn't reserve a nickname in classic IRC. This is exploiting the system basically to play a different game than the game other people in the system want to play. You've got a bunch of people playing chess, but you want to play "Throw the chess pieces all over the park!" (laughs)

Atwood: (laughs) And that is fun, just throwing the chess pieces, what an enjoyable game.

Spolsky: And then it becomes a wrestling fighting game.

Atwood: (laughs) That's awesome!

Spolsky: It's just a different game that the other people don't want to play. So it's really really important not to fall into the trap of playing the game that the people want to play. So when somebody is being provocative or when they're being obnoxious or when they're trying to abuse the system in some way, for them, they want to have the little "I'm having a fight on the Internet" game, and none of us want have this game on StackOverflow. We'll have that game on your website, just not on ours. (laughs) And so it's really important to have the "Don't feed the trolls" method. You need the people to not be fed and there's always going to be somebody there who is going to have the adrenaline rush of "Oh my God, this guy's being obnoxious. How can I punish him? How can I fight him? How can I create some kind of fight? I can't go to sleep; somebody is wrong on the Internet." It's very very hard. A couple of years ago I got into a flamewar somewhere on Reddit. I've learned that there's no point to this. I can't even remember what it was about.

Atwood: Wow, a few years ago, that's shocking. I thought you would have learned that much earlier. Even Joel gets sucked in every now and then.

Spolsky: I wonder if anybody remembers this. I got into a fight with somebody on Reddit over something - I don't even remember what it was. I'm sure it was something that I wrote, and I don't remember. Obviously he was wrong. (laughs) But it didn't matter, I was like, "I cannot go to sleep, because I have to see how he responds to my last thing so I can respond to that."

Atwood: Yes.

Spolsky: Okay, so what's the philosophy? The practical result we've had from this FogBugz algorithm of making the person see their things but it disappears for everybody else has been very successful because 9 out of 10 people don't know this is happening and, all else being equal, that means you only have 10% as many disruptive users that you actually have to deal with.

Atwood: Right, so let me distill this down to a recommendation. This is all very interesting conversation and I think very useful but how do we turn this into actionable stuff we can do on the site? Because we still do have every now and then - it's not a major problem at the moment - but we do see users who create accounts for the explicit purpose of messing around.  And one thing, you talked about not engaging with them and maybe I'm not doing this correctly, let me give you an example one thing a user did. A user created an account and registered an OpenID and actually posted a post which was all giant images. This image of this 18th century French painter.  This very very funny image of this painter - it's a painting - he's gesturing at the viewer and pointing and laughing. It's a very funny image. And I actually went and looked this image up because I was so intrigued by it because it had sort of a name on it. So I looked this guy up on Wikipedia, I'll link it in the show notes. So I actually went in - it's awkward at the moment for us to delete users because there's a lot of cascading deletes that we have to do, and a lot of key relationships that have to be there, and we haven't honestly implemented this very well at the the moment, so I couldn't actually delete his account.  So I went in and changed his account name to this French guy's name because he posted that image and then he went in later and actually populated the profile with all this guy's information and created a Gravatar for him and everything. (laughs) It was actually kind of funny, but maybe I was engaging with the troll at that point like I was encouraging the behavior when I didn't really mean to.  So with these posts, these giant posts that he created, we of course went in and A: Edited them so they weren't giant anymore, and then B: Deleted them. But what would you would do in that case to this user?

 Spolsky: There's two ways we can look at it. Number one, even the deleting of posts may be unnecessary.  With our current data model, the posts are voted up and voted down and they can also be deleted. In fact the deleting is almost an extreme case of voting down.  If something had -124 points it should be as good as deleted.  for all intents and purposes, it doesn't come up in searches, maybe if there's only zero other things that might come up. In fact, in a lot of these systems that work on voting, the only reason they even have a delete capability that is given to moderators, is if somebody posts something that is actionable. Like it's a copyright violation and you get a DMCA notice, you have to actually take it down, you can't just bury it. So the first possibility is maybe you'll find that the system just works well enough. This is something, Jeff, I mentioned to you when we were chatting last night. If you look at the new submissions coming through on Reddit and Digg, it's 99.999% spam. I would not be surprised if it was that high because a lot of people set up automatic scripts. There are online PR agencies that have automatic scripts that submit every single press release about every single one of their clients to Reddit and Digg. And it's hopeless... 

Atwood: Wait, brief detour there.  They don't have a CAPTCHA around new item submission? Really?

Spolsky: You know what? It's worth it to these PR agencies. They're getting paid to do this. But it doesn't matter, it's still spam. Just because it costs a penny - and CAPTCHA is a myth anyway. CAPTCHA is easily broken, and it's especially easily broken now that there are these Mechanical Turk ways of breaking these - not that I've ever found one of these - but there are these porno websites that will give you a free account if you type in a CAPTCHA. There are mechanical ways to get people to do CAPTCHAs at very very low cost. Getting CAPTCHAs solved is so cheap that it's almost worth it for spammers. And it is in some cases.

Atwood: I don't agree with that, but I don't want to go on a tangent, so keep going.

Spolsky: Let's put it this way: CAPTCHA is not waterproof, but it does help. It's not waterproof and it probably won't be that helpful for long, but go look at the new page on Reddit or on Digg and you'll see an awful lot of crap coming through and an awful lot of people that have written scripts that resubmit things, or just submitting a lot of garbage, CAPTCHA or no CAPTCHA. And I think you could try to moderate that, but it's almost not worth it; the stuff just gets voted into oblivion very very quickly. And so the system itself does work. I don't think they need any kind of special protection on a website where they're only showing you things that have been voted up in most cases. The protection you need, then, is for people who have created fake shill accounts in order to get votes that they can use to vote things up. It's a little bit too complicated but the whole point of that is it may not be necessary to use this delete feature or the offensive feature because maybe the voting is good enough.

Atwood: Well, hold on, though. Because we have posts that, even though they are downvoted, are giant posts. They have images that are 999x999 times 100. Even editing them is awkward.

Spolsky: Yeah, okay. But it doesn't matter, you can vote them down. Oh, you have to go into them and then vote them down.

Atwood: But the alignment of the page is completely messed up. I think this is where the Digg comparison breaks down a little bit because Digg is just links. How are you going to grief somebody in 100 characters of descriptive text? We're more like Wikipedia in that you can grief somebody by creating a whole giant article, right?

Spolsky: Yeah.

Atwood: Now, the advantage for us is it's very easy to undo.  And this is something that's in the Clay Shirky book 'Here Comes Everybody' is that it's so easy to undo vandalism, that the effort required to undo it is an order of magnitude lower than it takes to do it, and that's how you win that particular battle. Because there's lots and lots of people willing to click Rollback or Edit and very very few people willing to build it.

Spolsky: So wait, I have an even better answer, though. That was almost an aside which is that I think that voting may get everything you want from deleting. But even if that wasn't true, let's take it as a given that we have abusive users, and we can maybe detect they're abusive because the number of things they've posted which have been called offensive is very high, or some other reason. Let's say that we knew that somebody was offensive. Well, you have two possibilities. One is you can ban them and the next time they try to log on you can say, "You, buddy, are banned!" Well, what are they going to do? They're going to go piss on you every way they can. They're going to make a new account, they're going to tell all their friends that you suck, they're going to go to the Crap Overflow site where they have their - what's the word? Support group?

Atwood: Community of outcasts.

Spolsky: Yeah. No, they're gonna get back in and just going to try to delete everything because they hate you now. You got in their face and you've caused an adrenaline rush, you've given them an adrenaline rush which is a fight or flight reaction, and they're gonna fight. Some of them will flight, but most of them will fight because this is on the Internet, right? What can they do, nobody's gonna come to my house. Although that helps. (laughs) You could go to their house. Second possibility, let them keep posting and silently sort down everything that they ever post. You make a new field in the database called "user quality" and everybody starts with a zero, and they get a -1.  Every time you're displaying something, the sort order is "user quality", "rank".  So now they're still there, they can see their post, there's nothing visible from the user interface that lets them even know in any way that they've been bad, it's just that they keep posting questions and nothing happens and they keep posting answers and they wind up at the bottom of the page for some reason, and nothing that they do really matters.

Atwood: It's essentially the same basic system, but don't make it visible that you're interfering with them in any way.  Kinda the solution you were talking about earlier.

Spolsky: Hey, I heard a trick that I've never seen used.  If you've ever been on a plane where a passenger has had about 18 drinks and is getting rowdy and the flight attendant refuses to give them another drink, or in a bar where the bartender says "I think you've had too many.  I'm cutting you off!", that aways turns into a fight.  There's always a guy with a lot of bravado, totally embarrassing his date, being like "I give so much money to this airline!  I can't believe you would be blah, blah, blah...".  And the guy's obviously drunk. There's a trick that good bartenders know.  The guy orders a drink, and you bring him a ginger ale or a water if he's drinking vodka, something that makes him look like he's still drinking so he doesn't lose face in front of his friends.  Then you very quietly say, so nobody else can hear, whisper to him, you say something like "Let me give you water for this one drink.  If you want another drink later, you can have one, but finish the water first because I don't want you to get a hangover."  You basically say something nice that's of the effect of...  You're not even announcing that you're not giving him a drink.  You're going through the motions. You're letting him save face in front of his friends. You're not announcing to the world that you're cutting him off in a way where he feels he needs to stand up, and you're also not even telling him that you're cutting him off.  You're just saying "Let's do water for this one drink. Next drink you can have a regular drink."  And hope that he will have forgotten by then.

Atwood: Right. There certainly are aspects of that I think are very true, like not being confrontational, and just sort of being more zen about the whole situation of, like, you're not going to escalate into anything weird

Spolsky: It's also called passive-aggressive (laughs) that's another way of putting it.

Atwood: Right. Also, like, I kind of disagree a little bit that you're supposed to disengage completely from people that are attacking. like I think, I guess it depends on the attacker. But I found that a lot of the people that are attacking are really just playing. And if you can engage with them, kind of like what I did with the guy where I changed his name to the picture he had used. He saw that I had actually taken an interest in what he was doing, and actually didn't view it as totally negative. Although I obviously deleted his post. I didn't delete his account or anything like that. Maybe there's a half-step way where you can essentially turn your detractors into usable members of the community. This is the paradox: if you're motivated enough to post a long blog entry about how StackOverflow sucks or do this griefing stuff, you're pretty engaged with the system, right? Most people don't care. That's the paradox of - that's why maybe, as Dave Winer said, they're usually your competitors. Because who else would have the time to work with something that they hate. What's the value to them? I think the answer is they don't really -- the whole -- the thin line between love and hate... there's something they like about your site. They're just not willing to acknowledge it, right? And if you can flip a bit there, maybe they can come back around or you can address what they're actually complaining about. I don't know, maybe that's possible.

Spolsky: Absolutely. Sometimes it is, and sometimes... one thing that I've found very helpful when being a moderator, or in a moderator role, or even when you just get an email that seems idiotic in some way, is: just pick somebody you love, and imagine they sent it. Like "imagine that this was an email from my dad". (laughs) Just visualize. Just have that visualization. And you say "Wait a minute". All of a sudden you say, OK, maybe he's making a joke and I didn't read it as a joke, I read it seriously, maybe he really doesn't know how X, Y and Z works, maybe he's really young, maybe it's a child. Because, see, that happens on the Internet and you don't know. Maybe it's not a native English speaker. And that's actually really common on the Internet, where things come across sounding a particular way just because the writer is not a native English speaker. And so in moderating, in trying to figure out how to engage those people in a way that's more, humane, I think, than we usually do, it's helpful just to imagine it's that nice old man that sells cigars on the street corner. And this is the person who's typing in this comment. What would be a nice way to engage with them? You know. If you have the energy. Otherwise just ban'em. Ban'em, I tell ya!

Atwood: (laughs) Yeah, it's tricky. And that's one reason we've taken a while to implement this feature. Because I don't know entirely how to do it right. And I agree there's a lot of pitfalls here. There's a lot of ways you can turn people against you. And that's absolutely not what I want to do.

Spolsky: There's one more thing we might want to consider doing. Sort of like the Amazon RealNames. Google did it in a reasonable way. I don't know if they still do this, where you've got to give us a phone number of a cell phone, and we're going to call it. And then we're going to read you a phone number and you're going to type it into the web site, and then you get this special badge on your account that says that you are a little bit more unique, because there's just a finite number of cell phones that any individual can control. So at least we know it's not a throwaway account where somebody is generating accounts for the purpose of spamming and then they're going to disappear.

Atwood: Well, we have reputation. That achieves it for us, right? I mean, once you get to a certain reputation level, it would just -- there's no way...

Spolsky: No, but somebody could manufacture ten accounts, and have his nine accounts vote up his one stupid thing just to generate reputation on that account.

Atwood: Not enough to get to... It depends on how you set the level. But I think our reputation system does kind of work in that regard. If you set the threshold to a thousand, there's no way. It would take too long.

Spolsky: It's working well enough. But what if you said that you can buy 300 reputation by giving us your real identity, by paying us a dollar with a credit card?

Atwood: Hmm.

Spolsky: And a dollar just covers the cost of -- literally -- processing the credit card. Because... believe me, that's about all it does. And it guarantees us that you are a human being, at a given address, with a certain name and... it vastly reduces the amount of... I don't think we're anywhere close to needing to do this, but those -- the cell phone and credit card -- are two systems that some people have used when they really need to kind of raise the stakes, and solve the problem of garbage, generated fake accounts that are being used to abuse the system in some very bad way. Like you're getting thousands of posts a day from IP addresses from all over the world, and you just don't know how to prevent them.

Atwood: Right. And to be clear, there's only been really isolated incidents. This is not some large-scale problem. I just like to think it through, because we want to go public Monday, that's the current date. I just want to make sure we have some infrastructure in place for dealing with users that are just really basically -- I don't want to say evil, because it's probably not the right word, but -- really attacking. And I want to have some strategy. Because right now we have basically no strategy. I don't think that's a good idea.

Spolsky: You know what? Every one of these sites that I know has basically started out with no strategy -- including PayPal -- started out with no strategy, and then page faulted in a strategy. You've got to wait and see how people attack, and what particular problems actually exist, and then you have to build specific defenses against the specific problems that you are specifically having. Because otherwise there's just too much work to build against every eventuality of every possible kind of person.

Atwood: Right. And you bring up a really excellent point, which is, my philosophy has been, during StackOverflow, was to try to build up based on the behavior I was seeing. Because you're right: otherwise it's the classic programmer thing of building these grandiose predictive systems that never really come to pass, like I do try to build features that target specific behavior I'm seeing versus what I imagine the behavior is going to be. So that's also the trick. There hasn't been really enough of the griefing to really have a coherent strategy yet. I'm sure that will change once we go public. Hey, before we end the call, do we have any questions?

Spolsky: We do, and what's funny is that a lot of the questions that are queued up are actually directly -- we actually talked about them. Just coincidentally. Because a lot of the questions are about things like reputation and karma. Let me play one from Clay...

Clay Nichols: Hi Joel and Jeff, this is Clay Nichols, a long time listener...

Spolsky: By the way, let me... Clay Nichols has this little company called Bungalow Software, and they've been in business for about, I'd say 10-15 years. They make speech therapy software for people who have recovered from a stroke or aphasia or something like that, so it's nice that he's actually been kind of a software company success story doing something that is so helpful to so many people. So, back to Clay:

Clay Nichols: ... first time caller. I was a bit dubious about the idea of the badges. I figured, "we don't need no steenking badges", however I think they're working. It's like the ribbons we wore in the Navy, they let you really quickly distinguish someone's experience, tell a difference between a nuclear power technician who got an automatic promotion to third class petty officer with no experience, and somebody who'd been busted back in rank five times, maybe didn't have a lot of rank, but a whole lot of experience. So here's my question: did the idea for badges come from Joel's experience in the Israeli military, or from the flair on Jennifer Aniston's uniform in Office Space? Thanks!

Spolsky: Office Space!

Atwood: (chuckles) I'm glad he finds them useful. And again, because I've said this many times, everything that is in StackOverflow is in there for a very specific reason. There's nothing that's in there just for fun. Everything that's in there is to drive a certain kind of behavior. Now, any fun that you have is certainly welcome, and that's a goal as well, but the primary goal is driving good behaviors. And that's why, a lot of times when people propose badges, I have to say to them: you know, that's funny as a badge, but it doesn't really drive the kind of behavior I want. So I can't do that. So I'm glad that other people are seeing the badges and actually getting use out of them. But really, it's neither of the above. The badges were based on the Xbox 360 achievements model, which has a very very similar sort of structure. And as I've related on previous calls, you go in thinking, "Oh, these are meaningless achievements, who would bother doing this", and you get totally sucked in. And not only do you get sucked in by the achievements, you realize the achievements are actually making you enjoy the game more. On so many levels. Other people can see them, they're public, you explore parts of the game you wouldn't normally have seen, you have more fun. They work on so many levels. And I'm very pleased with them.

Spolsky: Well, Clay, having been in the military, I have to admit that my answer is "the first". Because I did see actually -- the Israeli army is not big on badges, does not have a lot of medals that you can earn. But there are things you can earn, and in a normal three-year military service that you do, you're going to get three or four of them, and they're very meaningful to people. It's very surprising how something like that is meaningful to people. Long after I got out of the military, I was in college reading history and I read a quote by Napoleon Bonaparte, he was famous for saying "A soldier will fight long and hard for a bit of colored ribbon". And I think until you've done that, (laughs) fought long and hard for the bit of colored ribbon, it just sounds silly. But there is actually a lot of truth to that. If you create a system in which the little bits of ribbon are reflective of respect that should have been earned, then people will respect that system, and they will fight for the ribbons. Great example: when I started working at Microsoft, it was back in the days of shrinkwrap software, so everybody had a box of shrinkwrap software for their products, and when they shipped a product, they would get a copy of the box with the shrinkwrap and pass it out to everybody on the team, and they would stick it up on their bookshelf. And so, as you walked down the hallways at Microsoft, seeing all these people and you didn't know who they were, some of them had one product up on their shelf, and some of them had like ten. And some of them had these really exotic and interesting products, and some had these ancient products, you were like "Wow, that is the shrinkwrap box of Microsoft FORTRAN 1.0". (laughs) Something ancient, before Microsoft -- I think the oldest thing I ever saw... I'm trying to think but... I think I met somebody that had... what was that game, Adventure? It was like an Apple II Game, I think?

Atwood: Oh, yeah, yeah. Microsoft made a lot of their early games, that's true!

Spolsky: So you see that on somebody's shelf, and you're like "Whoa, that's an old-timer". And otherwise you're walking around in this company with 5000 programmers, you have no idea who's who and who's done anything and what the hell they've done in their life, and why you should listen to them, or whether you should listen to them. But on their shelf there, some of them have like ten things and some of them have one thing, and some of them have nothing. And that was really meaningful to people. And they really cherished getting those boxes. To the point at which, when they stopped actually making shrinkwrap boxes, I think people were a little bit jittery. They didn't know what to put up there. And it's funny, they hired an HR manager who had been working, I think at Apple. To be head of HR for Microsoft. And he knew the concept, but just did not know the company. And so he came up with this idea called the "ShipIt Award" that you got for shipping software. And what it was was you got this slab of lucite -- if you didn't already have one -- and they would stick these little bronze stamps on the slab of lucite every time you shipped some software. And the Microsoft community hated these. They loathed them. There's a scene in that book, Microserfs, where they're trying to destroy one with a blowtorch, and that is not an exaggeration. Because there already was a system by which people got respect for what they had worked on. And the senior developers at Microsoft, you'd go into their office and they had 18 interesting, famous applications that had made a lot of money, up on their shelf. And then they got this slab of lucite that had one little stamp on it. Because all it had was what they shipped last year, because the lucite thing was invented much later. And so it was this quasi-attempt to make a badge system that actually did not reflect what you had done. It did not show any knowledge or respect of an existing Microsoft culture. And it really teed people off. It really really pissed people off. Anyway, that's my little story. I think that it is kind of surprising, how much this matters. And it's really about respect and credibility, and when you get to a large enough community where you don't actually know everybody -- it's not just the family, and we don't know your history, it's not just your classroom, and "maybe you've been with these kids and you know who's who" -- maybe it's a big company with 5000 people, or it's the Internet, with a billion people. Or StackOverflow, with thousands and thousands of people you've never seen before, and we need to have a way of showing you who's who and what they've accomplished and how long they've been here. People will make it if you don't give it to them. On Slashdot everybody talks about how many digits are in there Slashdot user ID number. Because it's a sequential integer. But that's something that is kind of hard to earn, that kind of credibility, and Slashdot was never really explicit about giving you very many kind of badges or anything visible.

Atwood: Right. Well, we're probably at the hour.

Spolsky: We are.

Atwood: I feel bad we didn't get to too many questions. But you said we... maybe I'll get the names from you so I can list the questions we actually did answer in a sort of drive-by way.

Spolsky: Yeah. I don't really know what they are. Some of them were more specific. We're starting to get a lot of pretty detailed questions about, like "How should this particular thing in karma work" for example. I don't want to go into it today. But somebody was asking, and they came up and asked me this at the Business of Software Conference as well, if you edit your own post a bunch of times, why does it flip into community mode? Because it's just you, and it's just your post, it should be yours. So you know, these are like-

Atwood: There is a reason.

Spolsky: Oh. Okay. You better... (laughs) You've got to just say it.

Atwood: Well, the reason is, there's certain users that were using this as a way to continually bump posts...

Spolsky: Oh! They were doing it so their post would come up as new.

Atwood: Yeah... or in other subtle ways, like they would edit posts where they had a really high-ranked answer, or they were editing someone else's post...

Spolsky: OK. Got it. These are relatively small details. We're going live to the entire universe next week, and we're either going to have a deluge of newbies that are going to mess up the whole thing and it will be a disaster, or we're going to have great new behavior caused by the fact that suddenly Google can find answers, and now you can use Google. I don't know how long that will take, but now you'll be able to use Google to type questions and start getting results on StackOverflow, and that will actually bring -- I think what that will do is that will bring a lot of attention to the very very very niche questions that right now just don't have any community finding them, because there's like 10-20 people working on this technology in the universe. So I think it's really valuable for us to get live, and I think we'll start to see some different behavior when that happens. And we may want to re-evaluate what we want to do. So I think the theme of this entire show has been about the anthropology of writing software. How when you're creating social software in any way, you have to think like an anthropologist. You can't think like a software developer any more. It's all about how do people work in teams and what's...

Atwood: Yeah. I like to view it as just encouraging what you like and discouraging, without punishings -- that's the fine line we talked about -- behaviors that you don't like. Because there are already uses of the system that I don't consider valid. But I'm not going to like go in and step on it. Because people want to use it in that way. That's fine. As long as they're not hurting anything -- as long as they stay on topic, broadly, in terms of just programming in some way, I'm fine with that, so... it's a very fine line. It's very much a balancing act. You don't want to turn the crowd against you. Because I think that's the whole point of this. Is to have a crowd that sort of gets behind what you're doing, and enjoys building the system. So the minute you turn it into an us-vs-them, I think you're lost. Like the whole Friendster debacle from early on. But yeah, it's tricky. And we're trying to be very careful about how we do it, we really are. So, any questions like that, please send them in.

Spolsky: Let me just write down that mispronunciation, so I....

Atwood: (laughs) What did I mispronounce?

Spolsky: No no, I'm not going to say it on the air, I'll tell you about it afterwards. (laughs) Just taking notes. Okay, we got some announcements. I think we sort of let it slip that we're planning now to go live early next week, probably Monday. Fog Creek is going to be in chaos, because we're going to be in boxes, so there's actually a chance that you may push us live and I won't really announce it on Joel on Software until I can somehow get my computer to come back to life. But we'll try. What other... we had some other announcements. Oh yes, I have something I want to mention. We are, I believe, looking for a sponsor for this podcast. And it's very much like a NPR style sponsorship, those of you that have been listening here on IT Conversations, they're spending a lot of money, putting a lot of effort into this, and they actually wanted to line up audible.com which makes those audio books that are all DRM'd and yucky and if you change computers they get deleted, and they wanted to line them up as a sponsor for this show, but the requirement was that we had to interrupt the show you and I in our own words had to talk about how great Audible is. So I basically... Doug and I talked about that and we completely vetoed that idea. Big time. Although they would have been happy to provide a significant sponsorship. So if you are any company that makes products for developers, or that just generally wants to help out with this podcast by underwriting it in exchange for somebody quietly saying "this show is sponsored by suchandsuch company" at the beginning and at the end, and I think... you know, Doug said, what did he say... did you see that email?

Atwood: Probably.

Spolsky: By the end of the month, 30,000-40,000 people have listened to every show. It's actually more than I thought. So a bunch of people will hear it, and it's a good way to use your company's name, or get good reputation. In a way. If you're interested, or able to sponsor this weekly podcast, go ahead and send me email at joel@joelonsoftware.com. It's a different email address than the one we use for questions for the show, which should be sent to podcast@stackoverflow.com. And if you send in questions for the show, we want to be able to play them and let people listen to them, so send them in MP3 format, or WAV or Ogg Vorbis are all good formats, so that we can play the audio version on the air. There's something called Blog Talk Radio that you can use if you don't have a microphone on your computer and you want to record audio, it's a phone number you can call. And it puts an MP3 up on the web for you. And we have a wiki for the people that want to either be able to search the archives of the show without listening to the whole thing, or they just want to read it. And the transcript wiki is also located at stackoverflow.fogbugz.com and there will be a link in the show notes, and we always get a bunch of people going in there, whom we thank, and just transcribing a minute, two minutes, three minutes of the show, which has been very valuable and very helpful to people. Usually the whole thing is transcribed in a day or two. It's just awesome. And that's been very helpful to a bunch of people who can't listen to the audio version for whatever reason.

Atwood: I actually have had those quoted back at me. So you can hold me to my previous claims if you create a transcript of what I said, so there is value there.

Spolsky: So that's a very nice volunteer effort that follows. That's all we have, so we'll see you next week.

Atwood: See you next week.

Outro, ads.