View  Info
Compare   Restore  
   Revision #18 - 10/29/2008 6:54 PM     

Podcast 026


Intro, ads


Atwood: ...Joel.

Spolsky: We're back.  With another fabulous episode of: StackOverflow Podcast.  Brought to you by...

Atwood: That's right.

Spolsky: Jeff and Joel.

Atwood: Do we have a new sponsor?  I didn't know if there was some surprise like the Gummy Bears for extra grubby children.

Spolsky: StackOverflow Podcast is brought to you by Gummy Bears for grubby children.  [Laughs] I can't say that with a straight face.  We need to get a better sponsor.

Atwood: That's right.  We had talked about doing a lot of questions 'cause the last few shows we haven't done any at all.

Spolsky: That's true.  We have five.

Atwood: OK, why don't we start with that then?

Spolsky: You just want to launch into the questions?

Atwood: Yeah, I think so. I think we owe it to the audience.

Spolsky: No like; "how have you been? What's new? How's your 401K doing?"

Atwood: Nope, just right into the content, man. That's how we roll.

Spolsky: Alright, well most of our questions here are about StackOverflow as a matter of fact. So let's just start throwin' them out there.


Christopher Leary: Hey Jeff and Joel, I've been really interested in the development of StackOverflow's Karma system since I've been trying to come up with one of my own. You guys seem to assume that there exists a mapping of actions to Karmic weights that will steer the whole userbase in the right direction in a way that's also going to scale.  I've read the transcript of Shirky's "A group is it's own worst enemy" talk, which made me wonder if this is even possible?  Do you really think that a theoretical greatest possible weighting system would be good enough to deter the pathological users and encourage everybody else?

Spolsky: Yes.

That's the whole question.  Go ahead Jeff.

Atwood: I think the system we came up with - I mean I don't pretend to have an idea of what the end state is for our system.  I actually don't know, I mean because there's going to be continued growth in reputation.

Spolsky: Yeah.  It might have to change.

Atwood: I think one thing we changed - one thing we did early on that I think was hugely important and we should have done it earlier - was put a cap on how much reputation you can get in any given day.  And I was just looking at a request on UserVoice that someone thinks we should also have a cap on the amount of reputation you can get from any one response.  Jon Skeet was noting that he felt it was a little unfair that he got 46 up votes on a really simple answer to, I think it was a question about whether you should do loops like "<" or "<=".  It was like a really simple little thing but he just felt like 46 up votes was a lot for sort of an ephemeral thought like that.


Spolsky: Was that a question that -- why did that question have so much attention on it?

Atwood: I don't know. 

Spolsky: Why were many people liking that question and voting it up?

Atwood: Well, I think the questions that anybody can relate to tend to get a lot of votes.  So if you ask some really narrow technical question only a very narrow technical subset of the audience is going to be able to understand it and read it and process it and vote on it.  But if you ask a really generic question, like say, "What's your favorite programming cartoon?", everybody in the world can relate to that.  And to a lessor extent "How do you build loop structures?".  Everybody builds loops.  It's all we do, right?  So everybody's got an opinion about it.  So a lot of people are going to have eyeballs on it and have input into it.

Spolsky: [grunts encouragement]

Atwood: And Jon I thought had a very nice considered opinion about this.  And he's onto something because, it's like I said; until we have the reputation cap it was sort of a huge looming problem.  So I would say that's the number one thing I would look at.  That is you have to have really strong limits in the system. 


Spolsky: Maybe they should be logarithmic or something like Richter scales?

Atwood: Ya, I thought about that but it just it gets... hairy.  It just seems complex at some level versus the one man one vote rule.  I mean there's some people who think that the more reputation you have the more your votes should count which I think is extremely bad, like I think that's totally the wrong way to do the system.

Spolsky: Wait wait don't answer that yet because that's a question we have coming up.

Atwood: Oh really?

Spolsky: We'll come back to that.

Atwood: Yes, We'll come back to that.

Atwood: But I would say in building reputation type systems, the other key thing we looked at - so the the central tenets:
1. Have some sort of built-in limits.
2. Try to make it so that other people can give you reputation.  It's like the whole Google model of PageRank where you building a website: you can't vote for yourself only other people can vote for you.  I think that's really important as well.

So I would say those would say those would be the two guidelines for anyone asking my opinion on these things.  Although I guess we have not seen the ramifications of the system that we have built yet.  But so far, so good.

Spolsky: I think it would be fair to say that it was definitely our goal that this thing - or it was our expectation - that these things would change all the time.  That we would constantly be changing things.  Adding new badges, removing badges, adding scoring rules. Right?

Atwood: But as you have noted before, once the system gets out there and people get used to the way it works there's a lot of bellyaching when you change things, there's some inertia that you start to get in the system around the way things are so it's difficult to just radically change course.  Our hope was that during the private beta we got all that stuff out of our system - all the major, crazy changes.  But we'll see.

Spolsky: Oh I got a cowbell.  [ding ding ding]


Atwood: Oh. [laughs]  Did the cowbell arrive?  Did the tambourine arrive?

Spolsky: Yeah, oh tambourine, I'll take that out for the next question but [ding ding ding] that's the cowbell.

Atwood: By the way, these aren't random.  These are actually Rockband accessories for the vocalist.  They're actually part of the song where you can hit the cowbell and actually have it register.

Spolsky: I was looking for the USB port and this just looks like a piece of metal.  I was not seeing... I thought this was going to be a USB cowbell.  I guess it's wireless or something?  I don't see where you put in the batteries.

Atwood: I'm excited that you are using it as a prop on our show though.  That makes it all worthwile.  It's excellent.

Spolsky: I've got all this gear here now, for those that don't pay any attention to my website, I wrote up a little article last Friday about what all the gear is here in the new studio setup.  I've actually got two mixers, and a recorder, and all kinds of USB... It's a lot of gear and the joke is that I think this entire thing could be done in software if I could just figure that stuff out.

Atwood: [laughs] Yeah, but it's fun to have knobs and dials.

Spolsky: The knobs and dials are fun and there are flashing lights here that don't mean anything.  For example, one of these mixers has a flashing light that says 152.2 and I don't know what that means.  I think that's the number of beats per minute coming from you Jeff.

Atwood: But it looks cool.  It looks complicated and interesting.

Spolsky: It does. I personally believe that almost anything you try to come up with will over time tend to create people gaming the system a little bit and maybe non-optimal behavior that is "point optimal" but not "site optimal" like it optimizes for somebody's personal karma not for the actual outcome that you want.  And as that starts to happen you have to constantly adjust and tweak these systems, kind of endlessly.

Atwood: Right.

Spolsky: That's what happened with PageRank obviously and Google's ranking algorithm.  They pretty much had to keep continuously adding new twists.

Spolsky: What does the knob do on the cowbell? [ding]

Atwood: But the central thing is still there.  I mean, I agree that there's been all these new--It's funny because blogs were kind of an exploit of the PageRank system in my opinion.  Blogs, just the way they are constructed, and the way people form these communities and link to each other, just naturally--they're not trying to game the system.  But it was a huge PageRank farm, effectively.  I know Google did several updates related to blogs because it got ridiculous.  I remember some bloggers had PageRank 7, which is astronomical.  It's a logarithmic scale right?  On one end you have and some really major sites like some of the Google sites are PageRank 10.  Wikipedia, things like that, and then PageRank 1.  By the time you get to 4 and 5, you're a pretty serious website and I was surprised to see bloggers at 7.  I love my fellow bloggers, but good Lord.  They're not institutions unto themselves, although some of them might think that they are.

Spolsky: Well, I for one have PageRank 7, so there!

Atwood: But you're not a blogger.

Spolsky: Oh yeah, that's right.  Sorry.

Atwood: You always tell people over and over again, so you don't count.  You're a writer!

Spolsky: I notice you only have 6 Jeff. [laughs]

Atwood: You're one of those fancy writers, with your words and your sentences and your paragraphs.  I just use bullet points and pictures.

Spolsky: Maybe I don't mind [not] being a blogger anymore.  I was thinking about the definition of a blog and how it's sort of changed a little bit.  And it used to be more about the fact that you were linking.  There used to be some implication that if you're a blogger that what you're doing is like Glen Reynolds, where everything you write is two sentences and it's just linking to other people writing longer form things.  There used to be a little implication of that in the idea of blogging, now there isn't so much.  I think now there's all kinds of different ways that [??] work, I think that's no longer--the word either has no  meaning or it only means the reverse chronological order thing.  Hey, let's not talk about blogging, that's boring.  Let's talk about podcasting!

Atwood: Yeah, that's much more exciting. [laughs]

Spolsky: That's what people talk about on podcasts.  So anything else we want to tell Chris about Karma systems?  I think you do have to adjust all the time, I think you have to come up with the first good guess and you just have to be ready to tweak it and watch what people are doing, what people are saying.  Part of the reason I even thought it was important to have the Stack Overflow podcast is that I really did want people--this is something the community will pay attention to.  They're going to say "Hey, I have an idea.  Maybe I should get 6 points when I do this, and only -2 points when I do that."  Having those conversations continuously is important for us to hear from the users of the site.

Atwood: It is.  It's important for them to go on User Voice and enter it, and then I'll decline it immediately.

Spolsky: Or just, even better, call into the podcast hotline.  By the way, I've mentioned the phone number a couple of times because I want to encourage more calls.  646-826-3879.  That's 646-826-3879, and that's the number where you can leave a voice mail for us to play on the show and we need more questions.  When you do call, please tell us your name and try to keep it under 90 seconds.  All right.  Another question?

Atwood: Sure, let's do it.


Unknown Caller: Hey Jeff and Joel--downvoting.  Basically not all downvotes are created equal.  People who go around just downvoting everybody, their voice shouldn't really have as much weight as somebody who is very selective.  What do you think about a system where the voter's reputation is a multiple--so people who have better reputation, their votes count more.  What do you think?  Bye.

Spolsky: So I guess you were very much against this idea?

Atwood: I think it's very, for lack of a better word, undemocratic.  I mean, the idea that your vote counts for more than mine?  There was a gem of something in there that's a little bit interesting that we've thought about which is, we have had incidents where people who--they get pissed off at another user so they go in and downvote everything that user has ever done.  Right?  First of all, that's not as bad as it sounds because an upvote is worth five times as much as a downvote, so it would take five downvotes to cancel out one upvote so there's a limited amount of damage you can do.  Plus you only have 30 votes in a 24 hour period anyway, so you can only do that on 30 posts by that user.  But still, it is aggravating and I totally empathize with people that are complaining because this is annoying.  Somebody told me, it might have been you--I don't remember, Mr. Reddit fan, but on Reddit they track this and if they see a user consistently downvoting another user, they basically silently undo all those votes, like those don't even count.

Spolsky: I do not remember saying that, but it just occurred to me that we should get--you know who we should get on as a guest?  Some of those Reddit guys like Steve or Alexis.

Atwood: Your bestest pals.

Spolsky: Alexis.  He's in a band called Breadpig.  Are you familiar with that band?

Atwood: I am not familiar with Breadpig.

Spolsky: They play the Rockband instruments without the X-Box.

Atwood: Wow, that's cool.  That's very cool.  Yeah, definitely let's get him on.  That's doubly awesome now.  I'm totally convinced.  Joel and I have been talking more about getting more guests on because I think it just livens up the show and plus we get to broaden our horizons a little bit which always kinda nice.

Spolsky: Yeah and we get better ratings when it's just not this old couple, you and me.

Atwood: [laughs] Whiling away the hours, sipping on our Bartles & James on the porch.

Spolsky: [laughs] That's what it's going to be eventually, isn't it?

Atwood: Yes.

Spolsky: We just need to find a place where we can both move.

Atwood: Yeah.  One final thing on that question.  Another thing that has been floated, and I guess you've kinda talked about this is just sort of a logarithmic thing.  Instead of something having a bajillion votes, where another post has 4, some sort of normalization you do where your deviation's above the mean--and I'm kinda open to that but again I think it's kinda complicated to figure out versus the whole "one man, one vote" thing, which people kinda get.

Spolsky: What I'm wondering is, what's it going to be like when there are people out there with 100,000 karma?  Are new people going to come in and say "Well, I can never achieve that kind of karma, so there's just no point" and lose interest?

Atwood: Another thing that's been floated, and I think some communities do this--once you get to a certain level, you just have "Max."  That's the number--you're at "Max" and then--we could cap it that way.  So it doesn't feel like oh--because someone will eventually have 20k rep. which is just huge.  We're almost to the point where someone has 10k.  I don't know, maybe we should have a cap?

Spolsky: How about nobody can have more than Joel? [laughs]

Atwood: [laughs] Because the intent of the system is that we trust you.  Reputation doesn't mean that you're a genius or that you know everything there is to know about anything at all actually.  It just means that Stack Overflow, the system, trusts you.  Once you get to a very high level of trust, you're just infinitely trusted.

Spolsky: I don't really like the "Max" idea.  I feel like you should always have something more to strive for.

Atwood: Yeah. No, I--

Spolsky: "Max" is like the President of the United States of America.

Atwood: [laughs]  Well, we've also thought about maybe extending some moderator priviledges to other users just so that if we're not around--the four moderators we have--although to be fair, to be clear, almost everything in the system, users can do the essential things that need to happen on the site.  But there are some unusual things that might happen.

Spolsky: I've never really used my moderator priviledges, except for once and then I messed it up because I thought I was just flagging something and I was actually banning a user for life, [laughing] I think--I don't remember.  I'm not really moderating properly.

Atwood: That user felt very chastized by that, accidentally.  He actually emailed me, that was kinda funny.  What Joel is referring to is, normally you can vote something "offensive" or flag something "offensive," and once you get to a certain threshold of offensive, it's soft automatically deleted and so on, and so forth.  What Joel didn't realize is as a moderator, your offensive vote is binding and permanent.  If you cast an offensive vote, immediately you reach the threshold--one of the very few special things about moderators.  So when Joel did that, he banished the post.  He thought he was just casting an innocent little flag, but, no, he was removing the post.

Spolsky: I have a question.

Atwood: Are you in the list of questioners?  Did you call the number?  Did you leave your name?  Did you--

Spolsky: Sorry, I won't ask it.  Let's see... I kinda got a new system here so I'm--repuation, downvoting--I feel like you should be able to go on forever just earning some kind of additional points but maybe not in a way that discourages new users from--maybe there's something about--maybe your karma kinda rots if you don't keep it up to date.

Atwood: That's another thing that's been floated, but I don't know.  Gosh.  Some of this stuff--I feel like you'd just have to throw it out there and try it and see what the reation is.  But some of it is so broad and so sweeping, it's such a major change to behavior.  Gosh, I worry about that, particularly that one.  If people come in--because already people freak out if their reputation changes by any way that they don't understand.  So if there's this global thing where every week everybody's rep drops, unless they participated--let's say you go a week without doing something so your rep would drop, well that's like--

Spolsky: Well, maybe you have two kinds of rep.  You know, you've got "old" rep, which is shown in Roman numerals and you always get to keep that.  [laughs] But it's old fashioned which is why it's in Roman numerals.

Atwood: [sarcastic] Yes.  That's great, that sounds simple and very easy to understand, the system that you're describing.  I don't think anyone would be confused.  I don't think I would get any emails about that.

Spolsky: [laughs] You'd just have to explain it in the blog.  Let's go on to another question, here--this is a good one.


{to be transcribed}

Atwood: Edit me!

Spolsky: Edit me!


Spolsky: You know what we could have badges for?  We could have badges for like: "Real Address".  Like I have proven that you live at a certain address.

Atwood: We could.

Spolsky: And then certain things could only be turned on if we can actually go to your house.  Not that we will.  But we could.

Atwood: We could.

Spolsky: And then you just use one of these--there are these online services that will mail a postcard to an address with a, you know, with a code on it, and if they type it in correctly then you register that they at least control the mail that goes to that address.


{to be transcribed}

Atwood: Edit me!

Spolsky: Edit me!


Atwood: Let's do the trail-out.  You want to say the number again?

Spolsky: What's the trail out?  If you have any questions for the StackOverflow podcast, we'd love to hear them.  Why don't you give us a call?  The number is 646-826-3879.  Don't forget to mention your name and where you're calling from and try to keep it to under ninety seconds, or you can record an audio file in MP3 or Ogg Vorbis format and email that to  We've got a transcript wiki which is getting a little bit behind so I'd like to encourage you, if you'd like to do a service for the hearing impaired it would be really nice if you could go to and click on the wiki link there for every podcast episode and edit that with the transcript of what we're saying here.  And finally, I think I should have one more thing there, but I ran out of things to say.  So we'll see you next week!

Atwood: See you next week.


Outro, Advertising

[52:31 ends]
Last Modified: 11/12/2008 1:58 AM

You can subscribe to this wiki article using an RSS feed reader.