Ads, introduction

(Intro Music)

Hi, this is Phil Windley the excecutive producer of IT Conversations. Today I'm excited to bring you another episode of StackOverflow with Jeff Atwood and Joel Spolsky, here on IT Conversations. 

[...]

[1:00]

Atwood: So, we are in a call now with the development team for StackOverflow. That means with me, Jeff Atwood, Jarrod Dixon, and Geoff Dalgas. So this is Podcast, I believe number 17. It technically should be 18, but Joel Spolsky is on a jet-setting international vacation all around the world, so he is not available. When I posted that we wouldn't have two podcasts for two weeks, one of the suggestions we got in the comments was, "How about a developer podcast?"  I thought that was a great idea, because I remember when Jarrod was visiting with me in California, I talked about actually having Jarrod on one of the podcast calls but the timing didn't work out.  But I think it's exciting to have a call with the people who are really doing the work, because we know Joel doesn't really do anything, right?

I'm only kidding: Joel does a lot of stuff.  But these are literally the people building the site so that's why it's exciting to have an opportunity to give them a voice.  You guys can see what we're doing and how we're doing it.  So I thought we would start by introducing ourselves, everyone knows me, probably, so I'm not gonna bother introducing myself, so Geoff, do you wanna take it away and introduce yourself?

Dalgas : Sure, yeah, you bet.  My name is Geoff Dalgas and I've been working with Jeff Atwood on StackOverflow. I was one of the late members that was added to the team.  I've been happy, it's been great, it's been a lot of fun.  I got to know Jeff back in Denver, Colorado working with him through a small software development company there. And ah got to sit through a lot of his beratement [sic] and as a result of it, I think I'm a better developer, I'm not really sure though, so we'll find out when the bugs come my way.

Atwood : So Geoff, you got a lot of the rants, that I think would become blog entries later in my life, so that's...

Dalgas : Yes, before blogging was hip, I got to hear the majority of the rants as in, "Don't touch my monitor!" and a few of those. [laughing]

Atwood : Yes, exactly.  That was Denver and I think the time we worked together was, I wanna say '96... yeah, '96 was when I first met you and we worked together till like gosh '98 or '99?  It was a couple years.  It was a couple good years, but it was a lot of fun.

Dalgas : Long years, long years. [laughing]

Atwood : Thats right. And how about you, Jarrod?  You want to introduce yourself to the group?

Dixon: Yeah, my name is Jarrod Dixon and I've been working with Jeff now on Stack Overflow I guess the past four months now. I got to know him back in Raleigh, North Carolina. We worked together for a large pharmaceutical organization that just loved overcharging people I guess.  No not really.  But I guess I've been a developer for about  seven years, and really enjoy the opportunity that Jeff has given me to work with him.

[4:00]

Atwood: Now both of you guys, I think - and I've written about this in my blog before - you guys kind of grew up with computers like I did.  Because I remember, Jared, you telling me that one book, what was it - Computer Monsters?  That you used to read and type in programs.  At what age would you say you guys started getting into computers?  Like Jarrod, what would you say?

Dixon: Oh man. I guess my parents, they got an old Apple II+ when I was three or four years old, so I was messing around from that early of an age, you know with BASIC and stuff.  Just messing, playing around.  It wasn't like--I look back at how ignorant I  was--

Atwood: --Well everybody--

Dixon: --It was a real young age.

Atwood: Yeah, and what about you, Geoff?

Dalgas: I remember quite well.  Commodore 64.  And a good friend of mine, Ethan, had one.  We didn't have one; we couldn't afford one as a young chap.  I still barely can afford one now.  But we used to sit up nights, pounding in the BASIC programming, just right out of those magazines like you're referring to, and trying to get it to compile.  Trying to get it to run.  We'd get it going for a while and hit a bug, or hit a breakpoint, and spend the next 12 hours trying to figure it out so.  [laughs]

Atwood: Right.  And when I met you guys, I think I was drawn to you guys because we had that shared background.  We both, at an early age, had been really fascinated by computers and programming, and I feel like programmers who started out that way can sort of recognize each other.  I don't know what it is; there's some innate love of the craft that you have.  There's a lot of different kinds of good programmers.  I don't mean to say anything negative about people who don't do it this way, but I think we can recognize each other. [laughs]  And you have that common bond, which is kind of fun.

[6:00]
Atwood: So let's talk a little bit about the logistics, of where we are in beta.  So we've been in beta since last Thursday, so it hasn't.... has it even been a week?  It's been over a week now. 

Dalgas: Hah!

Atwood: It feels like months now right?

Dixon: It does

Atwood: But the beta is going really well, I was very pleased.  I didn't know what to expect honestly.  And I think when we came into private beta I think I was a little scared, just because you have been working on this thing for a while and you want to show it to people, but on the other hand it's like, its like your baby and you feel like people are going to abuse it and do means things to it, say mean things to you: all that stuff.  But the response have been great!  And by that I mean when people are exposed to the site - and maybe it's just as Geoff said he calls this the "honeymoon phase", when we are drawing in people that were interested in the project from an early date - but I feel like they are using it the way I envisioned people would use it, the way I wanted people to use it.  Which to me is very gratifying.  In other words:

• People are asking good questions (for the most part);
  
• People are getting good answers, like really quickly;
• The community part of it is working very well, in that the reputation system is mostly doing what it's supposed to be doing;
  
• The badges system is doing what it's supposed to be doing; and
  
• We've gotten tons of really positive feedback.
  

There was an article Robert Scoble (Scobleizer — Tech geek blogger » Blog Archive PR-less launch kicks off a stack overflow of praise «) wrote completely on his own, I didn't contact him or anything: very positive.  And a number of blog entries that we've seen from various people that are getting into StackOverflow. And then there's the "CrackOverflow", which was very amusing.

Dixon: [laughing] Yes, that was...

Atwood: They made a logo, and started calling it "CrackOverflow", and stuff like that.  I think that's obviously a compliment if people are comparing your site with something that's addictive, and hopefully addictive in a good way, right. 

Dixon: Crack for programmers.

Atwood: People are learning stuff, and seeing stuff, and getting answers to their questions.  So people have also compared it with Digg.  I'm ambivalent about that, because on one hand I like Digg, Digg's okay.  I feel like Digg is the daytime television of the Internet.  Like, does anything really good come from visiting the links on Digg?  What do you really learn from going to Digg?  You might learn something, I'm not saying it's totally without value, but it's not like PBS right.  Whereas on StackOverflow, I think participating in StackOverflow - sure it's fun and time sink - but hopefully you are learning stuff about our craft of what we do, how we do and how other people do it and that was always sort of the goal when we started StackOverflow, was to take all the stuff I have learnt, from all the years being on the Internet, like CompuServe before that and just running my blog and seeing, okay if a blog entry gets 200 comments, who's going to read all the 200 of those comments, right?  So there are certain issues of scale around blog I was trying address, and to have a forum and a place for people to go, do things like that, in a way thats more scalable.  StackOverflow is kinda to me CodingHorror 2.0 - its Coding Horror so that you guys can be in charge of it and run it and get stuff back out of it.  And to that end another observation I want to make is that StackOverflow is supposed to be run by you guys - and by you guys I mean not us the moderators, but the people participating in it - and that's one of the big reasons why we have the reputation system.  The system learns to trust you.  And as it learns to trust you it basically lets you run it.  You will be able to edit other people's stuff - thats probably the biggest feature, maybe even delete stuff, certainly re-tag things, shift things around on the site, you know mark things down that are bad, mark things up that are good, obviously with voting.  And also our spam flag.  Somebody is spamming and you can flag it and it will get deleted.  My view is if it takes us, if we have to intervene a lot in the site - me, Geoff and Jarrod, then the site is basically failing - it's not gonna scale, because we don't scale.  So it very much is a community site run by the people participating in it.  So that was a very long-winded answer, but I wanted to get it out of the way because I've been very very pleased with the way the beta is going so far.  And with that, I think, let's go ahead and get into the questions; I don't want to take up too much time with my "jibba-jabba."

I do want to start with one question I think was kind of funny. This is from Stefan--I'm the king of mispronunciation, so—Stefan C-I-O-B-A-C-A, I can't pronounce that, I apologize. But I thought this was a funny question, 'cause—let me just read it. "Number one, how many people stopped following StackOverflow because Joel insisted on allowing audio-only questions in the podcast?  Does he admit it was a mistake?  Do you think this was a decisive case of the podcast becoming boring?" This is great, because this is like, "Have you stopped beating your wife?" [Laughter] No! I love beating my wife.  It's great.  Number two, which is more directed to us: "What took so long to develop it?  It seems to me that the first version of such a simple site would take a week, maybe.  Is it because you used Windows, which takes more time to set up as a server?  Or is it the stack, which makes it more difficult to create stuff that seems easy?  Or is the product really complex?" [Laughter] So, basically, "Why do we suck?"  I don't know.  That's a tough question.  It's like, why do we suck so much?  I think we should have hired Stefan, because I think Stefan could—

Dalgas: Definitely.

Atwood: —could have written this in a week using some unknown tool that, y'know… yeah.

Dixon: It's my rampant alcoholism that's holding me back, that's what I'm saying. [Laughter] Coulda been done a long time ago.

Atwood: So let's get to the real questions here—no, let me take that back: Questions that can actually be answered.

[11:46]

Atwood: One question that I enjoyed, and since you guys are - and just to be clear - I'm in El Cerrito, California, Jarrod is in North Carolina, and Geoff is in Oregon, so we are in three different areas, and a three hour time difference for Jarrod as well.  And somebody asked a question about how do you manage the planning and the team-work when you are so far apart.  And certainly using with Skype and sitting down side by side and things like that.  So I wanted to comment on that a little bit, and let you guys comment on that as well.  I think one of the reason this worked is because Geoff and Jarrod sort of have very flexible work arrangements currently.  I mean like they're not going in 9 to 5 to a traditional job, I don't think that this would work in that scenario at all.  I guess I got lucky because Geoff and Jarrod - in addition to being some of the best programmers I've ever worked with - happen to be kind of available in the sense that they have very flexible job-schedules.  So I would say; number one - you have to have a very flexible schedule for that to work, because - you know - people being in different time zones, and being distributed, things just don't always happen in perfect sync, like - you know - ten in the morning, or nine in the morning when people arrive at work.  It's gonna happen in a more distributed fashion throughout the day, so I think flexibility in schedule is really important.  And also, I got to give props to Skype.  We're using Skype to recording this call now, and Skype has been great.  With Skype - you know - we can call each other any time we want; there's no long-distance charge; it's right on our computers so it's always just one click away; so I would say Skype has been a really huge factor in our team work.  We also try to have at least one team meeting per week, where we sort of have a day - and currently it's Wednesday, which is tomorrow - where we kind of try to be available throughout the day to just work on StackOverflow stuff, and message each other and call each other at any time.  So we do have a little tiny bit of coordination that goes on, and then most of it - honestly- is just very ad-hoc.  And I think another thing that works in out favor is that Geoff and Jarrod are like me in that they're addicted to the computer, so they are like always on the computer.  [laughter]

Dalgas: That's true.

Atwood: They are like kind of always available.  Ah so, there are some unique things about our situation that make it work.  But certainly, Geoff, you worked from home for a while, and you work with other guys not on StackOverflow, but on other stuff, and what are you thoughts on working remotely?

Dalgas: Yeah, I mean, working remotely is actually what landed me here in this nice little town Corvallis, because there's not really a whole lot of tech around me, and I don't go in on a daily basis to work, and sit down at a desk and code away, even though I do miss a lot of that, I actually miss being around co-workers and such, but the physical presence is one piece that I do miss, but having the Skype-calls and having a dialog on a regular basis really helps a lot. The technology is so good now that i think it's even easy to do - it's easy to work collaboratively with the team members that are all throughout the United States.

Atwood: Well, also, Geoff, talk a little bit about—you set up the Cruise Control integration for us - which has been huge - because when you do a check-in, I see the e-mail and things like that.

Dalgas: To be clear, I set it up, and Jarrod refined it for me. [Laughter]  Cruise Control has been great.  We do have a nightly build—well, not nightly, an "every check-in" build process, right now—where, once we find something, after we've added a little bit more awesome to it, we check it in, and it goes through a series of unit tests we have also coded.  And [it] sends us all an e-mail with the comment that was made attached to that check-in.  So we're all kind of able to watch each other's work as it gets checked in to our server.  It's kind of neat to watch the whole thing evolve, and hopefully I continue to add more awesome and not more crap. [Laughter]

Atwood: Right.  And sometimes what I'll do, if it's a check-in I'm particularly interested in, I'll just do the diff right there.  We use Beyond Compare—I'm still hooking up the licenses for you guys on that, the formal ones—and also VisualSVN, which gives us the integration with Visual Studio.  So it's like a one-click operation, to see the differences when we get latest—they don't call it "get latest," but when you get the current list of changes—I'll sometimes diff it just to see what happened there.  Jarrod, what about you?  Have you worked remotely before, Jarrod?

Dixon: Not really, they kinda frowned upon it at our previous employer.  They wanted warm bodies in the cubes.

Atwood: So this is your first time?

Dixon: Yeah, it really is, and I can say that I personally loathe the nine-to-five.  I can't stand it, so this has been kinda a wake-up call that I want to write software from the comfort of my own home.

Atwood: Right, in your lazyboy. [Laughs]

Dixon: Right. Exactly!  Kick back... [Laughs]

Atwood: Yeah.  I was very excited Jarrod finally let me buy him a Herman Miller Mirra chair. Because he was still coding in his lazy boy, which he was, "happy" with. (Laughs).  But I finally talked him into it so we are looking forward to...

Dalgas: That's another fun thing working with Jeff, is those perks.

Dixon: Yeah.

Atwood: Yes.  Well I figure I can't pay you guys very well, so I want to at least get you the best equipment.  So I have gotten them both 30 inch monitors - and I've talked about this on my blog.  I am really just living the things I am talking about on my blog.  Having good equipment is so easy relative to how much people really cost, that its a no-brainer.  So 30 inch monitors.  I bought Geoff some hardware to upgrade his PC; the chair.  You know, little things like that.  And of course: real money as well.  There is some - not a lot - but some.  So I think that covers working remotely.  And it's also a little bit of an experiment for me.  I don't think that I've done it to the degree that I'm doing it now.  But I think flexibility is the order of the day.  You have to have people that are really flexible with their schedule, their time, to even remotely work at all.

[17:50]

The second part of this question I thought was pretty good: it's about code ownership, like when you write code do you like, own it?  If something has to be changed or bug fixed, do you prefer to send it to the other person?  You know basically, about code ownership and I think that goes hand in hand with working on a team.  And I think the way I've looked at this is - now that Jarrod did a tremendous job, and I want to give Jarrod a lot of credit because initially on this project Jarrod pretty much did all the lifting in terms of setting things up and getting it started.  So Jarrod really laid the foundation for the project in terms of the technology and all the initial code that we wrote... and he did a great job because we're still building on that framework - but I feel that everybody should be able to go in and edit any part of the code.  But the question that this particular person brought up, he pointed out that sometimes it is more convenient for the last person, like say... some feature like Geoff did a lot of work around paging, so there's currently a bug in paging that he's working on, so I could pick up that bug.  Uh, but it's a low priority bug, it's not like broken or anything, its just a little off.  I just think it's easier for Geoff to go in and do it, because he originally wrote it.  But I think the core parts of the code, like certainly on our site, the core would be question-ask and question-answer, I've gone over that code a dozen times, so I definitely know that part inside and out.  So my answer would be, as you get towards the edges of the code, I think it's okay to have some stuff on the edges that has semi-ownership around it that people, you know, indirectly own it because they wrote most of it.  I think that's okay.  But the closer you get into the core, the more you want to have just total shared ownership - everybody should be comfortable changing that stuff. And did you guys have any reactions on that, Jarrod, Geoff?  About who owns what?

Dixon: I sort of felt the same way that y'know at first I had some things that I kinda like "owned" or whatever, but now it's willy nilly... edit, do whatever, beat it up, change it.  I don't have, y'know, a care any longer. [Chuckles]

Atwood: Right, what about you Geoff?

Dixon: Complete apathy... complete apathy on the code base.  That's what I feel now. [Chuckle]

Dalgas: Yeah, I feel the same way.  Well, if you break something, fix it.  That's kind of a little thing there... [Laughs] which I'm learning on paging right now.  So that's part of that.  But as far as the code ownership goes, there's this whole sense when you write some code that you really are like, "That's mine!"  There is [sic] a lot of attitudes I've seen like that, and I think that with this particular project with us working in three different corners of the world, I've been able to detach myself from the actual code itself and think about it more as, "we're building something together. We're building something that's pretty awesome, and I don't really care who changes my code or what it looks like in the end." In fact, I learn more from people's changes to my code than I would [have] ever learned from just trying to do it myself.

Atwood: Yeah, I think the only time it gets, I think, annoying is when somebody changes your code and breaks it in a way that breaks the site.  And I don't think this has really even happened.  I mean, because everybody makes mistakes, right, so... I think it's a question of (I think) trust with your teammates and basically just competency.  And you guys are so good that, I mean, if some little thing is broken it's not malice or incompetence.  It's just an oversight like I would make.  And Geoff, I remember, I was a little annoyed at Geoff because he had gone and actually deleted a bunch of code that I had written [Laughs] I was like, "Grrr", and I was a little annoyed, but then at the end it's like "I don't even care" because I just, y'know, I got the part that I needed back out of the source control and put it back in, and it was just a trust issue at that point.  It's like Geoff was trying to clean up... this was a like big refactoring we went through with implementing the first design, and I appreciate that Geoff had a ton of work to do around that, and he didn't want to deal with looking at this big blob of code I had written, so he just y'know essentially deleted it for the time being.  And that's fine because that was such a big unit of work that I just went in and later added it back in.  It wasn't essential or anything. 

So Jarrod, did you have anything to follow up on that?

Dixon: Oh there's just... since there's really no possibility of physical altercations, there's no use getting... I'll just go in and change all your stuff.  Just wait.

[22:08] [Group laughter]

Dalgas: Awesome.

Dixon: That's right... that's right.

Atwood: Well that was a great question because I do like talking about how the team works, and I would prefer it if we actually - it's impossible obviously but - I really enjoyed it when Jarrod came out to visit to California, because Jarrod had never been out to California even, and we spent at least one day working on StackOverflow very early on.  This was in the OpenID phase when, remember Jarrod when we were getting all the OpenID stuff [working]?

Dixon: Yeah yeah yeah.

Atwood: That was fun too because we had decided on OpenID and Jarrod had written like a ton of like login code already-

Dixon: ugh!

Atwood: -so you had to go in and like throw out all that code. [Laughs]

Dixon: Oh... it was... Actually, that saved... I'm sure that has saved so many headaches. Even dealing with OpenID.

Dalgas: Oh yeah.

Atwood: And to be clear, and maybe this is a good segue into the UserVoice stuff, because people are asking like how do we prioritize stuff... and to me it's all about UserVoice.  It's like what issues are people actually running into that they feel strongly enough to comment on in UserVoice.  And I want to also say, to people that are putting a lot of stuff into UserVoice and... and if you see me declining a lot of that stuff and being a little, shall we say, "ruthless" about declining: I don't mean to be rude.  The reason I do that is because we have basically 3.2 people on this project, right, that are actually writing code.  Me: I'm the only full time person.  Geoff and Jarrod and then designer Jeremy who works very, very, very part time.  So we have a really limited amount of time that we can look at this stuff, and I have to be realistic about what we can do.  And if I decline something, it doesn't mean that we're never gonna do it.  It just means that I don't want to think about it for the next... four weeks.  Right?  Like this is a project we're gonna grow.  This is one of these things where we're not building this to flip it and make some massive amount of money.  I'm building it because I genuinely think it's something useful that belongs on the Internet and that we can all own, and like I can be proud of and Geoff and Jarrod can be proud of - and Joel as well - can be proud of participating in this thing, and building something useful for everybody.  So, we want to grow this.  It's like gardening or farming.  So six months from now, four months from now, we might very well go back in and implement a lot of this stuff that you're asking for.

But getting more specifically, now that I've got all the disclaimers about me being a bastard on UserVoice and declining everybody's requests, how do we prioritize fixes that we're putting in?

[24:25]

So, I think initially we had a set of core features, that were mostly in, and I think we looked at... There have to be enough features for people to understand what the site is and how it works, and we're almost there. I think the only piece that's really missing at this point is rollbacks, which Jarrod is working on. So, the reason that's important, to be specific in the example that I'm using, is that makes it very easy, if someone makes an edit to your post you don't like, rather than freaking out about "OK, somebody touched my stuff, and somebody edited my thing, and I don't like that", there needs to be tools in place where it's very easy, like, literally one click for you to go "You know what? I don't like your edit. I'm gonna rollback to the last version that I owned, OK, because I own the post". There's a concept of post ownership at that point, and it's totally legitimate for you to do that, and then the flipside is, you should have a page where you can see edits the people have made to your stuff, and we currently don't have that, right. So these are missing pieces of the system, so I tend to prioritize those a little bit higher than "X is bust" or "Y is bust", depending on how busticated it is, because without that feature in there, people don't really get to understand the system, so they'll be making requests and complaints that don't reflect the final system at that point, cause the system isn't there. So I want to put that caveat in, as that we have to build out the missing limbs. In a previous podcast Joel talked about how "Software can be missing limbs." I was like "Well, it can, but people will think you're a monster if you're missing limbs", right? like they don't see the person or the thing, they see "Oh, this is a monster, it's got three arms and it's missing a leg, and it's disgusting", right?

You have to able to see the outline of what it is to understand it and use it, so those kind of requests go first. The other thing that I look at is request people keep opening over and over. Currently paging would be one of those things. Geoff?

Dalgas: Thank you, thank you.

Atwood: And for a while, like, the user page was just kind of busted, like it wasn't showing the correct information on the user page, like the questions and answers you participated in. And I kept getting, like every day there would be two or three UserVoice requests on that, and I bumped that up and actually told Jarrod "Jarrod, look, we have to have this in, because every day I have to deny or duplicate three request on the same darn thing." So, if I'm getting lots of requests for something, than I'll definitely bump it up. So, I wanna be clear that, on UserVoice we really do listen, and I read everything that goes in there. So, do use that.  And that's actually how we prioritize everything. So the more noise we hear around a feature, certainly the more we bump it up on the list.

Now, the other part of that question is testing. It's, like, how do we test? So, let's talk, and I know Jarrod did some great work early on, basically single-handedly giving us unit testing, and I wanna talk about that a little bit. So, our current development structure, we have, obviously we can run the solution on our local machines with local databases and all that stuff. We have a development tier, which is currently hosted on the same server as beta. So, the normal process is: we would make a change, test it locally, make sure it doesn't break anything locally, check it in, at this point it triggers the CruiseControl which does the build on dev, so that forces an integration point at that juncture. Then we go and sort of play with it on dev, and make sure it works, and unit tests run, which I'm gonna come back to, and finally if that looks good, we deploy what's on dev to beta. So, there's sort of a three tier thing we do there, not that we don't make mistakes, cause we definitely do. Now, the unit tests that we have in, are pretty incomplete and I blame me for that, because I've really been pushing, like new features over going back and adding a lot of tests for features. But, I was very pleased that Jarrod set up a test project for us, and it does run on every build. So, if we do really heinous, that breaks tons of stuff, we will see that in unit test, and it will block the deployment on dev, so we can't even deploy, until we fix the unit tests. But, Jarrod, did you wanna talk a little bit about just setting up the unit tests, some of the decisions you made, cause... all that stuff?

Dixon: Well, at first, y'know, it started out with Microsoft's testing, like the Test Project, but I was kinda running into issues that, you know, you have to, in order to get MSBuild to run those test, you basically have to install like a full blown Visual Studio on your server and that's kind of a big no-no. You have to sort of, frankenstein up some dlls just to get it to be able to push databases through MSBuild.

Atwood: Right.

Dixon: So, I went to, y'know, switched back out, I've had, y'know, lots of experience with NUnit, so, I was comfortable with that, and I guess a lot of the stuff, if there are, y'know, some pretty intricate business logic, like the stuff around reputation, how reputation changes happen. Like, we had unit tests for that, but NUnit is one of the best open source projects out there. I can't recommend that enough. So other than that, hopefully , as we give more time, as we fix some of the more glaring bugs, we'll be able to add test for, like, the controllers. Cause right now, really, our unit tests only touch the models, and, y'know, I guess, I'm to blame for that too.

Atwood: No, no, no. That's all me, because I made that judgment call. It's like, we have to plow forward. And, y'know, the interesting thing to me about not having enough unit tests, which is where we are, we have, like, ridiculously small test coverage, is: your community ends up being your defacto unit test. Like, when your unit test failure is somebody opening a ticket on UserVoice, which is very traditional, and it would be nice to block, not block that, but to short circuit that, and actually be able to fix the bug before people even see it. Obviously, that would be ideal. But on the other hand, being able to deploy all the time and add new features without, y'know, doubling or however much additional work there is to add a test, not that we won't circle back and add this stuff in later, because I definitely plan to. But it certainly makes you a little more agile, I think, with responding to people's requests for features and things like that. I mean, if I had to go in and add, y'know, five new unit tests for this feature, y'know, would I really get it done in the same amount of time. And again, I gotta to be careful here, cause I'm by no m... I'd love... Testing is a very, very good thing. I just think it's a question of limited resources on a small team and where you apply them. And I certainly look forward, like, Jarrod, where would you like to see, give me some examples of... I can think of some examples of stuff I would like to add testing around. But, give me some examples of things you would like to add unit tests around.

Dixon: Well basicaly, like, around the badges, y'know, whenever certain critera are met where you receive a new badge, we should actually have a unit test that would first the criteria to a user and run the badges, y'know, the function that grants badges and see if that badges are actually grantded. Thats one area that we could have. Around messages, when you get new badges, or when system messages occur. Editing, when revisions should happen. But we do have them on a lot of a core functionality, we do, like creating a'question and answer, a new user, we do have good unit tests there, so, it's more bells'n'whistles of the site, thats where we need some more tests.

Atwood: Geoff, what about you? I mean, have you done a lot of unit testing on your previous projects, cause I know one thing you mentioned to, which I thought was very funny, by the way, and deserves to be repeated is that when working with us it's like "Wow! You guys actually know what you're doing!" [Laughter] I guess you worked with some unsofisticated clients.

Dalgas: Oh, yeah, oh, yeah. It seems that this is, this is really the first project I've seen where it has some structure, some well formed organization. I mean it's a tribute to the ,obviously, team we have. It's great working with you guys. But, yeah, previous projects, where I go into a corporate environment, those are hardly ever thought of, at least in the places I have worked, and of course they're not developing mission critical type of applications, but even it seems when they are, they're just... there are a lot of weaknesses around testing.

Atwood: They're just, pretty much just randomly typing on the keyboard...?[Laughter]

Dalgas: Yeah, yeah. So, it's nice to see a good framework around what we have now. It's good.

Atwood: Right. Great. So, I definitily wanted to cover testing, so, that's good. So, lets see here.

"What was the biggest single technical challenge you had to overcome?"

This is from Michael. He is one of the biggest StackOverflow users actually. So I want to answer his question. Hum ...

Dixon: How did he get so much reputation ?

Atwood: He is asking questions and providing good answers. It's all.... Remember, reputation is all about other people voting your stuff up.

Dixon: That's right, Wow.

Atwood: So coming back to his question, Jarrod what would you say was the biggest hurdle on the project so far ?

Dixon: I'm gonna have to stick with the build, I mean that has been probably the most frustrating aspect. You know, just working with MS Build, trying to get certain things working on the server, they work perfectly locally but then trying to get them to work on the server that doesn't have everything installed, like a full blown Visual Studio installed. I think that Microsoft is actually, for the next version, they're actually going to try to make it so that MS Build can be stand alone and can run unit tests and can do database deployments, I'd say that right now that's given me the most heartache and angst it's just getting  automated builds up and running correctly.

Atwood: Right, actually in my previous job I would talk a lot about Team System and Build is a big part of Team System, and I used to realize that Build Engineering is almost like a full time job, I mean it is like almost, depending on the size of your team you want somebody that it's almost all they do, is think about like "how do we make the build?", "what should the build do?", "where should the build servers be?" There is a lot of engineering that goes into the build and it is legitimate software engineering it's not dummy grunt work, it's real software development.  So I totally concur there.  And it's so important, right--I mean what do we use everyday? We check in it goes to the build and the build ... This is a core part of the development process; it's hugely important.

Dixon: Yeah and mad props to Dalgas for getting Cruise Control up and running, he said something about Cruise Control and I thought it was something for pay, like you had to buy it or something. I did not even know it was open source and it is an excellent product. And I am amazed at everything it can do! I could even get rid of some of the MSBuild tasks in favor of just using Cruise Control tasks, so that was really nice.

Atwood: So I think build you would point out as the biggest challenge. What about you Geoff? Because you came in a little bit later, what would you view as the biggest challenge?

Dalgas: This was kinda a personal endeavour for me, because there's a few technology stacks that I had never worked in before.  One being MVC obviously which is just now kinda hitting mainstream.  Or it's been around now  -- what are we on, preview 3 now?

Atwood: They have a preview 4 but it's not as official as 3.

Dalgas: Yeah, so currently using 3.  So there was that learning curve.  Picking up the new concepts of using models, views, and controllers.  Along with using LINQ.  Its really my first endeavor into LINQ as well.  Its kind of a new paradigm for me.  Its one I really embrace and really enjoyed getting to learn, but all three of those new technologies right out of the gate was probably one of the more challenging, and again, more rewarding experiences for me.

Atwood: Right, so you would say the three would be LINQ, MVC, and ... what?  What is the third one. 

Dalgas: Uh, I don't know.  Being awesome? (laughs)

Atwood: Working with losers.  That would be the third one.

Dalgas: Yeah. (laughs)

Atwood: But I concur, and my answer would be a little like Geoff's, in that I had a vision of what this would be, but I felt pretty strongly that I was just kinda fed up with the whole Web Forms model; I had done several apps with Web Forms before in ASP.net and I just felt "there has to be a better way".  And I have been pretty pleased with MVC; there was a little bit of a learning curve, but I think I know it pretty well now.  But I think the bigger learning curve, honestly, is around LINQ.  I'm still... struggling... I think you guys have done a better job than I have.. these guys will go in and I will write a bunch of crappy SQL and they will be like "oh, you can just do it in LINQ" and they will just magically turn it into LINQ and I will be like "oh, thats how you do it".  Its been a challenge to kinda convert that, and I think the principle of least resistance, like I know SQL very well, so its really tempting for me to just go in and do everything in SQL, but on the other hand, for really simple queries, the LINQ is not hard, so its actually simpler to do it in LINQ because you can get your intellisense and just start typing.  So I have been really pleased with our technology choices.  I know there have been some criticism in some corners like, "why did you pick all this crazy experimental stuff for stackoverflow.  Why didn't you use proven technologies?"  What are we going to use?  COBOL? Punch cards?  [All Laugh]  Part of the fun of developing is picking things that are a little bit on the cutting edge.  I mean if you were really on the cutting edge you would be using Scala or Ruby, or something crazy like that.  So I don't really view these choices as really that controversial.  We are still in the Microsoft stack, we are still using Visual Studio, and it's kinda fun to be a little bit on the cutting edge.  Even though it can be a painful, but certainly we have run into some edge conditions with LINQ and MVC that were a little weird we have gotten past all of them.

Dixon: The interesting thing to me is it seems with this new MVC that Microsoft has done a lot of work putting together the Web Forms technology where you're really abstracted away from the - what really takes place behind the scenes, where in MVC you're writing the nuts and bolts of it, your URLs mean things; you can watch the code path, there's not this magic of view state that just happens for you, automatically You have to really pay attention to what's going on. So, it's good.

Atwood: Yeah, I agree and I think that LINQ is fantastic, I much prefer LINQ to any other Object-Relational mapper that I've used in terms of it's simplicity and in being linked to the language.  MVC as well, I think MVC, once you get it, seems so logical, like "Why weren't we all doing it this way?"  Also, I want to comment that Guy left this comment: "How are we going to stop Michael from hitting a bazillion rep points by the end of the month?"  I don't know.  We're going to have to code in like a limiter so that he doesn't get billion points by then.

Dixon: Let's make a special badge for billion?  I don't know.

Atwood: So somebody's asking about surprises, things that happened that you did not anticipate.  And I think to me one of the most gratifying things about StackOverflow is it has played out fairly close to the way I hoped it would, in that people are using it for good not evil.  One thing that has surprised me is early on we talked about one of the key things about a system like this because it's so low friction anybody can come in and leave comments we don't have this huge login barrier behind 20 walls of iron doors that slam closed how do you deal with spam, right? So from the very first day we'll have an offensive link we didn't know what to call it we went back and forth on a couple different names, but like a flag that people applied that said this stuff is evil, we might as well just call it evil right, evil!  This is evil.  And then if enough people mark something as evil, then it would disappear, poof!  Like if magical fairy dust had been applied to it or whatever, like "I'm melting," like holy water was poured on it or something. And what really surprised me that people were using, we called it offensive, Jarrod what did we call it first, like something else.

Dixon: It's... I don't remember.  We started with "offensive", but you changed it to something else.  It didn't have quite the power of "offensive"; it was kinda like fluffy.

Atwood: Right, and what surprises me is, people are using it on things which are clearly not "evil".  So either our messaging is bad, around what this function is for, or people just like tagging other people's stuff "evil".  And there is actually some user unrest around this, and I think understandably so, because people don't really like their stuff to be tagged "evil" when it's clearly not "evil".  My thing about the "evil" tag is, maybe the reason "offensive" isn't working too well, is because the system is like a bio-dome, we're too protected from the outside world at the moment, all we have is people who know what they're doing in here, and good people.  So hopefully--well not hopefully but--sadly, maybe once people come from the outside, you'll see more "evil", people try to really game the system.  But that is what the function is for, I would say that has been the most surprising thing to me, is that users are willing to tag other people's stuff with "offensive".

[42:01]
Because I wouldn't ... I mean it wouldn't have even occurred to me to do that. Because, you're supposed to ... there's two axis here. One is 'up vote, down vote'. And the other is the evil axis. Right? And these are very independent things. And you would think if you disagreed with something or think it's stupid or dumb or you just hate that person, fine, whatever, just down vote it. Right? And another adjustment we made was that 'down votes' used to be worth, uh, -5 to a person's rep. That went down to -2. And then finally -2 to the receiver and -1 to you (laughs) to really discourage people from doing this just blindly, um, and down voting people's stuff. And that seems to be working well. I've been pleased with the use of voting so far. But what about you guys? What have you found surprising? Like, Jarrod, what have you found surprising about the way people have used StackOverflow?

Dixon: Well, I guess in in hindsight. It really wasn't ... well, just exactly what you said ... it really, but, in hindsight, it wasn't surprising that people would, you know, tag other people's posts as offensive or down vote. Simply. But it was, it was really the users that had you know, accrued the most reputation that were being targeted. You know, in sort of like a petty competition. You know how, some users were jealous of all the rep that certain users had.

Atwood: Well, okay, we do have to clarify a little, because this is partially Leon Bambrick's fault—and I have no problem calling him out by name.  He intentionally—and I know Leon, I mean I've never met him but I know him through his blog and I feel like I know him, so he's kind of a "friend" —I think he was just playing with the system and trying to see what would happen if he did this, and he went around and actually downvoted other people's stuff to try to get ahead of them in reputation as sort of a game, and I think he was also tagging people's stuff "offensive."  In a way, this is good.  This is what the beta's about, right?  If everybody follows all the rules and doesn't do anything weird, then it's not a very good beta.  So I encourage this.  I think this is good.  But one downside of that was that I think certain people got really upset about that behavior, and I think Leon should have been more clear that, "hey, I'm testing the system," or whatever.  Just to give some background to what you're saying there.  So I think there was one specific person and one specific event… [laughter] … that triggered some of this.  And I want to put some of that blame on Leon's doorstep.  But go ahead.

[43:15]

Dixon: Right, well I'm sure that, you know, once, everything, you know, we go, fully public, that that would, happen anyway.

Atwood: Oh totally, I mean I agree and that's why it's good ultimately to have it happen I just wish he had been more open about the background of what he was doing, that's all.

Dixon: I guess, ah, another surprising aspect is the performance of the site. I've been very pleased with um, I mean, granted we don't have a lot of data and we don't really have that many users, but, you know, the, the server is holding up pretty well, you know, with everything that we're doing.

Atwood: Right.

Dixon: I'm very pleased with LINQ, you know, the, the queries that LINQ generates, you know um, LINQ to SQL has, has been very, a pleasant surprise, you know, just how well it's performing.

Atwood: Right. Right, we also had a question, and this might answer that question, it's like "You guys went with a WISA stack, [Windows, IIS, SQL, ASP.NET]. Do you think WISA will scale?" And I think, actually, compared to LAMP, and I think the funny thing here is, Windows scales like crazy! I don't know why it gets a bad rap, because honestly, if we had built this in PHP I can tell you first hand PHP is a dog for performance. [laughs] I mean there are ways to get around it and I don't want to offend anybody that writes PHP, but PHP is an interpreted language, right, I mean you can add some stuff onto it, like bolt-on these compiler things and stuff, but it is not a very high-performing stack dude, I mean, like, our stack, ah because it's compiled and you know, it's been through three iterations so far. I mean, our, we have a ok so, our server box is 8 CPUs. I, I don't think I've, it's rare we see any spike at all, even with all the users we have now and granted, we have a limited set of users etcetera etcetera but, I mean, but I don't think you can measure our load, at the moment, and, and we're not totally optimal either, I mean we're doing things that I think could be further optimized, so we're very much in a, a mode where we're giving up performance and it doesn't matter. So I want to say that: gosh, for Windows and ASP.NET you're going to get really high performing stack, I think much better than a LAMP stack actually. On, on, coz, if you look at Facebook and well, granted Facebook is a ridiculous example because they have like a billion servers but their scaling model and I also talked to Ted Dziuba of PressFlip and they, they used Amazons' ah, EC, like server-cloud stuff, and they have like 12 servers to do their stuff, and we're doing everything on one server and granted, I, I don't know that we'll offer scale to those levels, but I don't, it's going to take us a while to outgrow even this one server I think based on what I've seen so far, so, anyway, I want to give a shout out there. What about you Geoff, what have you been surprised about with the way people have been using the site or anything like that?

Dalgas : Well, I've actually kind of been watching the more social aspect of the site. And this is kind of one of those endeavors into a social realm where we're hitting people that are really smart really talented developers. But, just, I'm kind of, I'm interested in what they're, what they're asking as far as questions and what answers they're giving and how this reputation system's being played out, um, it's really interesting to me to see how, how people will be motivated by this. I thought before, that, well, what's the point? You know? Badges and points, it just doesn't make a whole lot of sense, so what? But, as I see more and more people get into this, they're really, they're really motivating the use of this site. And that to me is very interesting, very fascinating.

Atwood: Right. And, well, I think that's great, too,  because, the reason, all this stuff is there for a reason. Nothing on StackOverflow is there because, 'Oh, I thought it would be fun to have some stupid feature that is fun'. I mean, we actually put a lot of thought into how we build the site. And, Jarrod being involved earlier on and Joel as well. I mean, we went through several cycles with Joel, with Jarrod, um, and Jarrod, Jarrod and I spent, what was it? Like two weeks? Just like looking at all the 'Q&A' sites on the Internet, like all, all, the one, we signed up for all of 'em.

Dixon: I now consider myself, dumber for having done that.

Atwood: It's a running joke that Yahoo answers is just kinda a black hole of, of stupidity honestly.

Dixon: How do I get my baby's daddy to pay?

Atwood: Yeah, that's the classic.  Then in some email newsletter, they sent out recently their, their featured question was like "I don't read and write too good, how do I get edumacated?"  Pretty much.

Dixon: Yeah.

Atwood: So, and I think what works for us on StackOverflow is one, programmers. Programmers, and I know I'm preaching to the choir here. Programmers are smarter than the average person I think, because you have to be, because it's a really difficult field, because everything goes wrong all the time. Anyway, I do believe that programmers are cut from a slightly better cloth then the average person, the good programmers and I think Joel and I, having large audiences of very good programmers, again that honeymoon, honeymoon phase we enjoy the benefits of having a really talented audience basically. And the site is about those people showing off how good they are and sharing that with, with the other developers so they can become better. So it's very much to be a virtuous cycle, that we get into here. Um and, and hopefully that will continue. Going back to the answers thing as the audience gets wider and wider, does it get dumbed down? I don't know. Um, my hope is that stack overflow will attract, the really good developers who are interested in sharing, um and letting other developers get better, through, through posts and questions and answers in a very low friction way that's easy.

So, Alan Le, who I used to work with at Vertigo, had a question. He's asking about the API for StackOverflow. I think it's too early to think about APIs at the moment. Certainly RSS is huge on our feature list. I think it's the next thing Geoff is gonna work on. Getting a feed of stuff that's coming in and your stuff and all that. So, I think for now, when you say API, think RSS for now, I know it's not really an API, but I'm gonna pretend like that answers the question. Then Alan asked how are we dealing with mischievous, he says "mischievous," users, spammers, griefers, marketers, etc. So, we talked earlier about the "offensive" link, which, that's really what that's about. And that's what that's supposed to be used for. So, hopefully people will use that. Also, we are gonna implement a CAPTCHA, that's another thing Geoff is working on. We're gonna have a jQuery JavaScript CAPTCHA, just sort of a basically force whoever is sending request to us to use JavaScript, and we're still debating the fallback path, cause it is possible to use StackOverflow with JavaScript disabled. It's not a great experience, but it does work. And it's really tricky to come up with a CAPTCHA that works well with JavaScript disabled, that's not annoying to you. So, we're still debating that internally. So, my hope is... Oh and, then the third tier, that... I think, Jarrod, this is on your plate still: throttling. Like, I believe throttling is really central to any "mischievous" user, or evil users. Throttling is huge. If you can throttle people from doing N actions per minute, that disabuses you of so many possible problems. So, the beta is gonna run, I think,  for the full month of August, and by the time we come out of beta, I wanna be hardened, I'm gonna use security word "hardened", where there is not much you can do to us. Barring, like some international distributed denial of service. There's always something. I'm not gonna come out and say...

Someone: [through laghter]You don't wanna dare.

Atwood: I'm not daring anyone to attack us, cause I don't want you to want to attack us. All I want is, and I think this has come up too, we will not tolerate abusive users. I wanna be very a 100%, 1000% clear on that. If there are members of our community that are abusive, we will ban them, we will block them, we're gonna do all the things, and hopefully again, you guys will be able some of this, right? Like, you're not gonna have to call a moderator. You yourselves will be able to do this. One thing I've thought about, and we may have to end up doing, is, I think there should be ways you can vote against a user, right? Cause if you look at... Achievements are modeled on Xbox 360. One thing you can also do on Xbox 360 is you can essentially vote on a person. Like, say you are in a multiplayer game and somebody griefs you and does a bunch of stuff. You then vote against that person. So, I think we might need basically an "offensive" flag that you're gonna apply to users account. That's a little bit off in the distance, but again the goal is for people participating in this site to run it to some pretty high degree with very minimal moderator input, because moderators  don't scale, but the community does. So, that's pretty much my thought on that.

So, did you guys have anything, any particular questions you wanted to get to. I know, here's a funny one for you, Jarrod: "What is Jarrod looking at, and why is it making in so happy?" in the picture that I posted on the blog.

Dixon: [laughs]

Atwood: That's my fault. I told you to look maje..., I think the world I used was majestic. I was like "Jarrod, look majestic!". So, I was taunting.

Dixon: That was great, what's that called, Lookout hill, Lookout point? What's the name of that?

Atwood: Oh gosh, some point, some place in San Francisco, some tourist...

Dixon: ...and I was soo amazed at how chilly it was there in San Francisco

Atwood: Dude, it's [garbled] almost May. It gets really really cold in San Francisco. If you ever travel to San Francisco, bring like your heavy jacket. I know you may feel ridiculous, but you'll probably need it, and you always have that conversation with people that are visiting "Bring you heavy jacket, even though it's the middle of summer". So, Jarrod, any other questions you wanna get to?

Dixon: One thing I wanted to... well, I wanted to say what a pleasure it's been working with ASP.NET MVC. I mean, I was so fed up with the Web Forms, y'know, having to deal with that. Some of the last projects I've been working on, I was actually writing HTTP handlers, just to render out the straight HTML, just so I wouldn't have to deal with all those stuff that goes along with Web Forms. And it's been such a pleasure, I guess... Is it Phil Haack, he's the project lead on that?

Atwood: He's on of... there's like twenty people involved.

Dixon: Big shout out to him and his team. So, it's been wonderfull, I really enjoy, especially the changes they made from preview 2 to 3. Like, the controller actions returning an action result. That's been great, and it's been so great for doing AJAX, so now our controller actions, they can just easily render JSON back to the browser, they don't have to go through like any, intermediary steps, intermediate steps. So, just, that's what I wanted to say... [incomprehensible]

Atwood: Yeah, I agree completely. MVC maps much better to my mental model of how I want web stuff to work by far. It's not even close. To me it makes Web Forms totaly obsolete, to the point I don't wanna touch it any more.

What about you Geoff? Any final questions you wanna get to?

Dalgas: To kinda put a step on top of that, RSS feeds are gonna be basically just another View on top of the... on the Models, Views and Controllers. So, that's gonna be an easy task because of MVC.

Atwood: Oh, Geoff, you always say it's gonna be easy.

Dalgas: It's beautiful. Simple. Just simple.

Atwood: So, Geoff, any last questions you wanna get to? Anything we didn't cover? Comments you wanna make?

Dalgas: Umm, let's see. No, I'm just really pleased to see the developer turnout for this, and continue on a...y'know, wanna have them keep on keeping on. There's a lot of questions here, I really wanted to know the answer to, and I never asked it? And so, I'm learning a lot of things, just by reading question here. It's been great.

Atwood: One question that was funny, kinda funny "How do we come to a consensus on decisions? Is it a democracy?" And these guys are laughing, cause they know, it's, I pretend that it's a democracy,  I pretend to listen to their feelings and thoughts and then I overrule them with, y'know, impunity.. .

Dalgas: Then I go and cry, I go sob in the corner..

Atwood: It is probably a little like... I know, I apologize, but I do... I can be convinced. I think it's the whole "strong opinions, weakly held" thing. Like, I pick a direction "No, we're gonna go this way", but you can really... you can talk me out of that, but you have to, like, convince me, you have to convince me, you have to talk me out of that and make me believe. You have to make me believe what you're saying is the better choice. But, it does happen. I don't wanna sound like a total jerk, or anything [laughs]

Dalgas: To have to justify why you're making a choice, verbally or technically or whatever, is actually really good exercise to go through. If people don't challenge you, then why are doing that you're doing?

Atwood: Oh, yeah. Great. I do wanna people to challenge me. I enjoy working with people that are like "Y'know, Jeff, I think this is really the wrong way, I think this is not a good solution, and step up a little bit and take a different direction" and defend it. I think it's great. I like that style of communication, and I think, obviously, I tend to pick people that are comfortable with that style of interaction over time.

And, then so, as we get closer to the end of the podcast here, I just have a final note to say how, I know people joke that it's called CrackOverflow. It's been really addictive for me to, like, work on a project now. As people start to use it more, like, I wanna work on it more, I feel like, as you put changes into production and stuff, you're impacting all the people that are using it, and you're satisfying their requests, and making it better, and growing an improved site. And, it's actually, hurting my other, my CodingHorror blog a little, because I'm actually really enjoying working on StackOverflow with Geoff and Jarrod. I'm a little bit addicted... if you guys are addicted to using it, I'm kinda addicted to building it, a little bit, and that's really all I have to say about that.

So, any final comments, you two guys, before we cut off the podcast here?

Dixon: No.

Dalgas: No. We can continue on...

Atwood: Speaking of continuing on, one of my goals... people ask about money and stuff. One of my big goals for money on this project is - I would love to make enough with StackOverflow to pay Geoff and Jarrod, so that they could legitimately work on this as a job. I would love that. So when you look at the site thinking "Jeff's gonna get a gold plated Hummer", I mean, I might get a gold plate[laughter]... but really, what I would like to do more than anything else is to be able to pay Geoff and Jarrod to work on the site, part time if not full time. I would love that, that would be wonderful, that's one of my major goals for this thing to be sustainable, and to grow it, and to have people that are attached to it, that love it, and care for it, and are into the community, right, and part of the community as well. So that's one of my long term goals for the site.

I would like to say thanks to Jarrod and Geoff for participating. I know it them puts on the spot a little bit, but hopefully you guys enjoyed this, people asked for it.

Dixon: Yeah, man.

Atwood: And I know I enjoyed it, and, y'know, maybe we'll do this again sometime in the future.

Dalgas: Was it good for you, Jeff?

Atwood: It was very good. So, thanks to everybody for listening. I do wanna cover... we do have a wiki of transcriptions of this podcasts, and if you participate in the wiki, and do like say one or two minutes of transcription, I will bump you in the beta list tonight. Cause, right now we have a backlog of about three thousand people, and I'm sending like hundred and fifty per night, so if you wanna get in on the beta like today or tomorrow or whenever you're hearing this, do a little bit of transcription work and then email me directly. And that's at the stackoverflow.fogbugz.com. And then for future podcasts, as we always say, email your audio questions to podcast@stackoverflow.com if you want to answer them in future, and I think that's it. Thanks everybody for listening, and thanks Geoff and Jarrod for doing this.

Dalgas: Thank you guys.

Dixon: Thank you.

Atwood: Alright. Goodbye.

[1:00:30]

Outro, Advertising

Ads, intro

[1:00]

Atwood: So, we are in a call now with the development team for StackOverflow. That means with me, Jeff Atwood, Jarrod Dixon, and Geoff Dalgas. So this is Podcast, I believe number 17. It technically should be 18, but Joel Spolsky is on a jet-setting international vacation all around the world, so he is not available. When I posted that we wouldn't have two podcasts for two weeks, one of the suggestions we got in the comments was, "How about a developer podcast?"  I thought that was a great idea, because I remember when Jarrod was visiting with me in California, I talked about actually having Jarrod on one of the podcast calls but the timing didn't work out.  But I think it's exciting to have a call with the people who are really doing the work, because we know Joel doesn't really do anything, right?

I'm only kidding: Joel does a lot of stuff.  But these are literally the people building the site so that's why it's exciting to have an opportunity to give them a voice.  You guys can see what we're doing and how we're doing it.  So I thought we would start by introducing ourselves, everyone knows me, probably, so I'm not gonna bother introducing myself, so Geoff, do you wanna take it away and introduce yourself?

Dalgas : Sure, yeah, you bet.  My name is Geoff Dalgas and I've been working with Jeff Atwood on StackOverflow. I was one of the late members that was added to the team.  I've been happy, it's been great, it's been a lot of fun.  I got to know Jeff back in Denver, Colorado working with him through a small software development company there. And ah got to sit through a lot of his beratement [sic] and as a result of it, I think I'm a better developer, I'm not really sure though, so we'll find out when the bugs come my way.

Atwood : So Geoff, you got a lot of the rants, that I think would become blog entries later in my life, so that's...

Dalgas : Yes, before blogging was hip, I got to hear the majority of the rants as in, "Don't touch my monitor!" and a few of those. [laughing]

Atwood : Yes, exactly.  That was Denver and I think the time we worked together was, I wanna say '96... yeah, '96 was when I first met you and we worked together till like gosh '98 or '99?  It was a couple years.  It was a couple good years, but it was a lot of fun.

Dalgas : Long years, long years. [laughing]

Atwood : Thats right. And how about you, Jarrod?  You want to introduce yourself to the group?

Dixon: Yeah, my name is Jarrod Dixon and I've been working with Jeff now on StackOverflow I guess the past four months now. I got to know him back in Raleigh, North Carolina. We worked together for a large pharmaceutical organization that just loved overcharging people I guess.  No not really.  But I guess I've been a developer for about  seven years, and really enjoy the opportunity that Jeff has given me to work with him.

[4:00]

Atwood: Now both of you guys, I think - and I've written about this in my blog before - you guys kind of grew up with computers like I did.  Because I remember, Jared, you telling me that one book, what was it - Computer Monsters?  That you used to read and type in programs.  At what age would you say you guys started getting into computers?  Like Jarrod, what would you say?

Dixon: Oh man. I guess my parents, they got an old Apple II+ when I was three or four years old, so I was messing around from that early of an age, you know with BASIC and stuff.  Just messing, playing around.  It wasn't like--I look back at how ignorant I  was--

Atwood: --Well everybody--

Dixon: --It was a real young age.

Atwood: Yeah, and what about you, Geoff?

Dalgas: I remember quite well.  Commodore 64.  And a good friend of mine, Ethan, had one.  We didn't have one; we couldn't afford one as a young chap.  I still barely can afford one now.  But we used to sit up nights, pounding in the BASIC programming, just right out of those magazines like you're referring to, and trying to get it to compile.  Trying to get it to run.  We'd get it going for a while and hit a bug, or hit a breakpoint, and spend the next 12 hours trying to figure it out so.  [laughs]

Atwood: Right.  And when I met you guys, I think I was drawn to you guys because we had that shared background.  We both, at an early age, had been really fascinated by computers and programming, and I feel like programmers who started out that way can sort of recognize each other.  I don't know what it is; there's some innate love of the craft that you have.  There's a lot of different kinds of good programmers.  I don't mean to say anything negative about people who don't do it this way, but I think we can recognize each other. [laughs]  And you have that common bond, which is kind of fun.

[6:00]
Atwood: So let's talk a little bit about the logistics, of where we are in beta.  So we've been in beta since last Thursday, so it hasn't.... has it even been a week?  It's been over a week now. 

Dalgas: Hah!

Atwood: It feels like months now right?

Dixon: It does

Atwood: But the beta is going really well, I was very pleased.  I didn't know what to expect honestly.  And I think when we came into private beta I think I was a little scared, just because you have been working on this thing for a while and you want to show it to people, but on the other hand it's like, its like your baby and you feel like people are going to abuse it and do means things to it, say mean things to you: all that stuff.  But the response have been great!  And by that I mean when people are exposed to the site - and maybe it's just as Geoff said he calls this the "honeymoon phase", when we are drawing in people that were interested in the project from an early date - but I feel like they are using it the way I envisioned people would use it, the way I wanted people to use it.  Which to me is very gratifying.  In other words:

• People are asking good questions (for the most part);
  
• People are getting good answers, like really quickly;
• The community part of it is working very well, in that the reputation system is mostly doing what it's supposed to be doing;
  
• The badges system is doing what it's supposed to be doing; and
  
• We've gotten tons of really positive feedback.
  

There was an article Robert Scoble (Scobleizer — Tech geek blogger » Blog Archive PR-less launch kicks off a stack overflow of praise «) wrote completely on his own, I didn't contact him or anything: very positive.  And a number of blog entries that we've seen from various people that are getting into StackOverflow. And then there's the "CrackOverflow", which was very amusing.

Dixon: [laughing] Yes, that was...

Atwood: They made a logo, and started calling it "CrackOverflow", and stuff like that.  I think that's obviously a compliment if people are comparing your site with something that's addictive, and hopefully addictive in a good way, right. 

Dixon: Crack for programmers.

Atwood: People are learning stuff, and seeing stuff, and getting answers to their questions.  So people have also compared it with Digg.  I'm ambivalent about that, because on one hand I like Digg, Digg's okay.  I feel like Digg is the daytime television of the Internet.  Like, does anything really good come from visiting the links on Digg?  What do you really learn from going to Digg?  You might learn something, I'm not saying it's totally without value, but it's not like PBS right.  Whereas on StackOverflow, I think participating in StackOverflow - sure it's fun and time sink - but hopefully you are learning stuff about our craft of what we do, how we do and how other people do it and that was always sort of the goal when we started StackOverflow, was to take all the stuff I have learnt, from all the years being on the Internet, like CompuServe before that and just running my blog and seeing, okay if a blog entry gets 200 comments, who's going to read all the 200 of those comments, right?  So there are certain issues of scale around blog I was trying address, and to have a forum and a place for people to go, do things like that, in a way thats more scalable.  StackOverflow is kinda to me CodingHorror 2.0 - its Coding Horror so that you guys can be in charge of it and run it and get stuff back out of it.  And to that end another observation I want to make is that StackOverflow is supposed to be run by you guys - and by you guys I mean not us the moderators, but the people participating in it - and that's one of the big reasons why we have the reputation system.  The system learns to trust you.  And as it learns to trust you it basically lets you run it.  You will be able to edit other people's stuff - thats probably the biggest feature, maybe even delete stuff, certainly re-tag things, shift things around on the site, you know mark things down that are bad, mark things up that are good, obviously with voting.  And also our spam flag.  Somebody is spamming and you can flag it and it will get deleted.  My view is if it takes us, if we have to intervene a lot in the site - me, Geoff and Jarrod, then the site is basically failing - it's not gonna scale, because we don't scale.  So it very much is a community site run by the people participating in it.  So that was a very long-winded answer, but I wanted to get it out of the way because I've been very very pleased with the way the beta is going so far.  And with that, I think, let's go ahead and get into the questions; I don't want to take up too much time with my "jibba-jabba."

I do want to start with one question I think was kind of funny. This is from Stefan--I'm the king of mispronunciation, so—Stefan C-I-O-B-A-C-A, I can't pronounce that, I apologize. But I thought this was a funny question, 'cause—let me just read it. "Number one, how many people stopped following StackOverflow because Joel insisted on allowing audio-only questions in the podcast?  Does he admit it was a mistake?  Do you think this was a decisive case of the podcast becoming boring?" This is great, because this is like, "Have you stopped beating your wife?" [Laughter] No! I love beating my wife.  It's great.  Number two, which is more directed to us: "What took so long to develop it?  It seems to me that the first version of such a simple site would take a week, maybe.  Is it because you used Windows, which takes more time to set up as a server?  Or is it the stack, which makes it more difficult to create stuff that seems easy?  Or is the product really complex?" [Laughter] So, basically, "Why do we suck?"  I don't know.  That's a tough question.  It's like, why do we suck so much?  I think we should have hired Stefan, because I think Stefan could—

Dalgas: Definitely.

Atwood: —could have written this in a week using some unknown tool that, y'know… yeah.

Dixon: It's my rampant alcoholism that's holding me back, that's what I'm saying. [Laughter] Coulda been done a long time ago.

Atwood: So let's get to the real questions here—no, let me take that back: Questions that can actually be answered.

[11:46]

Atwood: One question that I enjoyed, and since you guys are - and just to be clear - I'm in El Cerrito, California, Jarrod is in North Carolina, and Geoff is in Oregon, so we are in three different areas, and a three hour time difference for Jarrod as well.  And somebody asked a question about how do you manage the planning and the team-work when you are so far apart.  And certainly using with Skype and sitting down side by side and things like that.  So I wanted to comment on that a little bit, and let you guys comment on that as well.  I think one of the reason this worked is because Geoff and Jarrod sort of have very flexible work arrangements currently.  I mean like they're not going in 9 to 5 to a traditional job, I don't think that this would work in that scenario at all.  I guess I got lucky because Geoff and Jarrod - in addition to being some of the best programmers I've ever worked with - happen to be kind of available in the sense that they have very flexible job-schedules.  So I would say; number one - you have to have a very flexible schedule for that to work, because - you know - people being in different time zones, and being distributed, things just don't always happen in perfect sync, like - you know - ten in the morning, or nine in the morning when people arrive at work.  It's gonna happen in a more distributed fashion throughout the day, so I think flexibility in schedule is really important.  And also, I got to give props to Skype.  We're using Skype to recording this call now, and Skype has been great.  With Skype - you know - we can call each other any time we want; there's no long-distance charge; it's right on our computers so it's always just one click away; so I would say Skype has been a really huge factor in our team work.  We also try to have at least one team meeting per week, where we sort of have a day - and currently it's Wednesday, which is tomorrow - where we kind of try to be available throughout the day to just work on StackOverflow stuff, and message each other and call each other at any time.  So we do have a little tiny bit of coordination that goes on, and then most of it - honestly- is just very ad-hoc.  And I think another thing that works in out favor is that Geoff and Jarrod are like me in that they're addicted to the computer, so they are like always on the computer.  [laughter]

Dalgas: That's true.

Atwood: They are like kind of always available.  Ah so, there are some unique things about our situation that make it work.  But certainly, Geoff, you worked from home for a while, and you work with other guys not on StackOverflow, but on other stuff, and what are you thoughts on working remotely?

Dalgas: Yeah, I mean, working remotely is actually what landed me here in this nice little town Corvallis, because there's not really a whole lot of tech around me, and I don't go in on a daily basis to work, and sit down at a desk and code away, even though I do miss a lot of that, I actually miss being around co-workers and such, but the physical presence is one piece that I do miss, but having the Skype-calls and having a dialog on a regular basis really helps a lot. The technology is so good now that i think it's even easy to do - it's easy to work collaboratively with the team members that are all throughout the United States.

Atwood: Well, also, Geoff, talk a little bit about—you set up the Cruise Control integration for us - which has been huge - because when you do a check-in, I see the e-mail and things like that.

Dalgas: To be clear, I set it up, and Jarrod refined it for me. [Laughter]  Cruise Control has been great.  We do have a nightly build—well, not nightly, an "every check-in" build process, right now—where, once we find something, after we've added a little bit more awesome to it, we check it in, and it goes through a series of unit tests we have also coded.  And [it] sends us all an e-mail with the comment that was made attached to that check-in.  So we're all kind of able to watch each other's work as it gets checked in to our server.  It's kind of neat to watch the whole thing evolve, and hopefully I continue to add more awesome and not more crap. [Laughter]

Atwood: Right.  And sometimes what I'll do, if it's a check-in I'm particularly interested in, I'll just do the diff right there.  We use Beyond Compare—I'm still hooking up the licenses for you guys on that, the formal ones—and also VisualSVN, which gives us the integration with Visual Studio.  So it's like a one-click operation, to see the differences when we get latest—they don't call it "get latest," but when you get the current list of changes—I'll sometimes diff it just to see what happened there.  Jarrod, what about you?  Have you worked remotely before, Jarrod?

Dixon: Not really, they kinda frowned upon it at our previous employer.  They wanted warm bodies in the cubes.

Atwood: So this is your first time?

Dixon: Yeah, it really is, and I can say that I personally loathe the nine-to-five.  I can't stand it, so this has been kinda a wake-up call that I want to write software from the comfort of my own home.

Atwood: Right, in your lazyboy. [Laughs]

Dixon: Right. Exactly!  Kick back... [Laughs]

Atwood: Yeah.  I was very excited Jarrod finally let me buy him a Herman Miller Mirra chair. Because he was still coding in his lazy boy, which he was, "happy" with. (Laughs).  But I finally talked him into it so we are looking forward to...

Dalgas: That's another fun thing working with Jeff, is those perks.

Dixon: Yeah.

Atwood: Yes.  Well I figure I can't pay you guys very well, so I want to at least get you the best equipment.  So I have gotten them both 30 inch monitors - and I've talked about this on my blog.  I am really just living the things I am talking about on my blog.  Having good equipment is so easy relative to how much people really cost, that its a no-brainer.  So 30 inch monitors.  I bought Geoff some hardware to upgrade his PC; the chair.  You know, little things like that.  And of course: real money as well.  There is some - not a lot - but some.  So I think that covers working remotely.  And it's also a little bit of an experiment for me.  I don't think that I've done it to the degree that I'm doing it now.  But I think flexibility is the order of the day.  You have to have people that are really flexible with their schedule, their time, to even remotely work at all.

[17:50]

The second part of this question I thought was pretty good: it's about code ownership, like when you write code do you like, own it?  If something has to be changed or bug fixed, do you prefer to send it to the other person?  You know basically, about code ownership and I think that goes hand in hand with working on a team.  And I think the way I've looked at this is - now that Jarrod did a tremendous job, and I want to give Jarrod a lot of credit because initially on this project Jarrod pretty much did all the lifting in terms of setting things up and getting it started.  So Jarrod really laid the foundation for the project in terms of the technology and all the initial code that we wrote... and he did a great job because we're still building on that framework - but I feel that everybody should be able to go in and edit any part of the code.  But the question that this particular person brought up, he pointed out that sometimes it is more convenient for the last person, like say... some feature like Geoff did a lot of work around paging, so there's currently a bug in paging that he's working on, so I could pick up that bug.  Uh, but it's a low priority bug, it's not like broken or anything, its just a little off.  I just think it's easier for Geoff to go in and do it, because he originally wrote it.  But I think the core parts of the code, like certainly on our site, the core would be question-ask and question-answer, I've gone over that code a dozen times, so I definitely know that part inside and out.  So my answer would be, as you get towards the edges of the code, I think it's okay to have some stuff on the edges that has semi-ownership around it that people, you know, indirectly own it because they wrote most of it.  I think that's okay.  But the closer you get into the core, the more you want to have just total shared ownership - everybody should be comfortable changing that stuff. And did you guys have any reactions on that, Jarrod, Geoff?  About who owns what?

Dixon: I sort of felt the same way that y'know at first I had some things that I kinda like "owned" or whatever, but now it's willy nilly... edit, do whatever, beat it up, change it.  I don't have, y'know, a care any longer. [Chuckles]

Atwood: Right, what about you Geoff?

Dixon: Complete apathy... complete apathy on the code base.  That's what I feel now. [Chuckle]

Dalgas: Yeah, I feel the same way.  Well, if you break something, fix it.  That's kind of a little thing there... [Laughs] which I'm learning on paging right now.  So that's part of that.  But as far as the code ownership goes, there's this whole sense when you write some code that you really are like, "That's mine!"  There is [sic] a lot of attitudes I've seen like that, and I think that with this particular project with us working in three different corners of the world, I've been able to detach myself from the actual code itself and think about it more as, "we're building something together. We're building something that's pretty awesome, and I don't really care who changes my code or what it looks like in the end." In fact, I learn more from people's changes to my code than I would [have] ever learned from just trying to do it myself.

Atwood: Yeah, I think the only time it gets, I think, annoying is when somebody changes your code and breaks it in a way that breaks the site.  And I don't think this has really even happened.  I mean, because everybody makes mistakes, right, so... I think it's a question of (I think) trust with your teammates and basically just competency.  And you guys are so good that, I mean, if some little thing is broken it's not malice or incompetence.  It's just an oversight like I would make.  And Geoff, I remember, I was a little annoyed at Geoff because he had gone and actually deleted a bunch of code that I had written [Laughs] I was like, "Grrr", and I was a little annoyed, but then at the end it's like "I don't even care" because I just, y'know, I got the part that I needed back out of the source control and put it back in, and it was just a trust issue at that point.  It's like Geoff was trying to clean up... this was a like big refactoring we went through with implementing the first design, and I appreciate that Geoff had a ton of work to do around that, and he didn't want to deal with looking at this big blob of code I had written, so he just y'know essentially deleted it for the time being.  And that's fine because that was such a big unit of work that I just went in and later added it back in.  It wasn't essential or anything. 

So Jarrod, did you have anything to follow up on that?

Dixon: Oh there's just... since there's really no possibility of physical altercations, there's no use getting... I'll just go in and change all your stuff.  Just wait.

[22:08] [Group laughter]

Dalgas: Awesome.

Dixon: That's right... that's right.

Atwood: Well that was a great question because I do like talking about how the team works, and I would prefer it if we actually - it's impossible obviously but - I really enjoyed it when Jarrod came out to visit to California, because Jarrod had never been out to California even, and we spent at least one day working on StackOverflow very early on.  This was in the OpenID phase when, remember Jarrod when we were getting all the OpenID stuff [working]?

Dixon: Yeah yeah yeah.

Atwood: That was fun too because we had decided on OpenID and Jarrod had written like a ton of like login code already-

Dixon: ugh!

Atwood: -so you had to go in and like throw out all that code. [Laughs]

Dixon: Oh... it was... Actually, that saved... I'm sure that has saved so many headaches. Even dealing with OpenID.

Dalgas: Oh yeah.

Atwood: And to be clear, and maybe this is a good segue into the UserVoice stuff, because people are asking like how do we prioritize stuff... and to me it's all about UserVoice.  It's like what issues are people actually running into that they feel strongly enough to comment on in UserVoice.  And I want to also say, to people that are putting a lot of stuff into UserVoice and... and if you see me declining a lot of that stuff and being a little, shall we say, "ruthless" about declining: I don't mean to be rude.  The reason I do that is because we have basically 3.2 people on this project, right, that are actually writing code.  Me: I'm the only full time person.  Geoff and Jarrod and then designer Jeremy who works very, very, very part time.  So we have a really limited amount of time that we can look at this stuff, and I have to be realistic about what we can do.  And if I decline something, it doesn't mean that we're never gonna do it.  It just means that I don't want to think about it for the next... four weeks.  Right?  Like this is a project we're gonna grow.  This is one of these things where we're not building this to flip it and make some massive amount of money.  I'm building it because I genuinely think it's something useful that belongs on the Internet and that we can all own, and like I can be proud of and Geoff and Jarrod can be proud of - and Joel as well - can be proud of participating in this thing, and building something useful for everybody.  So, we want to grow this.  It's like gardening or farming.  So six months from now, four months from now, we might very well go back in and implement a lot of this stuff that you're asking for.

But getting more specifically, now that I've got all the disclaimers about me being a bastard on UserVoice and declining everybody's requests, how do we prioritize fixes that we're putting in?

[24:25]

So, I think initially we had a set of core features, that were mostly in, and I think we looked at... There have to be enough features for people to understand what the site is and how it works, and we're almost there. I think the only piece that's really missing at this point is rollbacks, which Jarrod is working on. So, the reason that's important, to be specific in the example that I'm using, is that makes it very easy, if someone makes an edit to your post you don't like, rather than freaking out about "OK, somebody touched my stuff, and somebody edited my thing, and I don't like that", there needs to be tools in place where it's very easy, like, literally one click for you to go "You know what? I don't like your edit. I'm gonna rollback to the last version that I owned, OK, because I own the post". There's a concept of post ownership at that point, and it's totally legitimate for you to do that, and then the flip side is, you should have a page where you can see edits the people have made to your stuff, and we currently don't have that, right. So these are missing pieces of the system, so I tend to prioritize those a little bit higher than "X is bust" or "Y is bust", depending on how busticated it is, because without that feature in there, people don't really get to understand the system, so they'll be making requests and complaints that don't reflect the final system at that point, cause the system isn't there. So I want to put that caveat in, as that we have to build out the missing limbs. In a previous podcast Joel talked about how "Software can be missing limbs." I was like "Well, it can, but people will think you're a monster if you're missing limbs", right? like they don't see the person or the thing, they see "Oh, this is a monster, it's got three arms and it's missing a leg, and it's disgusting", right?

You have to able to see the outline of what it is to understand it and use it, so those kind of requests go first. The other thing that I look at is request people keep opening over and over. Currently paging would be one of those things. Geoff?

Dalgas: Thank you, thank you.

Atwood: And for a while, like, the user page was just kind of busted, like it wasn't showing the correct information on the user page, like the questions and answers you participated in. And I kept getting, like every day there would be two or three UserVoice requests on that, and I bumped that up and actually told Jarrod "Jarrod, look, we have to have this in, because every day I have to deny or duplicate three request on the same darn thing." So, if I'm getting lots of requests for something, than I'll definitely bump it up. So, I wanna be clear that, on UserVoice we really do listen, and I read everything that goes in there. So, do use that.  And that's actually how we prioritize everything. So the more noise we hear around a feature, certainly the more we bump it up on the list.

Now, the other part of that question is testing. It's, like, how do we test? So, let's talk, and I know Jarrod did some great work early on, basically single-handedly giving us unit testing, and I wanna talk about that a little bit. So, our current development structure, we have, obviously we can run the solution on our local machines with local databases and all that stuff. We have a development tier, which is currently hosted on the same server as beta. So, the normal process is: we would make a change, test it locally, make sure it doesn't break anything locally, check it in, at this point it triggers the CruiseControl which does the build on dev, so that forces an integration point at that juncture. Then we go and sort of play with it on dev, and make sure it works, and unit tests run, which I'm gonna come back to, and finally if that looks good, we deploy what's on dev to beta. So, there's sort of a three tier thing we do there, not that we don't make mistakes, cause we definitely do. Now, the unit tests that we have in, are pretty incomplete and I blame me for that, because I've really been pushing, like new features over going back and adding a lot of tests for features. But, I was very pleased that Jarrod set up a test project for us, and it does run on every build. So, if we do really heinous, that breaks tons of stuff, we will see that in unit test, and it will block the deployment on dev, so we can't even deploy, until we fix the unit tests. But, Jarrod, did you wanna talk a little bit about just setting up the unit tests, some of the decisions you made, cause... all that stuff?

Dixon: Well, at first, y'know, it started out with Microsoft's testing, like the Test Project, but I was kinda running into issues that, you know, you have to, in order to get MSBuild to run those test, you basically have to install like a full blown Visual Studio on your server and that's kind of a big no-no. You have to sort of, frankenstein up some dlls just to get it to be able to push databases through MSBuild.

Atwood: Right.

Dixon: So, I went to, y'know, switched back out, I've had, y'know, lots of experience with NUnit, so, I was comfortable with that, and I guess a lot of the stuff, if there are, y'know, some pretty intricate business logic, like the stuff around reputation, how reputation changes happen. Like, we had unit tests for that, but NUnit is one of the best open source projects out there. I can't recommend that enough. So other than that, hopefully , as we give more time, as we fix some of the more glaring bugs, we'll be able to add test for, like, the controllers. Cause right now, really, our unit tests only touch the models, and, y'know, I guess, I'm to blame for that too.

Atwood: No, no, no. That's all me, because I made that judgment call. It's like, we have to plow forward. And, y'know, the interesting thing to me about not having enough unit tests, which is where we are, we have, like, ridiculously small test coverage, is: your community ends up being your de facto unit test. Like, when your unit test failure is somebody opening a ticket on UserVoice, which is very traditional, and it would be nice to block, not block that, but to short circuit that, and actually be able to fix the bug before people even see it. Obviously, that would be ideal. But on the other hand, being able to deploy all the time and add new features without, y'know, doubling or however much additional work there is to add a test, not that we won't circle back and add this stuff in later, because I definitely plan to. But it certainly makes you a little more agile, I think, with responding to people's requests for features and things like that. I mean, if I had to go in and add, y'know, five new unit tests for this feature, y'know, would I really get it done in the same amount of time. And again, I gotta to be careful here, cause I'm by no m... I'd love... Testing is a very, very good thing. I just think it's a question of limited resources on a small team and where you apply them. And I certainly look forward, like, Jarrod, where would you like to see, give me some examples of... I can think of some examples of stuff I would like to add testing around. But, give me some examples of things you would like to add unit tests around.

Dixon: Well basically, like, around the badges, y'know, whenever certain criteria are met where you receive a new badge, we should actually have a unit test that would first the criteria to a user and run the badges, y'know, the function that grants badges and see if that badges are actually granted. Thats one area that we could have. Around messages, when you get new badges, or when system messages occur. Editing, when revisions should happen. But we do have them on a lot of a core functionality, we do, like creating a question and answer, a new user, we do have good unit tests there, so, it's more bells 'n whistles of the site, thats where we need some more tests.

Atwood: Geoff, what about you? I mean, have you done a lot of unit testing on your previous projects, cause I know one thing you mentioned to, which I thought was very funny, by the way, and deserves to be repeated is that when working with us it's like "Wow! You guys actually know what you're doing!" [Laughter] I guess you worked with some unsophisticated clients.

Dalgas: Oh, yeah, oh, yeah. It seems that this is, this is really the first project I've seen where it has some structure, some well formed organization. I mean it's a tribute to the ,obviously, team we have. It's great working with you guys. But, yeah, previous projects, where I go into a corporate environment, those are hardly ever thought of, at least in the places I have worked, and of course they're not developing mission critical type of applications, but even it seems when they are, they're just... there are a lot of weaknesses around testing.

Atwood: They're just, pretty much just randomly typing on the keyboard...?[Laughter]

Dalgas: Yeah, yeah. So, it's nice to see a good framework around what we have now. It's good.

Atwood: Right. Great. So, I definitely wanted to cover testing, so, that's good. So, lets see here.

"What was the biggest single technical challenge you had to overcome?"

This is from Michael. He is one of the biggest StackOverflow users actually. So I want to answer his question. Hum ...

Dixon: How did he get so much reputation ?

Atwood: He is asking questions and providing good answers. It's all.... Remember, reputation is all about other people voting your stuff up.

Dixon: That's right, Wow.

Atwood: So coming back to his question, Jarrod what would you say was the biggest hurdle on the project so far ?

Dixon: I'm gonna have to stick with the build, I mean that has been probably the most frustrating aspect. You know, just working with MS Build, trying to get certain things working on the server, they work perfectly locally but then trying to get them to work on the server that doesn't have everything installed, like a full blown Visual Studio installed. I think that Microsoft is actually, for the next version, they're actually going to try to make it so that MS Build can be stand alone and can run unit tests and can do database deployments, I'd say that right now that's given me the most heartache and angst it's just getting  automated builds up and running correctly.

Atwood: Right, actually in my previous job I would talk a lot about Team System and Build is a big part of Team System, and I used to realize that Build Engineering is almost like a full time job, I mean it is like almost, depending on the size of your team you want somebody that it's almost all they do, is think about like "how do we make the build?", "what should the build do?", "where should the build servers be?" There is a lot of engineering that goes into the build and it is legitimate software engineering it's not dummy grunt work, it's real software development.  So I totally concur there.  And it's so important, right--I mean what do we use everyday? We check in it goes to the build and the build ... This is a core part of the development process; it's hugely important.

Dixon: Yeah and mad props to Dalgas for getting Cruise Control up and running, he said something about Cruise Control and I thought it was something for pay, like you had to buy it or something. I did not even know it was open source and it is an excellent product. And I am amazed at everything it can do! I could even get rid of some of the MSBuild tasks in favor of just using Cruise Control tasks, so that was really nice.

Atwood: So I think build you would point out as the biggest challenge. What about you Geoff? Because you came in a little bit later, what would you view as the biggest challenge?

Dalgas: This was kinda a personal endeavour for me, because there's a few technology stacks that I had never worked in before.  One being MVC obviously which is just now kinda hitting mainstream.  Or it's been around now  -- what are we on, preview 3 now?

Atwood: They have a preview 4 but it's not as official as 3.

Dalgas: Yeah, so currently using 3.  So there was that learning curve.  Picking up the new concepts of using models, views, and controllers.  Along with using LINQ.  Its really my first endeavor into LINQ as well.  Its kind of a new paradigm for me.  Its one I really embrace and really enjoyed getting to learn, but all three of those new technologies right out of the gate was probably one of the more challenging, and again, more rewarding experiences for me.

Atwood: Right, so you would say the three would be LINQ, MVC, and ... what?  What is the third one. 

Dalgas: Uh, I don't know.  Being awesome? (laughs)

Atwood: Working with losers.  That would be the third one.

Dalgas: Yeah. (laughs)

Atwood: But I concur, and my answer would be a little like Geoff's, in that I had a vision of what this would be, but I felt pretty strongly that I was just kinda fed up with the whole Web Forms model; I had done several apps with Web Forms before in ASP.net and I just felt "there has to be a better way".  And I have been pretty pleased with MVC; there was a little bit of a learning curve, but I think I know it pretty well now.  But I think the bigger learning curve, honestly, is around LINQ.  I'm still... struggling... I think you guys have done a better job than I have.. these guys will go in and I will write a bunch of crappy SQL and they will be like "oh, you can just do it in LINQ" and they will just magically turn it into LINQ and I will be like "oh, thats how you do it".  Its been a challenge to kinda convert that, and I think the principle of least resistance, like I know SQL very well, so its really tempting for me to just go in and do everything in SQL, but on the other hand, for really simple queries, the LINQ is not hard, so its actually simpler to do it in LINQ because you can get your intellisense and just start typing.  So I have been really pleased with our technology choices.  I know there have been some criticism in some corners like, "why did you pick all this crazy experimental stuff for stackoverflow.  Why didn't you use proven technologies?"  What are we going to use?  COBOL? Punch cards?  [All Laugh]  Part of the fun of developing is picking things that are a little bit on the cutting edge.  I mean if you were really on the cutting edge you would be using Scala or Ruby, or something crazy like that.  So I don't really view these choices as really that controversial.  We are still in the Microsoft stack, we are still using Visual Studio, and it's kinda fun to be a little bit on the cutting edge.  Even though it can be a painful, but certainly we have run into some edge conditions with LINQ and MVC that were a little weird we have gotten past all of them.

Dixon: The interesting thing to me is it seems with this new MVC that Microsoft has done a lot of work putting together the Web Forms technology where you're really abstracted away from the - what really takes place behind the scenes, where in MVC you're writing the nuts and bolts of it, your URLs mean things; you can watch the code path, there's not this magic of view state that just happens for you, automatically You have to really pay attention to what's going on. So, it's good.

Atwood: Yeah, I agree and I think that LINQ is fantastic, I much prefer LINQ to any other Object-Relational mapper that I've used in terms of it's simplicity and in being linked to the language.  MVC as well, I think MVC, once you get it, seems so logical, like "Why weren't we all doing it this way?"  Also, I want to comment that Guy left this comment: "How are we going to stop Michael from hitting a bazillion rep points by the end of the month?"  I don't know.  We're going to have to code in like a limiter so that he doesn't get billion points by then.

Dixon: Let's make a special badge for billion?  I don't know.

Atwood: So somebody's asking about surprises, things that happened that you did not anticipate.  And I think to me one of the most gratifying things about StackOverflow is it has played out fairly close to the way I hoped it would, in that people are using it for good not evil.  One thing that has surprised me is early on we talked about one of the key things about a system like this because it's so low friction anybody can come in and leave comments we don't have this huge login barrier behind 20 walls of iron doors that slam closed how do you deal with spam, right? So from the very first day we'll have an offensive link we didn't know what to call it we went back and forth on a couple different names, but like a flag that people applied that said this stuff is evil, we might as well just call it evil right, evil!  This is evil.  And then if enough people mark something as evil, then it would disappear, poof!  Like if magical fairy dust had been applied to it or whatever, like "I'm melting," like holy water was poured on it or something. And what really surprised me that people were using, we called it offensive, Jarrod what did we call it first, like something else.

Dixon: It's... I don't remember.  We started with "offensive", but you changed it to something else.  It didn't have quite the power of "offensive"; it was kinda like fluffy.

Atwood: Right, and what surprises me is, people are using it on things which are clearly not "evil".  So either our messaging is bad, around what this function is for, or people just like tagging other people's stuff "evil".  And there is actually some user unrest around this, and I think understandably so, because people don't really like their stuff to be tagged "evil" when it's clearly not "evil".  My thing about the "evil" tag is, maybe the reason "offensive" isn't working too well, is because the system is like a bio-dome, we're too protected from the outside world at the moment, all we have is people who know what they're doing in here, and good people.  So hopefully--well not hopefully but--sadly, maybe once people come from the outside, you'll see more "evil", people try to really game the system.  But that is what the function is for, I would say that has been the most surprising thing to me, is that users are willing to tag other people's stuff with "offensive".

[42:01]
Because I wouldn't ... I mean it wouldn't have even occurred to me to do that. Because, you're supposed to ... there's two axis here. One is 'up vote, down vote'. And the other is the evil axis. Right? And these are very independent things. And you would think if you disagreed with something or think it's stupid or dumb or you just hate that person, fine, whatever, just down vote it. Right? And another adjustment we made was that 'down votes' used to be worth, uh, -5 to a person's rep. That went down to -2. And then finally -2 to the receiver and -1 to you (laughs) to really discourage people from doing this just blindly, um, and down voting people's stuff. And that seems to be working well. I've been pleased with the use of voting so far. But what about you guys? What have you found surprising? Like, Jarrod, what have you found surprising about the way people have used StackOverflow?

Dixon: Well, I guess in in hindsight. It really wasn't ... well, just exactly what you said ... it really, but, in hindsight, it wasn't surprising that people would, you know, tag other people's posts as offensive or down vote. Simply. But it was, it was really the users that had you know, accrued the most reputation that were being targeted. You know, in sort of like a petty competition. You know how, some users were jealous of all the rep that certain users had.

Atwood: Well, okay, we do have to clarify a little, because this is partially Leon Bambrick's fault—and I have no problem calling him out by name.  He intentionally—and I know Leon, I mean I've never met him but I know him through his blog and I feel like I know him, so he's kind of a "friend" —I think he was just playing with the system and trying to see what would happen if he did this, and he went around and actually downvoted other people's stuff to try to get ahead of them in reputation as sort of a game, and I think he was also tagging people's stuff "offensive."  In a way, this is good.  This is what the beta's about, right?  If everybody follows all the rules and doesn't do anything weird, then it's not a very good beta.  So I encourage this.  I think this is good.  But one downside of that was that I think certain people got really upset about that behavior, and I think Leon should have been more clear that, "hey, I'm testing the system," or whatever.  Just to give some background to what you're saying there.  So I think there was one specific person and one specific event… [laughter] … that triggered some of this.  And I want to put some of that blame on Leon's doorstep.  But go ahead.

[43:15]

Dixon: Right, well I'm sure that, you know, once, everything, you know, we go, fully public, that that would, happen anyway.

Atwood: Oh totally, I mean I agree and that's why it's good ultimately to have it happen I just wish he had been more open about the background of what he was doing, that's all.

Dixon: I guess, ah, another surprising aspect is the performance of the site. I've been very pleased with um, I mean, granted we don't have a lot of data and we don't really have that many users, but, you know, the, the server is holding up pretty well, you know, with everything that we're doing.

Atwood: Right.

Dixon: I'm very pleased with LINQ, you know, the, the queries that LINQ generates, you know um, LINQ to SQL has, has been very, a pleasant surprise, you know, just how well it's performing.

Atwood: Right. Right, we also had a question, and this might answer that question, it's like "You guys went with a WISA stack (Windows, IIS, SQL, ASP.NET). Do you think WISA will scale?" And I think, actually, compared to LAMP, and I think the funny thing here is, Windows scales like crazy! I don't know why it gets a bad rap, because honestly, if we had built this in PHP I can tell you first hand PHP is a dog for performance. [laughs] I mean there are ways to get around it and I don't want to offend anybody that writes PHP, but PHP is an interpreted language, right, I mean you can add some stuff onto it, like bolt-on these compiler things and stuff, but it is not a very high-performing stack dude, I mean, like, our stack, ah because it's compiled and you know, it's been through three iterations so far. I mean, our, we have a ok so, our server box is 8 CPUs. I, I don't think I've, it's rare we see any spike at all, even with all the users we have now and granted, we have a limited set of users etcetera etcetera but, I mean, but I don't think you can measure our load, at the moment, and, and we're not totally optimal either, I mean we're doing things that I think could be further optimized, so we're very much in a, a mode where we're giving up performance and it doesn't matter. So I want to say that: gosh, for Windows and ASP.NET you're going to get really high performing stack, I think much better than a LAMP stack actually. On, on, coz, if you look at Facebook and well, granted Facebook is a ridiculous example because they have like a billion servers but their scaling model and I also talked to Ted Dziuba of PressFlip and they, they used Amazons' ah, EC, like server-cloud stuff, and they have like 12 servers to do their stuff, and we're doing everything on one server and granted, I, I don't know that we'll offer scale to those levels, but I don't, it's going to take us a while to outgrow even this one server I think based on what I've seen so far, so, anyway, I want to give a shout out there. What about you Geoff, what have you been surprised about with the way people have been using the site or anything like that?

Dalgas : Well, I've actually kind of been watching the more social aspect of the site. And this is kind of one of those endeavors into a social realm where we're hitting people that are really smart really talented developers. But, just, I'm kind of, I'm interested in what they're, what they're asking as far as questions and what answers they're giving and how this reputation system's being played out, um, it's really interesting to me to see how, how people will be motivated by this. I thought before, that, well, what's the point? You know? Badges and points, it just doesn't make a whole lot of sense, so what? But, as I see more and more people get into this, they're really, they're really motivating the use of this site. And that to me is very interesting, very fascinating.

Atwood: Right. And, well, I think that's great, too,  because, the reason, all this stuff is there for a reason. Nothing on StackOverflow is there because, 'Oh, I thought it would be fun to have some stupid feature that is fun'. I mean, we actually put a lot of thought into how we build the site. And, Jarrod being involved earlier on and Joel as well. I mean, we went through several cycles with Joel, with Jarrod, um, and Jarrod, Jarrod and I spent, what was it? Like two weeks? Just like looking at all the 'Q&A' sites on the Internet, like all, all, the one, we signed up for all of 'em.

Dixon: I now consider myself, dumber for having done that.

Atwood: It's a running joke that Yahoo answers is just kinda a black hole of, of stupidity honestly.

Dixon: How do I get my baby's daddy to pay?

Atwood: Yeah, that's the classic.  Then in some email newsletter, they sent out recently their, their featured question was like "I don't read and write too good, how do I get edumacated?"  Pretty much.

Dixon: Yeah.

Atwood: So, and I think what works for us on StackOverflow is one, programmers. Programmers, and I know I'm preaching to the choir here. Programmers are smarter than the average person I think, because you have to be, because it's a really difficult field, because everything goes wrong all the time. Anyway, I do believe that programmers are cut from a slightly better cloth then the average person, the good programmers and I think Joel and I, having large audiences of very good programmers, again that honeymoon, honeymoon phase we enjoy the benefits of having a really talented audience basically. And the site is about those people showing off how good they are and sharing that with, with the other developers so they can become better. So it's very much to be a virtuous cycle, that we get into here. Um and, and hopefully that will continue. Going back to the answers thing as the audience gets wider and wider, does it get dumbed down? I don't know. Um, my hope is that StackOverflow will attract, the really good developers who are interested in sharing, um and letting other developers get better, through, through posts and questions and answers in a very low friction way that's easy.

So, Alan Le, who I used to work with at Vertigo, had a question. He's asking about the API for StackOverflow. I think it's too early to think about APIs at the moment. Certainly RSS is huge on our feature list. I think it's the next thing Geoff is gonna work on. Getting a feed of stuff that's coming in and your stuff and all that. So, I think for now, when you say API, think RSS for now, I know it's not really an API, but I'm gonna pretend like that answers the question. Then Alan asked how are we dealing with mischievous, he says "mischievous," users, spammers, griefers, marketers, etc. So, we talked earlier about the "offensive" link, which, that's really what that's about. And that's what that's supposed to be used for. So, hopefully people will use that. Also, we are gonna implement a CAPTCHA, that's another thing Geoff is working on. We're gonna have a jQuery JavaScript CAPTCHA, just sort of a basically force whoever is sending request to us to use JavaScript, and we're still debating the fall-back path, cause it is possible to use StackOverflow with JavaScript disabled. It's not a great experience, but it does work. And it's really tricky to come up with a CAPTCHA that works well with JavaScript disabled, that's not annoying to you. So, we're still debating that internally. So, my hope is... Oh and, then the third tier, that... I think, Jarrod, this is on your plate still: throttling. Like, I believe throttling is really central to any "mischievous" user, or evil users. Throttling is huge. If you can throttle people from doing N actions per minute, that disabuses you of so many possible problems. So, the beta is gonna run, I think,  for the full month of August, and by the time we come out of beta, I wanna be hardened, I'm gonna use security word "hardened", where there is not much you can do to us. Barring, like some international distributed denial of service. There's always something. I'm not gonna come out and say...

Someone: [through laughter] You don't wanna dare.

Atwood: I'm not daring anyone to attack us, cause I don't want you to want to attack us. All I want is, and I think this has come up too, we will not tolerate abusive users. I wanna be very a 100%, 1000% clear on that. If there are members of our community that are abusive, we will ban them, we will block them, we're gonna do all the things, and hopefully again, you guys will be able some of this, right? Like, you're not gonna have to call a moderator. You yourselves will be able to do this. One thing I've thought about, and we may have to end up doing, is, I think there should be ways you can vote against a user, right? Cause if you look at... Achievements are modeled on Xbox 360. One thing you can also do on Xbox 360 is you can essentially vote on a person. Like, say you are in a multiplayer game and somebody griefs you and does a bunch of stuff. You then vote against that person. So, I think we might need basically an "offensive" flag that you're gonna apply to users account. That's a little bit off in the distance, but again the goal is for people participating in this site to run it to some pretty high degree with very minimal moderator input, because moderators  don't scale, but the community does. So, that's pretty much my thought on that.

So, did you guys have anything, any particular questions you wanted to get to. I know, here's a funny one for you, Jarrod: "What is Jarrod looking at, and why is it making in so happy?" in the picture that I posted on the blog.

Dixon: [laughs]

Atwood: That's my fault. I told you to look maje..., I think the world I used was majestic. I was like "Jarrod, look majestic!". So, I was taunting.

Dixon: That was great, what's that called, Lookout hill, Lookout point? What's the name of that?

Atwood: Oh gosh, some point, some place in San Francisco, some tourist...

Dixon: ...and I was so amazed at how chilly it was there in San Francisco

Atwood: Dude, it's [garbled] almost May. It gets really really cold in San Francisco. If you ever travel to San Francisco, bring like your heavy jacket. I know you may feel ridiculous, but you'll probably need it, and you always have that conversation with people that are visiting "Bring you heavy jacket, even though it's the middle of summer". So, Jarrod, any other questions you wanna get to?

Dixon: One thing I wanted to... well, I wanted to say what a pleasure it's been working with ASP.NET MVC. I mean, I was so fed up with the Web Forms, y'know, having to deal with that. Some of the last projects I've been working on, I was actually writing HTTP handlers, just to render out the straight HTML, just so I wouldn't have to deal with all those stuff that goes along with Web Forms. And it's been such a pleasure, I guess... Is it Phil Haack, he's the project lead on that?

Atwood: He's on of... there's like twenty people involved.

Dixon: Big shout out to him and his team. So, it's been wonderful, I really enjoy, especially the changes they made from preview 2 to 3. Like, the controller actions returning an action result. That's been great, and it's been so great for doing AJAX, so now our controller actions, they can just easily render JSON back to the browser, they don't have to go through like any, intermediary steps, intermediate steps. So, just, that's what I wanted to say... [incomprehensible]

Atwood: Yeah, I agree completely. MVC maps much better to my mental model of how I want web stuff to work by far. It's not even close. To me it makes Web Forms totally obsolete, to the point I don't wanna touch it any more.

What about you Geoff? Any final questions you wanna get to?

Dalgas: To kinda put a step on top of that, RSS feeds are gonna be basically just another View on top of the... on the Models, Views and Controllers. So, that's gonna be an easy task because of MVC.

Atwood: Oh, Geoff, you always say it's gonna be easy.

Dalgas: It's beautiful. Simple. Just simple.

Atwood: So, Geoff, any last questions you wanna get to? Anything we didn't cover? Comments you wanna make?

Dalgas: Umm, let's see. No, I'm just really pleased to see the developer turnout for this, and continue on a...y'know, wanna have them keep on keeping on. There's a lot of questions here, I really wanted to know the answer to, and I never asked it? And so, I'm learning a lot of things, just by reading question here. It's been great.

Atwood: One question that was funny, kinda funny "How do we come to a consensus on decisions? Is it a democracy?" And these guys are laughing, cause they know, it's, I pretend that it's a democracy,  I pretend to listen to their feelings and thoughts and then I overrule them with, y'know, impunity.. .

Dalgas: Then I go and cry, I go sob in the corner..

Atwood: It is probably a little like... I know, I apologize, but I do... I can be convinced. I think it's the whole "strong opinions, weakly held" thing. Like, I pick a direction "No, we're gonna go this way", but you can really... you can talk me out of that, but you have to, like, convince me, you have to convince me, you have to talk me out of that and make me believe. You have to make me believe what you're saying is the better choice. But, it does happen. I don't wanna sound like a total jerk, or anything [laughs]

Dalgas: To have to justify why you're making a choice, verbally or technically or whatever, is actually really good exercise to go through. If people don't challenge you, then why are doing that you're doing?

Atwood: Oh, yeah. Great. I do wanna people to challenge me. I enjoy working with people that are like "Y'know, Jeff, I think this is really the wrong way, I think this is not a good solution, and step up a little bit and take a different direction" and defend it. I think it's great. I like that style of communication, and I think, obviously, I tend to pick people that are comfortable with that style of interaction over time.

And, then so, as we get closer to the end of the podcast here, I just have a final note to say how, I know people joke that it's called CrackOverflow. It's been really addictive for me to, like, work on a project now. As people start to use it more, like, I wanna work on it more, I feel like, as you put changes into production and stuff, you're impacting all the people that are using it, and you're satisfying their requests, and making it better, and growing an improved site. And, it's actually, hurting my other, my CodingHorror blog a little, because I'm actually really enjoying working on StackOverflow with Geoff and Jarrod. I'm a little bit addicted... if you guys are addicted to using it, I'm kinda addicted to building it, a little bit, and that's really all I have to say about that.

So, any final comments, you two guys, before we cut off the podcast here?

Dixon: No.

Dalgas: No. We can continue on...

Atwood: Speaking of continuing on, one of my goals... people ask about money and stuff. One of my big goals for money on this project is - I would love to make enough with StackOverflow to pay Geoff and Jarrod, so that they could legitimately work on this as a job. I would love that. So when you look at the site thinking "Jeff's gonna get a gold plated Hummer", I mean, I might get a gold plate[laughter]... but really, what I would like to do more than anything else is to be able to pay Geoff and Jarrod to work on the site, part time if not full time. I would love that, that would be wonderful, that's one of my major goals for this thing to be sustainable, and to grow it, and to have people that are attached to it, that love it, and care for it, and are into the community, right, and part of the community as well. So that's one of my long term goals for the site.

I would like to say thanks to Jarrod and Geoff for participating. I know it them puts on the spot a little bit, but hopefully you guys enjoyed this, people asked for it.

Dixon: Yeah, man.

Atwood: And I know I enjoyed it, and, y'know, maybe we'll do this again sometime in the future.

Dalgas: Was it good for you, Jeff?

Atwood: It was very good. So, thanks to everybody for listening. I do wanna cover... we do have a wiki of transcriptions of this podcasts, and if you participate in the wiki, and do like say one or two minutes of transcription, I will bump you in the beta list tonight. Cause, right now we have a backlog of about three thousand people, and I'm sending like hundred and fifty per night, so if you wanna get in on the beta like today or tomorrow or whenever you're hearing this, do a little bit of transcription work and then email me directly. And that's at the stackoverflow.fogbugz.com. And then for future podcasts, as we always say, email your audio questions to podcast@stackoverflow.com if you want to answer them in future, and I think that's it. Thanks everybody for listening, and thanks Geoff and Jarrod for doing this.

Dalgas: Thank you guys.

Dixon: Thank you.

Atwood: Alright. Goodbye.

[1:00:30]

Outro, Advertising