New
View
Edit
Info
Final Version (
Save and use this version
)
Page Title:
Tags:
<p>Ads, intro</p><p>[1:00]</p><p><b>Spolsky</b><b>:</b> Today is the day that we did not launch, although we planned to. But then... We'll wait for another week.</p><b>Atwood:</b> Yeah, well, the good news on that is that we did actually figure out what that problem was. <br /><p><b>Spolsky</b><b>:</b> Oh, oh, I want to hear, I want to hear, I want to hear.</p><b>Atwood:</b> Eh. So, it was a third party library. Indirectly, I mean. It's the third party library, and our particular use of it. It was <a href="http://logging.apache.org/log4net/" fc_href="http://logging.apache.org/log4net/" rel="nofollow noopener noreferrer" target="_blank">Log4Net</a>.<br /><p><b>Spolsky</b><b>:</b> Oh!</p><b>Atwood:</b> We were logging in such a way that the log.... during the log call was triggering another log call. Which is normally okay, but with the load that we have, eventually they would happen so close together that there's also a lock. So, there's two locks going on there. There's a lock of like disposing of the database stuff that's going on. Then there's lock of like actually writing to a file... <br /><p><b>Spolsky</b><b>:</b> Hm!</p><b>Atwood:</b> And... Huh... They happen in the opposite order, so it's like a classic deadlock, right. So, you release the lock on the database, then you release the lock on the file. And then the other call was doing in the other order. And they were happening so fast that... it was deadlocking eventually. And it was one of those things that would happen.. like... it was very intermittent, right. <br /><p><b>Spolsky</b><b>:</b> Right.</p><b>Atwood:</b> So we had to dust-off WinDbg.<p><b>Spolsky</b><b>:</b> How on Earth do you find things like that?</p><b>Atwood:</b> Well. You bust out Win Debug. One nice feature in Windows 2008, and I think this is in Vista as well. In Task Manager, you can right-click a task and take a dump of it.<br /><p><b>Spolsky</b><b>:</b> Yeah!</p><b>Atwood:</b> Like right there.<p><b>Spolsky</b><b>:</b> Aha.</p><b>Atwood:</b> So we took a dump of the W3Service process, and...<br /><p><b>Spolsky</b><b>:</b> Ha ha, take a dump.</p><b>Atwood:</b> Yeah, I know, any time you do this it's like.. It's like the territory for jokes. It's just...<br /><p><b>Spolsky</b><b>:</b> [giggles]</p><b>Atwood:</b> [laughs]. And then we loaded up a... Win... Debug.<br /><p><b>Spolsky</b><b>:</b> Windbg! Yeah</p><b>Atwood:</b> ...and then some .Net managed extensions you can, sort of load. You need like a cheat sheet to figure out what the commands are. And then you load the dump, and you load the manage tools. And then you can sort of just investigate all of the threads. You can take the "Show me all the managed threads." And then say "Show me what's the call stack was for that thread." And what we saw was like tons and tons of threads that were all going "Hey, I would like to log something..." And it was like "Hmmmm... [laughs]... Interesting!" Right, you have like 80 threads that all try to write something to the log. So... Right then we kind of knew where the problem was.<br /><br />And then somebody on Twitter actually volunteered to help us diagnose the dump. So I put it up on our server, and he a.... he nailed... he had a great description of it, like line by line, blow by blow of exactly what was happening. I mean, I'm... I'm competent enough to sort of figure out roughly what was going on, but he really knew this stuff and really helped us out, and I do appreciate that.<br /><br />[3:33]<br /><p><b>Spolsky</b><b>:</b> <span class="color03_highlight"><span class="color10_highlight">That's really awesome.</span><br /></span></p><b>Atwood:</b> So yeah. We.. we ripped out all the logging. I was never a huge fan of logging. I mean I guess there's a couple of philosophies on this. Like what's your philosophy on logging, like as you sit down to write a function, like would you add logging to it? I mean what's your ..what's your philosophy ? I'm curious before I go off on mine. <br /><b><br /></b><b>Spolsky:</b> I'd never .. no ... I never do... I nev... But you know I don't... I don't think I've ever worked on code that is sort of operational in the same way.<br /><b><br /></b><b>Atwood:</b> ah hm.<br /><b><br /></b><b>Spolsky:</b> Eh.. because we definitely eh.. put a lot more ... oh you know, you know what, I did, at Juno we used to have all kinds of logging.<br />The trouble is that my philosophy has always been that you .. you.. you have a tendency to wanna log everything. But then you just get logs that are, you know, a hundred megabyte per user and you get thirty of them a minute and it can't possibly be analyzed or stored in any reasonable way. So the next thing you have to do is to start culling your logs or just have different levels of debugging, where it's like in high debug mode everything is logged and in low debug mode nothing is logged. And... it's kind of hard to figure out what you really want in a log. You you know you know .. a lot of logs, like I think of the logging that we did in Juno, where people would call with a complaint and you try to figure out where this program is crashing. And obviously a log of the crash, that's easy. Ehm, but then there's some line above the crash which hopefully gives you a lot of information about where it happened. And there's some line you don't see that should have been after that, after the crash, but it never got there 'cause it crashed sometime before there. And essentially what you're doing as you're adding logging, is you're doing binary search, right, where you're sticking in like "well gosh, I got to here and then got to there. But there's an awful lot of code between point A and point B. So let's make an A you know half-way from A to B, log point of some sort". Then you put that in and then you eliminate 50 percent of the possible places to look for your crash.<br />Um, but I've never really been able to...<b><br /><br /></b><b>Atwood:</b> I mean that, ironically, to troubleshoot this hang, which turned out to be because of logging, we were adding more logging.<br /><br /><b>Spolsky:</b> [laughs]<br /><b><br /></b><b>Atwood:</b> The joke just writes itself! The joke just writes itself, right...<br /><br /><b>Spolsky:</b> It does... How many... How many third-party tools do you have... uhh... How many third-party tools are a part of the StackOverflow code base?<br /><b><br /></b><b>Atwood:</b> Well, okay, so... [chuckles] Uh, Dare [pronounces it as the English word "dare"] Obasanjo [pronounces it "oh-bih-san-ho"]... I don't know if I'm pronouncing it correctly.<br /><br /><b>Spolsky:</b> Okay, "Dare" [pronounces it "daray"]... Obasanjo [pronounces it "oh-bih-san-ja"]... It's "Dare."<br /><b><br /></b><b>Atwood:</b> Is it "Dare"?<br /><br /><b>Spolsky:</b> Yep.<br /><b><br /></b><b>Atwood:</b> Really... Okay, I didn't know that. Well, I've learned something. But he had a whole blog entry about how, you know, I had chosen to write my own sanitizer, and that was a very deliberate choice for me...<br /><br /><b>Spolsky:</b> Mm hmm.<br /><b><br /></b><b>Atwood:</b> ...for a number of reasons that I won't get into. But he was very critical of this, because, of course there were bugs in the sanitizer... <br /><br /><b>Spolsky:</b> Mm hmm.<br /><b><br /></b><b>Atwood:</b> ...which there were going to be, and to me, it's about, like, it's about your velocity; it's not about where you are; it's about where you're going, and we're gonna fix that stuff, right, and I'm making the sanitizer public as well, so other people can have a sanitizer that's not ten thousand lines of code, and ridiculous, and uh, so there's a philosophy there of building something that's reusable for everyone. Um, but I thought it was ironic, because he was talking about how developers should just pick a third-party library and go with it, and I think obvio... it's a balancing act, because we picked this logging library, right, which kind of caused a problem for us, right, I mean partially it was the way we were using it, but the way it was locking the files was a design issue in terms of the way Log4Net works.<br /><br /><b>Spolsky:</b> Right.<br /><br /><b>Atwood:</b> So I... I think it's a trade-off. I don't think it's always as clear-cut as "you should always pick a library" or "you should never pick a library," right? I think there's always some in-between there. So, for us, I'm definitely a minimalist—I don't like third-party libraries; I feel like we have a giant third-party library called "Windows," called ".NET"... huh... ASP.NET MVC is technically a third-party library. Um, but these are, you know, major vendor stacks. And I do feel like—as much as we talk about open source and stuff—there's a certain level of quality you associate with these major first-party stacks, right, whether it's from Apple or Microsoft or Sun or whoever. That may or may not be true, but hopefully usually is true: that these things are really heavily tested.<br /><br /><b>Spolsky:</b> There is definitely, yeah, there is definitely... I mean, there's something I've learned over the years, and, you know, I started out with working on the Excel team, um... The developers on that team had a motto, which was "Find the dependencies and eliminate them." You know, they had their own compiler; they would not use untested libraries from other groups at Microsoft even...<br /><br /><b>Atwood:</b> I love that they had their own compiler. That is so hardcore. I can't even, like, I could not even hang out with those guys... right... that hardcore.<br /><br /><b>Spolsky:</b> Hey, well, we have our own compiler, man.<br /><br /><b>Atwood:</b> Yeah...<br /><b><br />Spolsky:</b> Let me tell you why they had their own compiler: They had their own compiler because Excel was getting huge, and just compiled 8086 was just too large to fit on floppy disks and to fit in memory. You know, we were really trying to cram things in there. And so they developed a p-code compiler, which basically... you know, it's like bytecode. They called it p-code. This is a very old technique, and it compiled Excel into an imaginary machine, a virtual machine, which was a lot more expressive that an 8086, and had all kinds of additional features, and so the compiled code is about one-third the size, and in a lot of situations this made the performance a lot faster. So, for example, in those days when almost everybody was running programs off of floppy disks, the chances... Or no, not floppy, but the 3.5-inch, not-so-floppy disks. But the read time on those things is really really slow, so if you could launch your app—if your app was smaller at the time that you read it from disk—it didn't matter if it ran a little bit slower. The whole... the overall experience would be a lot faster. So if you could fit in memory without swapping, then obviously the whole thing would run faster, so it was worth doing this p-code thing for a long time, and about the time of Excel 5.0, the bit flipped on that and it suddenly became... suddenly everybody had hard drives, and nobody really cared about the size of the executable, and it was okay to have about a, I think, a four-megabyte executable instead of a one-megabyte executable, and so they got rid of that p-code back-end. But even then I think they had their own compiler for a while because in order to write really really efficient code, they wanted to be able to control... oh this is a long story.<br /><br />[9:39]<div><br /></div><div><b>Spolsky:</b> but a pointer on an 80386... for a while the 80386 was the target. On an 80386, a pointer consisted of (or even on the 8086 in general) a pointer consisted of two parts, the segment and the offset. So it's like "where do you want to start your pointer?" and then "what's your offset inside there?". And you couldn't just indirectly say "here's my pointer, just do something with this." You could, finally in 32-bit clean mode, but we didn't have that. What you had to do, was there was this thing called the segment register, and you loaded the segment register saying, "From now on my pointers which are 16 bits are going to be offset from this particular point. And the very loading of that segment register would cause all kinds of operating system traps to get executed and all kinds of interesting things to happen and it was a very, very slow operation. So if you were doing any kind of pointer manipulation, which you were cause it was C and everything was pointer manipulation, you wanted to load that segment register as infrequently as possible because that was a very, very expensive operation. And chances are, you're doing a whole bunch of pointer operations that are all in the same segment - for a while at least. You really want to be able to just load it once and then maybe do your next 20 operations with that as the base register. So all the Excel code had this assumption that they could do that - it gave them the ability to do that. And that made it just screamingly fast compared to the competition. I mean, I remember when Borland came out with Paradox for Windows, and they did not take this into account and they just used - it was C++ code, so they really had no choice but to use pointers for all their methods because it was all virtual tables and C++ objects. The net result was that they just used these pointers naively, pretending that the top 16 bits - every time they wanted to use a pointer, they reloaded the segment register, and that just made this app really, really slow. I mean, it took 90 seconds to start. You know Excel could launch in 10 or 15 seconds.<br /><br /><b>Atwood:</b> Wow.<br /><br /><b>Spolsky: </b>So this was this feature that they eventually got added to the regular Microsoft C compiler called base pointers, and I think then they stopped using their own compiler. But their philosophy was really not to trust anybody and to have control over everything so that there's some hope they can get it to work without having an external dependency. You know, I've sort of taken this with me a long ways, and every time I've failed to do that I've tended to regret it. Every time we've put outside technology into Fog Bugz we've regretted it. There's a lot of these excellent components, and they are really great components, that are made by vendors like .Net components and they're widgets like the cool calendar widget dropdown that you put into your web page and all that kind of stuff. And inevitably what I've found is that they are good enough for enterprise code, like internal apps that you're using at the insurance company, and they're just never good enough for the kind of app you want to ship that has to be perfect. Somehow there is something that's not commercial quality about them. You know it's fine if there's 20 people using it, and they're all using it the same way, it allows you to in fifteen seconds put a calendar dropdown into something. But then you'll get to some customer who says, "You know we don't start our week's on Sunday in my country". And you'll say, "Oh", and you'll find out this library doesn't have that feature. As a hypothetical example.<br /><br />[12:57]<br /><br /><b>Atwood:</b> Right! <br /><br /><b>Atwood:</b> I believe that's one advantage of some of the web stuff is because everything is just public facing by default. You don't have sort of this, this internal development ghetto effect. To be fair, any talk of dependencies, we have tons of dependencies, right? It's just a question of what dependencies you wanna take, I mean, jQuery is a dependency, right?<br /><br /><b>Spolsky:</b> Hhm.<br /><br /><b>Atwood:</b> We're using the WMD control, that's a dependency.<br /><br /><b>Spolsky:</b> Hhm.<br /><br /><b>Atwood:</b> There's these little add-ons for jQuery that...<br /><br /><b>Spolsky:</b> But you know what, if you found a bug in jQuery, you would just go, edit the source and you would be shipping your own private version of jQuery and problem solved. And it wouldn't be ideal, but at least you wouldn't be screwed.<br /><br />[13:32]<br /><br /><b>Atwood: </b>Right ... We have actually done that and let me actually give an example. So the WMD editor has a bug with international keyboards. Like there's no way we would have found this because we don't use international keyboards but obviously some of the people that use StackOverflow do and<b><br /><br />Spolsky: </b>Sure<b><br /><br />Atwood: </b>They were, I think, understandably very annoyed because they couldn't enter like right bracket. Which is like an important key particularly in markdown. That's one of the key, like one of the delimiters you use sometimes. <b><br /><br />Spolsky: </b>Yeah, the...<b><br /><br />Atwood: </b>Um. And they actually, huh, the prob...I'm still trying to get the source from from the authors so we don't actually have the source. What we have is well ob...not...I guess obfuscated is not the right word but minified javascript where they compress it down so all the variables are "a", "b", "c" and things like that. So it's not exactly fun code to look at anymore [laughs].<b><br /><br />Spolsky: </b>Yeah<b><br /><br /></b>[14:12]<b><br /><br />Atwood: </b>But somebody actually went through and actually found a little work-around and I feel bad because when they posted this I didn't realize that they actually found a work-around so I was like "Oh that's interesting" and I just didn't come back to it. But then somebody was complaining that this bug had stayed open for like three weeks and so I finally then went and then put in that fix. So you're right, having the source is great, because you can fix little problems that you run into and when you pick up components from the web (if you're talking about javascript) they're all sort of...um...you get the source by definition. I think that's actually one of the grea...I had a blog entry about this. That's one of the great strengths of the web. Everything is essentially open source by default, I mean, if you're curious about what Google is doing on you remember when uh you know maps came out and everyone's like "oooh". You know, maps, it's all this innovative zoom-in, zoom-out technology.<b><br /><br />Spolsky: </b>Yeah<br /><br />[14:57]<br /><br /><b>Atwood:</b> You could just view source and if-if you were, you know, motivated enough, you could figure it out, right? There wasn't like an executible that you had to decompile or anything like that. So I feel <br /><br /><b>Spolsky:</b> [garbled - interrupting] Flash<br /><br /><b>Atwood:</b> Go ahead<br /><br /><b>Spolsky:</b> Unless it's Flash all flash. I mean like Yahoo maps is all flash and you can't figure out what they're doing.<br /><br /><b>Atwood:</b> Oh, right. Right. Right. Right. Right. Well, that gets back into the whole we call it a rectangle problem - the browser where you have this alien rectangle [laughs] that lives in another universe and it pokes a hole into your dimension and then like this crazy stuff comes through and yeah. So it...it...on a related note, let's close out the topic. So the struggling with the deadlock put us back I would say at least four or five days. So in order to have a smooth landing, there's also a couple of features that I really desperately want us to get in before we open to the public like say a captcha for [laughs] I think that's kinda important when we go live. Um. So adding like a week to the schedule really helps us to have a smooth landing. I mean we could launch on the third. I mean we honestly could, but it would be a little desperate. We would be really flailing,<div>fixing things at the last minute.</div><div><br /></div><div>[15:55]</div><div><br /></div><div><p><b>Spolsky: </b><span class="Apple-style-span" style="font-weight:normal; font-size:15px">Yeah, no. We're in no rush, we can take another week. I thought that </span>the end of August plan was a little bit ambitious. I think we're both in the same position of really being on the fence, as to whether, I don't want to say on the fence but it's sort of a close call between we want to do the Hollywood launch, going back to last week talking about Aarons Swartz's thing, whether we want to do the Hollywood launch where everyone hits us at once and the world comes to an end versus the Gmail style launch where we just start taking a thousand people a day or give out invites or something to at least have some kind of control over the rate at which people come in.</p></div><div><br /></div><div><b>Atwood:</b> You know what I like now, what my philosophy of this has gone towards? It's almost like dating, where you don't really want to seem needy. If it comes up in the conversation, "Hey we have this website StackOverflow", if it's contextual, then talk about it. But maybe we won't have a whole post saying "Hey we're launching a new site called StackOverflow." Maybe not even do that. Sounds very counter-intuitive but just bring it up in the context of things you're discussing. Because already in Twitter and in email I'll want to reference things in StackOverflow because I have a problem or I found something interesting. It's just a natural side-effect of conversation that I have with someone. And to me it's completely organic, it's the way it's supposed to be. And that would maybe solve the problem of how we launch, like maybe people would find out about it organically as we have these conversations without us going "hey look at this new thing, poke poke, go over here and look at this new thing." It's just a thought, but I'm totally open to that.</div><div><br /></div><div><p><b>Spolsky: </b><span class="Apple-style-span" style="font-weight:normal; font-size:15px">Yeah. We have quite a finite number of people who listen to our </span>podcast and read out blogs anyway, so they're going to find out.</p></div><div><br /></div><div><b>Atwood:</b> Right. But the site is very sticky and very social, too. Along those same lines I just emailed Joel today. We know we've succeeded already in beta, and do you know how I know? Because we have a whole blog dedicated to hating StackOverflow on the Internet. So you know you're successful when that happens, it's like a stamp of approval.</div><div><br /></div><div><h4>Spolsky: <span class="Apple-style-span" style="font-weight:normal; font-size:15px">It's not even public!</span></h4></div><div><br /></div><div><b>Atwood:</b> I know. It's a huge success! If there are people who hate you and it's not even public then you're tremendously successful.</div><div><br /></div><div><p><b>Spolsky: </b><span class="Apple-style-span" style="font-weight:normal; font-size:15px">These people they're going to the backlash state before we even got to </span>the hype stage. Come on you guys! Backlash comes after the hype, that's why it's called backlash.</p></div><div><br /></div><div><b>Atwood:</b> And a funny thing that happened on the blog too. The way we secure StackOverflow is somewhat intentionally naive.</div><div><br /></div><div><h4>Spolsky: <span class="Apple-style-span" style="font-weight:normal; font-size:15px">For the beta.</span></h4></div><div><br /></div><div><b>Atwood:</b> Right, for the beta. The site is not supposed to be secure at all, it's supposed to be totally public, even in the sense that you can just walk up and type stuff in, literally, that's what the site is like. So securing it is just completely counter to everything the site does, and we even struggled to secure the site initially. How do you secure a site that's not designed to be secure? Do you want to write tons and tons of code around authentication? So, the minimal solution we have is basically a very simple cookie-based solution. And I love on this particular blog he found that out (I presume it's a he, it's always a guy), and he's like "look how lame their security is, they totally don't understand how cookies work. They don't understand security at all. You're going to trust them to build a website?" It just made me laugh because it very much missed the point of that whole thing.</div><div><br /></div><div><h4>Spolsky: <span class="Apple-style-span" style="font-weight:normal; font-size:15px">Don't even respond. Why are we even talking about them?</span></h4></div><div><br /></div><div><b>Atwood:</b> I know, I know. Some of the criticism is actually grounded. If there's something useful that comes out of it I will use it and I will respond to it. And it's not vitriol yet. He says it's a blog about flaming but it's actually somewhat reasonable. As long as it stays reasonable I have no problem responding to it. I'm not going to point it out or list the URL or anything like that, but we absolutely we are listening.</div><div><br /></div><div><p><b>Spolsky: </b><span class="Apple-style-span" style="font-weight:normal; font-size:15px">blogging-harmful.blogspot.com. Complete waste of time, but you know if </span><br />we get people to care about us whether it's positive or negative that means people care about us.</p></div><div><br /></div><div><b>Atwood:</b> Exactly. That's my point. If nobody cares, that's the real loss.</div><div><br /></div><div><p><b>Spolsky: </b><span class="Apple-style-span" style="font-weight:normal; font-size:15px">That's the real failure, exactly. If you can't get anyone to care one </span>way or another about what you've done. For example, this website blogging-harmful.blogspot.com is going to disappear without a trace. Even though I promoted it on the podcast it's going to make it all the more painful when nobody...</p></div><br />[20:26]<br /><b>Atwood:</b> But people who do something like that, they don't want attention or anything, they're not feeding on it at all, they're not interested in things like attention at all. The work is its own reward, it doesn't matter if anyone is looking. Have you seen that thing on the ... I meant to blog about this but the whole concept of just, not looking at things, to basically discourage them. Or that, conversely, looking at things encourages them, like the whole Paris Hilton thing and just talking about these things over and over incessantly actually reinforces the whole trend. There was a series of children's books I don't know if you've heard them, they're called: "The Great Brain". It's set in Utah -<br /><br /><b>Spolsky:</b> Yeah<br /></div><div><p><b>Atwood:</b> - at the turn of the century. I got these books as a kid and I was totally obsessed with them, because the Great Brain is all about a family. I don't remember the family's name, but there is one central character J.D. who is the Great Brain. Essentially he's always thinking up ways to, essentially social engineering before we had that word in computer circles. Basically getting people to do what you want them to do, completely of their own volition. The Great Brain is basically this genius of a kid, who is using all these social engineering exploits to get away with all this crazy stuff. In that family, if they found out the Great Brain was doing this stuff then of course he would get punished, but the ultimate penalty was what they called the silent treatment. The silent treatment meant that nobody would talk to you or acknowledge you for a certain period of time. They would give you food and stuff, but they wouldn't talk at you. It was just stunning in the book, you don't really think about this stuff as a kid (I was like 10 or so), how desperate it is as a person, as a social being, when nobody will acknowledge you. How profoundly affecting that is, right? Even the Great Brain, as a smart kid, hated the silent treatment and would do anything he could to avoid getting the silent treatment because it was just such a brutal penalty. I remember Jason Kottke talking about an episode of The Simpsons where these animated statues came to life, and the way they got rid of them was they started chanting "Just Don't Look!"</p></div><div><b>Spolsky:</b> (laughs)</div><div><br /></div><div><b>Atwood:</b> This is led by Lisa Simpson, who says "Just don't look at them and they'll go away!"</div><div><br /></div><div><b>Spolsky:</b> (laughs)</div><div><br /></div><div><b>Atwood:</b> It's amazing how powerful that philosophy is. If there are things happening you don't like just don't talk about it or don't give it any attention, and look at the things that you actually care about and actually want (to have happen).</div><div><br /></div><div><b>Spolsky:</b> Thats right. That was like the Clinton administration's policy on Rwanda.</div><div><br /></div><div><b>Atwood:</b> (uncertain chuckle) Well thats the issue of social injustices, which I think is a little bit different.</div><div><br /></div><div><b>Spolsky:</b> There are definitely people who are attention seeking, but you know what? That the thing about trolls, "don't feed the trolls" or whatever. You know what, trolls are doing a great service to the Internet, they're making it entertaining and interesting.</div><div><br /></div><div><b>Atwood:</b> Absolutely. There's definitely been, and I think I referred to it in the previous podcast, "youthful experimentation" at StackOverflow. We're still in fact seeing that.</div><div><br /></div><div><b>Spolsky:</b> (chuckles)</div><div><br /></div><div><b>Atwood:</b> We're still seeing people just try stuff to push buttons and see what they can get away with.</div><div><br /></div><div><b>Spolsky:</b> One of the things that will happen, and it happened with IRC all the time, when people get addicted to a social technology or something. They're addicted to the site, the love it, they come and they answer all the questions, but at some point there is just not enough of the main content that everybody is enjoying for them to entertain themselves. They have to find ways to entertain themselves for the next three hours after they've spent the 8 hours doing the regular thing they were supposed to be doing. So on IRC they would start these little flame-bot wars and they would write little bots to protect themselves against the wars, and they would try to cause splits so that they could take over somebody's nickname, and that kind of stuff. They weren't doing the kind of stuff that IRC was for, which was chatting, they'd taken it to the next level because they ran out of people to chat with and that became boring. So they started kind of attacking the system itself.</div><div><br /></div><div><b>Atwood: </b>Right, and we're definitely looking at that. I mentioned that CAPTCHA thing, and thats the next piece of the puzzle that has to go in. We have rate limiting mechanisms. One of the early things that happened to us was -</div><div><br /></div><div><b>Spolsky: </b>Flooding.</div><div><b><br /></b></div><div><b>Atwood</b><b>:</b> - somebody wrote a bot that would just revise posts every minute to keep it on the top of the stack. Actually there is a certain amount of people doing that still, which I'm trying to discourage. They way I like to discourage things is, where possible, creating rules in the system that make that behavior not desirable. Not negative, necessarily, but things happen that make it not worth much to you. So let me give you a specific example in that scenario. You have the user who's just editing their own posts every three hours so that it's always on the top of the stack. We have this concept now, it's actually implemented though I talked about it in previous podcasts, of the community owned post. Because one of the great divides in StackOverflow is that we have this ownership system where you get voted up and down, your content gets voted up and down, and that affects your reputation. You own stuff, so when you post something you own it. Then you contrast that with the Wikipedia model which is that nobody seems to own it, and we're trying to do both of those things. At the transition point we came up with a couple rules. The initial rule I had was that edits by four different people will cause a post to switch from being owned by Joel, for example, to being owned by the community user. At that point you don't lose any reputation that you got up to that point, but any future upvotes on that content don't go to anybody they go to the content. I think this is the way it should be. Ultimately you're voting on the content more than the person anyway, so hopefully people are ok with this. Seeing that people kept editing stuff over and over I bent the rules a little bit and said ok, if you edit your own thing more than N times then it also becomes a community owned post. There is no real value to the user, in terms of getting additional reputation, to bump stuff up to the top of the stack anymore because if you edit your own thing enough you won't get any reputation from it. It behooves you to only edit it once, or however many times you need to edit it but hopefully no more than once, and just let it sit there and have people find it organically and naturally the way it is supposed to happen.</div><div><br /></div><div><b>Spolsky</b>: That will happen more when we're open to Google for searches. I think right now one of the problems with StackOverflow that some people have been experiencing is that they ask a question, and it's a little too esoteric to get a response right away, and then it disappears from view for a while. Once the site has a much larger critical mass of people and is searchable by Google, those questions will naturally have people come to them so they won't need to try these little tricks to get them in front of people again to get an answer.</div><div><br /></div><div><p><b>Atwood:</b> I also try to copy a lot of things I've seen online that have been successful, like conventions. Let me give you a specific example: PHP-BB, and I'm sure there are other web discussion boards that do this too but PHP-BB is the one I know, has this editing convention. When you post you can edit your own posts, and I noticed that when people are using PHP-BB that right after you post something you'll always notice some goofy mistake that you made, like immediately. This happens to me 9 times out of ten I'll post and think "oh, I should have talked about this" or "I missed that word," so you immediately go in and edit. At a certain threshold these are not treated as real edits, they're treated as just going back in time to pretend that it is the post you originally made. It doesn't kick off the whole auditing trail of you having edited it 50 times. One of the first things we did in StackOverflow is actually implement that. I remember talking to Geoff Dalgas about that, and he's like "why do we have to have this?" I said, "you don't understand, this feature has to be in there on day 1 otherwise we're going to have so many revisions that are just in the first minute or two after posting and are just silly little things that are being corrected."</p></div><div>[28:20]<br /></div><div><b>Spolsky</b>: You not actually recording the revision? You're not doing the diff thing?</div><div><br /></div><div><b>Atwood</b>: [Not] within the threshold. Right now the threshold is actually 5 minutes. So up to 5 minutes after you post, if you edit your own stuff. Now if I go edit it, it's a real revision.</div><div><br /></div><div><b>Spolsky</b>: Now...that's OK, except that you sort of run the risk that the historical record is...if somebody posts something and it's seen as being...</div><div><br /></div><div><b>Atwood</b>: Sure, if the threshold is like 2 hours, right?</div><div><br /></div><div><b>Spolsky</b>: ...5 minutes. Let's say I ask the question "Is Jeff Atwood smart?" (laughs) And then you reply saying "Yes! Absolutely. Definitely." And then I go back and change "smart" to "stupid" one second later. "Ah-hah...you said you were stupid!" And there's no track record. Not only that, but everybody can say "There's supposed to be an audit trail here. Let's look at the audit trail...oh, look! It wasn't edited!"</div><div><br /></div><div><b>Atwood</b>: Right. No, that's true. That does leave you vulnerable. And I think that's where you come to playing with the rules, and seeing what happens. Another example of setting the rules...you can't always anticipate all the side effects of the rules you're going to set. And you don't always get the behavior that you wanted out of the rules that you set either. You have to be very, very careful. So when I instituted the rule of 5 edits by the owner causing a post to go to community mode, the next day somebody complained about that. Actually this was today. Somebody posted a thread on "What are your favorite developer magazines?" And because it's not in my opinion a great question, because it really can't be answered and it's very subjective. It's not a great question is the bottom line. So it got edited a lot, immediately. People started editing the tags. Somebody changed the title. The net effect was within 45 seconds of this guy posting it, it was in community mode. And he was kind of complaining about that [saying] "Look, I've had no chance to get any rep from this post at all." I kind of agreed, I can see where he's coming from--although again, not a great question.</div><div><br /></div><div><b>Spolsky</b>: You shouldn't get any credit for a not great question.</div><div><br /></div><div><b>Atwood</b>: You could argue that this is totally correct behavior. I can see where he's coming from, and one thing I thought of was having a certain time window where...it's four edits by four different people, but it has to be more than one hour. Or whatever time interval.</div><div><br /></div><div><b>Spolsky</b>: Ugh. Too much tweaking. You know what I would say? Just tell people "Listen, if you want to ask one of these poll questions like 'What's your favorite developer magazine?', that would be a good question if you phrased it in the following way: 'What is your <i>single</i> most favorite developer magazine?'" And instead of just replying, see if someone else has already given the answer, and vote it up. And then what you get is a ranked listing of everybody's favorite developer magazines, in order from most favorite to least favorite. But if you say "What is your favorite magazines?" and people are answering with three, and they're discussing some magazine that they subscribed to a long time ago and it's just chatter going on in there, the question becomes a mess and it just doesn't fit the StackOverflow data model. </div><div><br /></div><div><b>Atwood</b>: It doesn't. That's right. And I think that's the challenge, is also getting people to understand what is the StackOverflow model. </div><div><br /></div><div><b>Spolsky</b>: We can do polls, and we should do polls, for things like "Should I learn C? Yes/No" (laughs) There you go...everybody vote.</div><div><br /></div><div><b>Atwood</b>: One advantage, and I haven't put this in yet, but you will be able to opt-in to the community mode so that you don't get any rep. So then you <i>could</i> post things that are effectively voting questions. Let's say we had "What is your favorite developer magazine?" And everybody that posted in it went into community mode immediately, all those votes would go towards the magazine, no you. I said "BYTE. BYTE is my favorite programmer magazine. It's a classic." So everybody that voted for that would not be voting for me, they'd be voting for BYTE. I wouldn't get any reputation from that at all.</div><div><br /></div><div><b>Spolsky</b>: I could not program my Atari 800 without the great articles I've gotten from BYTE magazine. [Actually I had an Atari 800--back in 1981 or 1982, and while BYTE remains my all-time favorite programming magazine--especially in the early days before it went downhill, the magazine that helped me master the Atari 800 was Compute.]</div><div><br /></div><div><b>Atwood</b>: (laughs) There you go. I think I've said this before, but I think it bears repeating. I really enjoy working on StackOverflow--it's a really fun social experiment. Because we're getting useful stuff out of it for the most part, to me that makes it workwhile. You call that tweaking, but to me if it's a tweak that results in six months from now hundreds of much better questions then it's worth doing.</div><div><br /></div><div>[32:52]</div><div><br /><b>Spolsky</b>: My feeling is that if the tweak is a little bit subtle and little bit weird, people don't know quite what it is doing. Like that thing you were mentioning earlier about how if you do your own edit within about 5 minutes it doesn't get into the history. See, right then you are doing something that wouldn't be what people would expect. I mean, they might be able to learn that that's the way that it works, but it isn't what they would expect. They would expect that you either see the history, or they don't see the history. It would never occur to them that you'd do something more subtle than that, and therefore, they'd always assume the simple model, and therefore they may have usability problems because they don't understand what the app wants from them. You know what I mean? Basically, usability problems always occur at the intersection, of the user not understanding how the program model works. The program has a model as to how it works, and the user has some understanding as to how the program works, and when those are different that's when you have a usability problem. It may be small and it may be subtle, but that's where you have a usability problem. Your best hope, if you're setting things up and you're saying "Hey I going to give you points if you do X". and everyone does X, and X is something that you want, and you told them that that's what you'll give them points for, and it's obvious, then that's great. But if you're going to do something where you're doing something non-obvious, or a little bit tricky, or you're creating a little bit of a conflict between how they think it is going to work and how it really works, then in all those cases the best you can hope for is that they will accidentally stumble upon doing some behavior that you want them to do. Because of their misunderstanding will cause them to accidentally trip into the particular dark hallway that you want them to go down. That's the best you can do. I think you're always better off striving to make it that people understand what's going on. A lot of times that may mean that you can't have behavior that isn't clearly visible. There's going to be some kind of behavior in there like how you earn a badge, or when wiki edits don't show, there has to be some extreme visibility in the app. It has to explain itself a little bit, so that people understand what it is doing.<br /><br /><b>Atwood</b>: Right. Normally, I'd totally agree with you. I think this is a little bit of an exception just because again it came out of <a href="http://www.phpbb.com/" fc_href="http://www.phpbb.com/" rel="nofollow noopener noreferrer" target="_blank">phpBB</a>, and these other very long well established messaging systems. Were harvesting these ideas from Wikipedia, message boards, and wherever I've been online and I've had a community that I thought really worked. I try to steal those ideas, and fold them into StackOverflow. So I think it is a proven idea that works. It's just a peculiarity of human behavior that you're always going to make mistakes immediately after doing something. So you have that little cusion. It's kind of a special case based on human behavior. And then two, I think we have a community --<br /><br /><b>Spolsky</b>: What does it hurt to have at least the transaction history shown?<br /><br /><b>Atwood</b>: Well, because it becomes noise. It's really tiny, simple, edits. <br /><br /><b>Spolsky</b>: I don't know.<br /><br /><b>Atwood</b>: Well again, we're not exactly doing... we're a hybrid, right? So, we're in between. So I think we harvest those ideas from different places. I don't think --<br /><br /><b>Spolsky</b>: Hey I got a question. Heh. I got some questions. Want to listen to some questions?<br /><br /><b>Atwood</b>: Yes.<br /><br /><b>Spolsky</b>: Sorry, you'll see why I'm laughing in a minute.<br /><br />[36:00]<br /><b>Caller</b>: Hi, this is Trey Jackson from Corvallis Oregon. Joel, It's great hearing your ideas. Jeff, It's increasingly frustrating that you don't listen to a single argument that Joel makes during a conversation.<br /><br /><b>Spolsky</b>: Woohoo!<br /><br /><b>Caller</b>: My question is, assume for a moment that StackOverflow becomes a great success ..<br /><br /><b>Spolsky</b>: wait, let me just play that again (Laughs)<br /><br /><b>Caller</b>: .. and gets nothing but good questions and even better answers. How do you intend in keeping folks interested in answering questions when the sheer volume of those questions gets so high. People will google for answers but they'll google for questions ..<br /><br /><b>Spolsky</b>: (Starts playing question again)<br /> <br /><b>Caller</b>: Hi, this is Trey Jackson ...<br /><br />[36:32]<br /><b>Spolsky</b>: I have to play it again and again. (Laughs)<br /><br /><b>Atwood</b>: (Laughs) That's how much you like it, you wanna play it twice. Wait, I don't understand, why was I not listening to your argument there?<br /><br /><b>Spolsky</b>: (Laughs)<br /><br /><b>Atwood</b>: I don't get that. I totally was listening, I just think in this one particular case, I mean in the general case I agree the system should be totally discoverable and consistent. But I just disagree that in that particular case It should be. I think it's ..<br /><br /><b>Spolsky</b> : Well see there's a difference between listening and just accepting what I say and actually that was the reason that I wanted to play that ?? actually is that my problem is usually the opposite, is getting people not to listen to me. For those of you that don't know,for my day job I'm the CEO of a software company. Ahm, and we've got about 20 people and my biggest problem I would say, well actually my biggest problem is dealing with New York city landlords but, my second biggest problem is getting people to not do what I say just because I said so. And it's very hard for them because you know, I'm the CEO and their boss, and I probably have more experience than most of them and when I say do it this way, ah, you know there is a real tendency to try to do it that way. Ah, and it's very easy, it's the easiest thing you can possibly do is just do what you boss says because you have no responsibility over whether it was successful or not because you did what you were told and you can just lean back and just do it that way instead of making the really, really hard decisions yourself. And so I would say my biggest problem is making sure that the developers at Fog Creek understand that that decisions are in their hands, and that If I say "do it this way", I want them to, you know, take that for what it's worth but not to do it blindly and in a lot of cases I just don't have as detailed an involvement or as much intricate knowledge of what's going on. So ahm, I mean to be fair I, you know, I spend a couple of hours a week on StackOverflow or a couple of hours a day messing around on the site. I spend a couple of hours a week talking to Jeff on this podcast, ahm, but Jeff is spending every minute of every day on this and he's much more deeply involved in what's going on and he knows a lot more about how things are and how they work and how they should work, ah, than I do. And, ah, so I may have some general experience to give him but I'm really happy that Jeff makes his own decisions and takes responsibility for them and doesn't just kinda blindly do whatever I tell him because that would be a mess, that would be the person with the least information making the decision. If anything, you know,it takes me years to train a developer at Fog Creek that they should push back when I say stupid things. Ah, or when I suggest, ah, doing things they should make their own decisions, the decision is going to be up to them. I try to remember to always say "look you're going to decide, but here's my opinion". Ah, but it's hard for that to come across as anything other than "you're going to decide, but you're probably going to do it my way because I say so". And ahm, that's really important to me so, ah, this is a critisism that makes no sense .. whatsoever. So there. So Jeff, why don't you answer the part about the umm, the second part. Do you remember that part? <br /><br />[39:28]<br /><b>Atwood:</b> Well, right. I want to talk a little bit about the relationship. The parts, I think people aren't really seeing is the discussion that went into building StackOverflow before we even started the podcast. Where Joel has these message boards that he's run for a long time. Joel Knows a thing or two about building a community, rules that make sense, and really big picture stuff.<br /><br />So a lot of the design that you guys are seeing today in StackOverflow was really vetted by Joel early on. So I think we kind of agree on central tenets but getting down to sort of brass tacks of you know "Do A, Do B or Do C" There's a lot more harmony than you may hear on the call, necessarily. It's been a great relationship. Joel has been very intimately involved. I didn't know what to expect early on, but Joel has been very hands on. It's really been a pleasure. So hopefully you guys are, if not hearing it, then believing me when I say that there are much more harmonious aspects to the relationship as well.<br /><br /><b>Spolsky:</b> We're always trying to figure out ways to get into big fights on the podcast itself to make it more interesting. But that's just because we are trying to get our listenership up from the current low, two-digit numbers.<br /></div><div><br /><b>Atwood:</b> That's right, your dad needs to know other people are listening to the podcast that his son is on. So the other part of the question was about I think just sustaining interest in the questions over time..<br /><br /><b>Spolsky:</b> And answering the questions. How are we going to get people to answer questions? It is certainly easy to get people to pose questions, or to show up on Google when we have an answer.<br /><br /><b>Atwood:</b> Right, well everything that's in the site is in there for a reason. Like the whole reputation system, the way you sort of earn reputation is by answering questions. That's one of the best ways to get reputation. People are actually complaining that when you ask a question, you don't tend to get as much reputation from asking as answering, and I think maybe that's the way it should be. But currently even in the closed beta, people are constantly emailing me amazed that they are getting such great answers and so quickly. So even with the closed sort of petri dish that we have of the beta users, it's already working extraordinarily well. I expect that to scale, like in the Clay Shirky book where he's talking about Wikipedia is just attention divided by a million. You have a million developers with 2 or 3 minutes and we want to have our site be so low friction that they can come on in 2 or 3 minutes and find something they are interested in and that they know about and that they can be authoritative on and then just file a fragment of their knowledge away on our website. No signup , nothing complicated, very very easy. So we think as it scales,you're going to see even more questions get answered. It will be even better than the beta. So the fact that it's already working, to me means that it's definitely going to work and as we scale out it's going to work even better.<br /><br /><b>Spolsky:</b> Sometimes people even put a question and answer up for their own selfish reasons. You know, it's just a good place to record for their own benefit so they remember how they solved this particular problem.<br /><br /><b>Atwood:</b> And I totally encourage that. I plan to use the site that way, as a matter of fact.<br /><br /><b>Spolsky:</b> As a notebook where you put down some little notes as to how you solved a particular problem, no matter how obscure, and other people will come in and improve it.<br /><br /><b>Atwood:</b> Absolutely. Again the different inspirations for StackOverflow like blogging, Wikipedia, discussion boards, we're trying to hybridize all those things together and this one comes from the blog where I'll blog about something, I'll say "Oh, I thought about this and then I did this" and then I learn so much from the comments I sometimes totally change my solution based on the comments. So even if you come in and post "Okay, here's how I did it" you may still learn a better way to do it from other people seeing it and being able to collaboratively add comments to the bottom of what is effectively a blog post, right? But with no friction, without actually blogging. So I think it totally works and I encourage that.<br /><b><br />Spolsky:</b> Yea, we're definitely seeing it working and I think this question is sort of a non-starter. If it's working with 3,000 people, I don't see any reason... You know, we're getting answers with 3,000 people I don't think there's anything different about those people than... there's nothing unusual about those people, they're not... I guess they're early adopters, but... You know there's no reason why they would have a stronger tendency to be willing to go in and play with the site and answer questions just for fun. I mean there's a lot of people that are doing it because it's fun to answer questions.<br /><br /><b>Atwood:</b> Well, I think too we're harnessing sort of the natural tendencies of the software developer. We talked about this earlier on the call and I didn't actually get to answer it,But one of the reasons that some of the complexities of StackOverflow we can get away with that.. We try to keep it as simple as we can, but we have reputation, there's rules around the reputation... Developers love this crap, right? They spend all day working with rulesets all day long. This is what they know, so they love that crap and the fact that it's a game where you get to show off how much you know. Have you ever met a developer that wasn't dying to show you the best way to do something, like the only true way that anything should ever be done is the way that they do it of course? So we have a community of people that I think are naturally inclined to be very good at this kind of behavior. And we just want to channel it to a positive direction, where everybody's working together, not against each other. That's the only caveat I'll put around that. But I think also our audience helps us succeed because we're building for a very specific audience. We're not building Yahoo Answers, please don't go there, because you'll be very depressed when you leave that site.<br /><br /><b>Spolsky</b>: You'll be shocked. "How do I tell if my girlfriend got pregnant?"<br /><br /><b>Atwood:</b> It's just such a big audience and I fell bad because we've actually had a number of people from like WikiAnswers and Yahoo Answers and people that do sites that are similar have actually contacted me and I've been very friendly with them and very open about "Hey, we're just building a site, we're not competing." I want to learn from them,they're going to learn from me.<br /><br /><b>Spolsky</b>: We're just so much more narrow than Answers.com or, Google Answers is gone, Yahoo Answers, who else has one? Does Ask.com have an answers site?<br /><br />[45:15]<br /><b>Atwood</b>: There's tons of these sites out there, there's AnswerBag,<br /><br /><b>Spolsky</b>: Right<br /><br /><b>Atwood</b>: There's tons of these sites out there. There's Answerbag... there's tons. You'd be surprised how many Q&A sites there are out there. And I found on Live.com, Microsoft has a site like this. I went to Live.com, I don't even remember why, I guess it's ironic when I go to Live.com: "Oh, look, I'm not on Google, hahaha!", right? But there was a Q&A site embedded in there, you could click on it, it was answers.live.com, I think. And I'd never heard of it, but yeah, go figure, there's lots of these sites.<br /><br /><b>Spolsky</b>: Yeah, it seems easy to build, but to get the right audience... The other thing that's a little different here is that, to some extent, to be completely honest, we're not building something new from scratch. What we're trying to do is divert a lot of activity that's already happening on these sites behind pay-firewalls or that charge monthly fees, where people are basically contributing answers anyway, even though nobody can see them except for the people who pay for the memberships. So there's a lot of this stuff going on, it's just that it's happening in a kind of inferior way, in our opinion. And it's sort of surprising that developers haven't been able to build something for themselves. The people that get the best tools on the Internet are developers, because they build themselves their own tools. And the thing that Usenet is best for, or was best for when Usenet was popular, is getting answers to your programming questions. And, obviously, the comp.* group of discussion groups on Usenet was always the strongest of everything. So, developers do build things for themselves, but, for some reason, they get good enough and they kind of lose interest. And in the specific area of Q&A, the-state-of-the-art, there were sites that were good enough. But they weren't good enough. Some of those sites that are out there, most of which charge money, have literally millions of programmers contributing to them. So there's a big audience already doing this, and we're just trying to kind of snag some of them to do it in a better, cleaner, more open way, a way that's more visible to Google, a little bit less evil, and so on.<br /><br /><b>Atwood</b> : Let me give you a specific example: I talked about developers emailing me,surprised that they were getting answers so quickly, that were actually good answers, from the community. This particular person, Sarah Chips, who happens to be a female developer, which is somewhat rare, had emailed me and wanted an invite, and I was ok, if people are heavily motivated, and their emails are interesting enough, I will let them in the Beta, I know I'm going to get a lot of email now, so I'm really regret saying that, but I did let her in, and she actually wrote back a report that said she had posted initially not as herself, because she didn't want to bias the results, 'cause she had found that if you post as a "girl", people tend to pay more attention, and actually answer your stuff in a way that they wouldn't. But she found that even posting anonymously, not as herself, she got a very good answer to her question within like an hour and a half. So, it does, in fact, work. And now she's actually posting as herself, which is nice.<br /><br /><b>Spolsky</b>: Cool.<br /><br /><b>Atwood</b>: But yeah, I'm getting a lot of reports that it's working.<br /><br /><b>Spolsky</b>: Now everyone's falling all over themselves to answer her question as she asks it. Let's take another question!<br /><br /><b>Martin</b>: "Hi Jeff and Joel, this is Martin from Spain, I'd like to ask about your overall experience with StackOverflow so far. We all know that you two have been blogging for years now, about best practices, what not to do, or what the common mistakes doing software development are. So the question is, how well did that experience turn out in the real world? Thank you very much, and keep up the good work!"<br /><br /><b>Spolsky</b>: Thanks Martin!<br /><br /><b>Atwood</b>: So, what was the question again?<br /><br /><b>Spolsky</b>: How did your experience help? And it's a big broad question and you can answer it in any way you want.<br /><br /><b>Atwood</b>: Right, so what did we bring to the table in terms of our years of experience. We're starting a new project. Well, one thing I talked about before is really hand picking the people you work with, I really believe this is one of the most important things you can do in life. <b>really</b> think about who you want to work with and why. And it's all about the people. Every job I've been in, I think that's the one key thing I've learned, is pick a job where you work with people you love. And it doesn't really matter what the job is, on some level, right? Ostensibly it's something you're good at, people would pay you to do, it's all about the people you work with. So obviously working with Joel, god, Joel is legend, really, I'm saying that non ironically, that when I was starting out, I was reading Joel, so being able to work with Joel is, you know, fantastic, it's unbelievable. And being able to actually hand pick people from my previous lives in different jobs, and say "these are the guys that I want to work with", and it just turns out that they were available, I was very very lucky, in that regard, to me, I guess this is the lesson of people, where everyone has different levels of skills, and you can obsess about all the different levels of the process, tooling, and all that stuff, but to me it's such a people thing, programming. And maybe that's why StackOverflow and Coding Horror... It ends up being more about the people that I'm programming with, than the code itself. Which is really strange to me, because you start out with computers in the early 80s, and there's no Internet, there's no modems, barely, it's a very solitary experience. And now it's completely turned around, for me, computing is all about other programmers. It's completely turned around, I don't feel like I'm really programming, and people made fun of me for saying this, and rightfully so, until I produced something that somebody else can look at, I don't feel like I've done anything. If only the compiler has seen what I've done, I have failed. Which is crazy, if you think about pure...<br /><br /><b>Spolsky</b>: That's true, that's the whole demo, there's just an urge to say "demo", to shout "demo", and everybody comes into your office and see what it is you just built.<br /><br /><b>Atwood</b>: Right. The whole interaction between people is just what makes it enjoyable, it's the computers plus the people, that's what it's all about. I feel very fortunate to be in the position that I'm in now, where I can be at the confluence of the my two favorite things, it turns out I actually like people! Go figure. I'm not the social outcast that everyone thought I was in high school.<br /><br />[51:40]<br /><br /><b>Spolsky</b>: I have a question, this is off topic, but we've been talking about StackOverflow all day. I'm in Boston right now, tomorrow Neil and I are running this two day long Business of Software conference. Richard Stallman is scheduled to speak.<br /><br /><b>Atwood</b>: Nice<br /><br /><b>Spolsky</b>: Neil and I have been fighting over who gets to introduce him... Obviously we each want the other person to introduce him (laughs).<br /><br /><b>Atwood</b>: Right!<br />Spolsky: What would you say if you had to introduce Richard Stallman to a room full of people who obviously know, who he is, and what he stands for.<br /><br /><b>Atwood</b>: I think I would go with some of the more; "He's a colorful guy, isn't he?"<br /><br /><b>Spolsky</b>: Yeh, Red I believe is the color (laughs).<br /><br /><b>Atwood</b>: Without being a jerk about it, I think you can highlight some things that people may not know about him. That are cool, fun and interesting, and not weird, right? Which is a very fine line.<br /><br /><b>Spolsky</b>: We're really running out of ideas. I mean, there's the folk dancing. He's in a band I think.<br /><br /><b>Atwood</b>: Isn't there some online comic [http://geekz.co.uk/lovesraymond] that features him quite a bit, that would be a funny lead in.<br /><br /><b>Spolsky</b>: Unfortunately I'm scared by a 40 page rider I have to read. To make sure we're not in any way doing anything [bad?].<br /><br /><b>Atwood</b>: Well, if you have go the careful way, then it can't really be that fun.<br /><br /><b>Spolsky</b>: You know what would be my intuition, would to say: "When I got to College in 1987, and I had never heard of Richard Stallman, or the Free Software Foundation. The professors told me use Emacs as my text editor, and I launched up Emacs and the GNU manifesto appears. I think it might be one keystroke away from the homepage of the editor. And I read that and I thought; Oh, this is a cool manifesto, I'm into that - I was a founder of a commune in Asia. So the idea of a people's manifesto for free something, sounded great. And I think that the basic thesis, and I'm going to get this completely wrong, was that programmers should work for free in exchange for, maybe, some hardware that's given to them by the hardware companies who need software written for their hardware. And that's the way that I understood it. This extremely, non-subtle, and I know I"m going to get flames up the wazoo, but that's really how I understood it."<br /><br />And in fact all of the ideas about how free software has something to do with ... well I mean it has a lot to do with freedom and that's one thing ... but all of the ideas that it could have some kind of business benefit or a business structure that would work better are actually post-Stallman. I mean that is open source which Stallman is very much against. Or which he distinguishes very clearly from free software. And... anyway... and I remember thinking you know I kind of disagree with that. I think programmers should get paid and I think it is in programmers' interest to get paid. Now that's the end of that.<br /><br /><b>Atwood</b>: Right. No, that's fine I mean that's a good intro. I mean I guess I would go more non-traditional. I mean I think he's a very controversial figure and I would sort of use that as a springboard but like I said I don't know if you guys have to be really careful with the intros and things like that. But I mean surely a hugely important figure in computing. I mean I'm... not... you know don't let me take anything away from that. I mean the guy's pretty much a legend, right? <br /><br />[55:15]<br /><br /><b>Spolsky</b>: [...] [I don't think there's] been a day since 1987.<br /><br />[55:18] <br /><br /><b>Spolsky</b>: ... when I haven't used code that he wrote.<br /><br /><b>Atwood</b>: Yeah exactly, exactly that's my point. It's like agree with him or disagree with him. Whether you think he's just the weirdest guy in the world or you know somebody that's like a legend that deserves a lot of respect. Wherever you are in that continuum, you have to acknowledge his place in computing history. And very few people get to that position, right? And that's quite an achievement and something that deserves to be respected.<br /><br /><b>Spolsky</b>: Hear hear...<br /><br /><b>Atwood</b>: So who else is gonna be at this conference other than...<br /><br /><b>Spolsky</b>: Some speakers you might have heard of. I'll tell you ... I don't have the full list of speakers. It's kind of weird but half the speakers are people I know and half the speakers are people Neil knows. But Seth Godin who is kind of a management/marketing guru writes about a book a year on a marketing topic. They're awesome. Eric Sink who everybody knows as the founder of SourceGear and one of the original developers of the ... was it called SpyGlass? What was the first browser?<br /><br /><b>Atwood</b>: Yeah, SpyGlass. And he does ...<br /><br /><b>Spolsky</b>: So his current company is SourceGear, they have Vault which is a source code control system. And he wrote a book about the business of software which is awesome. And he coined the term "Micro ISV"... this concept that there is a... you know an ISV, a small independent software vendor who's just .... it's Micro because you're making I don't know ten thousand dollars a year writing some software in your spare time. It's really small but it's definitely a lifestyle choice and it's very easy to go from a Micro ISV to making your living writing software, as you do. And so he really coined that term.<br /><br />We've got Jason Fried from 37signals. We've got... Krug... what's ... remind me of his name... "Don't make me think" ...<br /><br /><b>Atwood</b>: Oh you got Steve... Steve Krug? Oh man now I'm jealous now I really wanna go.<br /><br /><b>Spolsky</b>: Well how fast can you get out?<br /><br /><b>Atwood</b>: That sucks...<br /><br /><b>Spolsky</b>: Take the red-eye, man...<br /><br /><b>Atwood</b>: I can't go, I can't go... I just went to Penny Arcade Expo in Seattle so I just had my little trip...<br /><br /><b>Spolsky</b>: Well next year you'll come but we'll have all different people probably.<br /><br /><b>Atwood</b>: So, hey... What about people like me who can't go? How can we participate? I mean is there like a recording of the sessions?<br /><br /><b>Spolsky</b>: Yeah we're gonna record the sessions except that no speaker worth their salt actually allows their recordings to be used like in-whole like as-is.<br /><br /><b>Atwood</b>: That sucks...<br /><br /><b>Spolsky</b>: But... you know there is... of Richard Stallman's big talk sure you'd be able to get in Ogg Vorbis format... I'm sure that's right on page one.<br /><br /></div><div><div><b>Spolsky</b>: [We will actually have] I'm in the process of making another movie. Did we talking about this on the podcast yet?<br /><br /></div><div><div><div><b>Atwood</b>: Oh really? You sent me a copy of Aardvark'd. I have seen that.<br /><br /></div><div><div><div><b>Spolsky</b>: That was our first movie. That was like a slightly serious reality TV show about the interns building Copilot. It was probably better than what on MTV. But the truth is the audience, the Joel on Software audience, really wanted something more technical and a lot of people wrote to me and said "You know, I really wanted to learn how software is developed at Fog Creek", and this movie was just a little bit too light on that content. This is going to be much more hardcore. SO it's really going to be about how software is developed at Fog Creek.<br /><br /></div><div><div><div><b>Atwood</b>: Ooh wow... Has this started?</div><div><br /><b>Spolsky</b>: It's about a one year project. We've got the same film maker as we had last time, except I'm basically interviewing everybody at Fog Creek. You know, three or four times, to get footage for this thing. And then we'll splice it all together, and the current plan is to have several different formats of this movie. There'll be a lot of short form little piece, like four minutes, we can put up on YouTube. If you want to hear about source code control, or you want to learn about hiring programmers or interviewing programmers or the phone interview. They'll be these little pieces that you can just download freely on the internet or watch. And then there's a [sort of a] more substantial ninety minute version, which is probably too much to distribute over the internet. But it'll probably distribute in DVD form. Maybe by that time, there'll be a good way to distribute a [you know] ninety minute DVD length thing over the internet. At kind of the high end, and what will hopefully pay for this whole operation, will be a corporate training video. Maybe a five to six hour thing that a team might buy inside a corporation to learn how to do software development a little bit better, and it'll just go into much more depth. There'll be like a whole hour of training on how to conduct interviews for programmers and a whole hour on how to setup your tools and that kind of stuff.<br /><br /></div><div><div><div><b>Atwood</b>: Wow. So you're like Howard Stern. You're like the kind of the media now. You have like your movie. You've got your podcasts with me. You've got, you know, your blog, your not-a-blog.<br /><br /></div><div><div><div><b>Spolsky</b>: But I mean, you know, that was the original idea behind this podcast remember how I said, like, "There's some people that you can reach in certain channels, and they need to hear you. I don't listen to Eminem's music, blog I saw the movie 8-mile, or miles... whatever it was. Different people, you will reach in different channels. Part of the goal is to do that, and the other part of the goal is: I have an inherent inability to ever give up. And instead of just saying, "You know, we tried to make a movie, and it wasn't as good as it should have been, and so maybe we're not so good at making movies.", we just said "You know, let's learn from the experience. Let's try and make a two point oh, that's just much better."<br /><br /></div><div><div><div><b>Atwood</b>: I didn't really hear a lot of negative stuff about it.<br /><br /></div><div><div><div><b>Spolsky</b>: You know who liked it? Programmers' girlfriends, wives and families. Because there's nothing about it that's inaccessible to somebody who knows nothing about programming. And it really does convey to the normal person a lot about what a programmer's life is like. Just the sort of feel or being a programmer. The fact that programmers get into these conversations about things. You know that was probably, I thought, one of the highlights of the movie - when we told the interns that their job was to figure out if they could jump across to the ledge of the neighboring building if there was a fire. Could they jump out the window and make it on to the ledge of the neighboring building? And programmers love to have conversations like that, and they immediately leapt up to the white board and started drawing equations from physics and stuff like that, and they tried to work it out. And that's really what our conversations are like, what our lives are like. And to geeks, nobody knows--, if you're a geek you probably don't think anything is strange about this. But to a normal human being... this is very bizarre. That we talk that way, and that we think about these kind of things, and that we then need to solve the problem mathematically, to decide if you can make it to the neighboring ledge before you died.<br /><br /></div><div><div><div><b>Atwood</b>: Right... Well that's cool. I'm looking forward to that coming out. Looks like it'll be a while. I definitely recommend people who haven't seen it, check out Aardvark'd. I enjoyed it. I didn't think it lacked anything.<br /><br /></div><div><div><div><b>Spolsky</b>: The way to find that is ProjectAardvark.com, and there's link to the movie. ProjectAardvark.com is a blog that the interns kept that summer. It was a couple of years ago.</div><div><br />So anyway the point of that is that I do actually have the filmmaker here at the conference who helps me film a bunch of stuff. And hopefully we'll be able to have you know maybe just a snippet like one sentence maybe from a speech or from a hallway conversation here or there that will in some way contribute to the overall story that we want to tell with this movie about how software is made.<br /><br /><b>Atwood</b>: Very cool. Well we should probably cut it off here I think that's the right time...<br /><br /><b>Spolsky</b>: Alright... let's see do we have any announcements? No we're about...<br /><br /><b>Atwood</b>: Same stuff. Same stuff.<br /><br /><b>Spolsky</b>: Ok why don't you do that...</div><div><br /><b>Atwood</b>: Two things; we have a wiki, for people who can't listen to this. Where people can contribute transcriptions of our incredibly boring podcasts, and we thank you very much for that. Although I do have one request for the transcriptionists, and the ironic thing is, you're going to transcribe this, which I think is hilarious. When you transcribe, don't write down every time I say 'uh' or pause or 'yeah'. Make me sound awesome, that's my one request for the transcriptionists.<br /><br /><b>Spolsky</b>: It doesn't have to be word for word. It doesn't necessarily read as well when it's word for word you can leave [out] 'uhms' and 'uhs'.<br /><br /><b>Atwood</b>: In fact, leave out whole words. If you think it reads better a certain way, just make me say whatever makes me sound the most awesome.<br /><br /><b>Spolsky</b>: And it's wiki, go ahead and edit it.<br /><br /><b>Atwood</b>: People edit anyway, you're right, it's hilarious. I've been reading the [revisions], it's very funny.<br /><br /><b>Atwood</b>: The other thing is, if you do contribute to the wiki. Since our beta has been pushed back a week, this will get you in the same day, to the StackOverflow beta. If you want to be in, just email me after you've done a little bit of transcription, one minute or whatever you're comfortable with. If you want to get your question answered on the air, send a less than 90 second recording to podcast@stackoverflow.com, we will put it in the queue and hopefully answer it on the next podcast.<br /><br /><b>Spolsky</b>: Alright, that's it. Thank you very much, see you next week!<br /><br /><b>Atwood</b>: See you next week.<br /><br />[1:04:04]<br /><br />[Outro]<br /><br /></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
Sorry, you must use either Internet Explorer (6+), Safari (3+, non-mobile), Google Chrome (5+), or a modern Gecko-based browser (such as Firefox 2+), and you must have javascript enabled in order to edit this article.
