The StackOverflow Podcast

[1:15]

Atwood: Oh yes, I love QuickBooks!

Spolsky: He he.

Atwood: Very exciting bit of software, very fun to use.

Spolsky: Yeah, uhh Michael and I were having a little laugh about it today

Atwood: Yeah, it's necessary, I appreciate the necessity of it.  But, I gotta tell ya, I really feel that QuickBooks is a good example of software that just doesn't sort of .. hasn't adapted to sort of the new paradigm that we have for software

Spolsky: Well, neither has accounting so he he he get used to it.  You know what I mean?  Like its .. the thing about QuickBooks is that it has evolved over many years to be the perfect accounting application.  Or I should say bookkeeping -- bookkeeping and accounting.

Atwood:  Really?  Perfect?  There's nothing?

Spolsky:  It's perfect, yes, it, yes, you'll see, it's perfect, it doesn't gather any data that you don't - yes, it takes you a long time to learn that it's doing things the way they are because you know - you have to have gone through about 5 or 6 years of filling out every possible tax form before you finally appreciate the craziness that's in there?

Atwood:  Right, well no, I appreciate the wisdom you guys are bestowing upon me because you've done this stuff and I do I do understand that it's necessary and I appologize for being very balky about it umm I know it's necessary but I do want to put this to our listeners, I'm kind of curious because Joel maintained and I can see your point on this Joel, to be fair I totally see your point on this that some of the emerging online stuff like I think, uh gosh, I can't really remember the names but there's a couple of really popular web based sort of accounting solutions.

Spolsky: Yeah, like Net Ledger is that one of them?

Atwood: uhhh, Wesabe I think is one of them..

Spolsky: Oh, those are .. Net Suite that was the one I was thinking of Net Suite, there was something called Net Ledger I'm pretty sure and uhh I don't know what Wesabe is but I think those are umm those are like personal accounting systems not suitable for business accounting..

Atwood: But, you can't use those for small business at all?

Spolsky: No, you have issues when you're a small business of of that that are different from .. yeah..

Atwood: Ok, that's fair, but one thing that you had warned about was the tendency for these guys to hold your data hostage which I think particularly as a business would be really dangerous right

Spolsky: Oh yeah yeah yeah, the web based ones definately did that and I don't even think I'm giving away the pooch if I can say that I think it was Net Suite, ahhhh, you know I should check ahhh definitely some of them, some of them ran for a few years and then raised their price a few times.

Atwood:  Yeah, no, I hear where you're coming from and I think as a small business that's a serious concern, I mean from a personal level it's kind of a concern but .. I mean, I don't know

Spolsky: Even QuickBooks every once in a while requires you to upgrade, um, you know because something changes in tax laws or something and they just - I don't know why - and they get - people get furious because they're forced to buy the upgrade after you know every three years or something but it's a lot more reasonable than the online ones usually and the other real reason to use QuickBooks is every accountant in the world knows how to use it because it's just become such a standard in the way that word is.

Atwood: Right, I still think, you know that said, I still think there's an opportunity for someone to be QuickBooks compatible but be a little less painful to use. Because I post it on Twitter to voice out my dissatisfaction and you know who, uh, concurred Richard White of User Voice saying "I challenge anyone to find out a software with a worse start-up experience than QuickBooks and he was using the Mac version." So apparently it's...

Spolsky: It's sort of hard to disentangle the start-up experience of QuickBooks with the fact you are doing business-level bookkeeping for the first time.

Atwood: Um-hmm.

Spolsky: Like if, it's really... once you know business-level bookkeeping, you know, the, the, the trouble is that, that the way you bookkeeping for a business is.. kind of hard, that double-entry bookkeeping and stuff. Uh, it isn't for all our listeners.

Atwood: Uh, OK. That's enough of that.

Spolsky: We are going to leave all that.

Atwood: It is reality, though. I mean, you are to be an entrepreneur, then this is something that you have to do unfortunately, believe me.

Spolsky: You do.

Atwood: I'm not happy about it but you have to do it

Spolsky: It's not that hard. ... You see, usually what they recommend and this is still what I'd recommend to a random person: um, get, wait, when you start your business, hire your accountant, the guy who is going to do your taxes at the end of the year, go to his office, have him to spend an hour to setting up QuickBook for you and showing you how to enter all the things that happens and how to enter them. And just have him, you know, check over every, you know, check over every once in a while. Uh, because you, you, you'll be presented with -- it doesn't add up to that much time and true to the history of Fog Creek, you know, I do not know ANYTHING about double-entry bookkeeping or anything like that, and I read one book about how double-entry bookkeeping should work, uh, which is very helpful. And I read the manual -- does QuickBook still comes with a printed manual?

Atwood: Uh... I bought it online, so... no.

Spolsky: Oooh. See now the printed book that used to come with QuickBooks was quite good in that it wasn't just teaching you how to use the application, it was actually teaching you how to track money for your small company. Which is, um, above and beyond what you'd expect from a user manual that was pretty well written in the sense that it tell you what you need to know and doesn't confuse you with the things that you don't need to know.

[6:15]

Atwood: Right. The good news is that, from the business viability standpoint, I mean, one of my goals is to make enough from advertising that I could, you know, pay other programmers each.. and ideally myself would be nice as well.

Spolsky: Yeah.

Atwood: And that's doing really well. I haven't given you guys the latest data but that's looking pretty good. So..

Spolsky: Awesome.

Atwood: ..that's encouraging..

Spolsky: Woo-hoo.

Atwood: 'cos I want this thing to be sustaining. And, the reason I want this to be sustaining, I'm going to segue into my next topic, is we are continuing to builing out new parts of the site and we wanted to continue to make it better.. and, you know, more complete.

Spolsky: Yeah.

Atwood: And to that end, now that Jarrod is full-time and I brought Geoff on as well, we are able to complete two major features last week.

Spolsky: I saw them.

Atwood: Yeah, so it's pretty exciting. So one is question bounty, that is actually live and working now. Although we did actually defer... so there's something that has to happen after seven days because there's a timeout for the bounty, we haven't actually done that seven day part yet [laughs] 'cause it, we figure out that we have seven days to get this working.

Spolsky: [laughs] I don't know how many times I, I have pull that stunt in my time.

Atwood: Oh, that's funny. But yeah, Jarrod is going to finish obviously before the seven day timeline we are going to finish the end state. But the, the actual core part of it is working. If you put up a bounty on a question, you put up some of your own reputation, it has to be at least two days of old... and we've seen it work. I have actually, we have a little admin page that shows, uh, closed bounties.

Spolsky: Jarrod, I would say Jarrod is pratically the definition of get things done.

Atwood: Oh, Jarrod is great.

Spolsky: Like there's no risk whatsoever of it isn't going to get it done.

Atwood: Oh, no no no. These guys are great, you know, these are, I have talked about this before, but these are hand-picked people I have worked with before so, you know, I have total confidence in their abilities. But bounty is working.

Spolsky: Bounty's working?

Atwood: We've actually seen people get really good answers from it. I mean, nothing works at everytime. It's a bit of a gamble, I think people are upset a little bit because like they'll go "How can you GUARANTEE we are going to get an answer?" or "How can you GUARANTEE...", I mean, there's no guarantee, when you're born there's not like a form that says "OK, we will guarantee x". I mean this is an illusion, I mean, you don't know what you're going to get. But the good news is...

Spolsky: Yeah.

Atwood: It definitely increases the interest in your question, and... I see many of them work. Like, really work.

Spolsky: Someone placed a bounty and they get a good answer

Atwood: Yes!

[8:20]

[Garbled]

Spolsky: I like it not being a guarantee because if you know that you're asking a question... like if a lot of times, there are these questions that are like how do I do... the following, and it's just not possible. Right? Like let's say I wanna make -- here's something I could post a question for. I wanna make it so that Chrome, you know, the new Google Web browser, does not animate animated gifs. 'Cos it gives me a frigging headache. And, and, that's all I want. And I don't - I don't even need ad - I'll look at as many ads as you want as long as they don't flash. Umm, but there's no way to do that. There's just not - they did not expose the functionality to turn off the animated gif-ness.

Atwood: Well there's ...

Spolsky: I guess you have the source code, right? So you could ... edit the source code. But I just want a -- I want a dialog box. Check off, it'll... turn off animated gifs. And they don't have it. So, if somebody's asking a question like that, and the answer is, "You can't do that. There is no way to do that, I'm sorry, tough it out," they're probably not gonna place a bounty, because there's a very good [laughing] risk that they're not gonna get an answer, and they're gonna lose their money at that point.

Atwood: Yeah, I have seen some like that. I've also seen some like, "I want to do this really complicated thing. How should I do it?" You know, and it's kind of like, "It's complicated! I mean, what do you want us to do, do it for you?" 

Spolsky: Right, right, right.

Atwood: So, it varies. Obviously.

Spolsky: Is there like a continuum from Stack Overflow to, like, those rent-a-guru sites that just write code for you? Or is there... I think there's a big enough gap in the middle. 

Atwood: Yeah, there could be. But there's definitely a gambling aspect to it. Now that I think about it. People have also complained that it theoretically turns reputation into a form of currency. I don't entirely agree with that, because --

Spolsky: [gasps]

Atwood: Because I feel like it's something you earn, 'cos, you know, you had that whole rant about --

Spolsky: Yeah.

Atwood: About how, "Don't pay people!" But this doesn't, this doesn't feel like payment to me. This feels like --

Spolsky: Tipping.

Atwood: -- something nice that you're doing... yes, tipping. Thank you. Perfect. And a little bit like gambling, honestly. [Laughing] 'Cos you're saying, Ooh, put 150 rep on the line, spin it around, and see what happens. But you know, gambling is fun, and there's definitely a game-like aspect to Stack Overflow and part of it is intentional, I mean you know we don't want to go too far with it because it gets ridiculous, but I'm comfortable with that.

Spolsky: So this is -- I was, ah, randomly, uh, yesterday sitting in the office -- sorry, the day before yesterday -- sitting in the offices of Google in Munich, um, with a bunch of developers there, and I was like "So how many of you have ever heard of Stack Overflow?" And it was all of them.

Atwood: Wow.

Spolsky: I mean people use it now. I just thought I'd mention that at random.

Atwood: No, that's great. Speaking of which, let's talk about it. How was your European vacation?

Spolsky: Well it wasn't a vacation... 

Atwood: I'm just going to pretend like it's the movie, because that's more exciting...

Spolsky: European Vacation? 

Atwood: Yes.

Spolsky: It was more like The Bourne Identity. I was running around European capitals, first-class lounges, taxis... oh my God, they go sooo fast on the road in Germany. When they picked me up from the airport, the speedometer was at 190. How much is that in dollars? Hehe, how much is that in dollars? 

[Laughter]

Spolsky: 190 kilometers per hour in miles per hour... 118. Oh  my God. It was so fast - and it was like a Mercedes limo, too, and it was passing all the other cars on the road, and you didn't even feel it moving. Practically. Because it was -- [shudders]. Anyway. That was cool. That was the first cool thing. But the conference was not so good. You know if you're ever involved at all in any kind of conferences, you don't want to ever be on a panel or in any way going to any kind of conference that has panels, or... Nothing with panels. Panels: bad.

Atwood: Well, they do those at Mix, and I think if you don't have too many of them they're OK. If everything's a panel you're in trouble. I agree with that, but... 

Spolsky: Yeah.

Atwood: A handful of panels, not too bad.

Spolsky: It depends how many people. If you and I were on a panel, it would be fun. But if there were four other random people there, also talking, you just -- you don't get enough airtime, so it's very hard to have -- maybe 3 people on a panel is okay.

Atwood: You also need a good moderator. Which, I'll tell you is a real skill. Moderating people on a panel. You have to have someone in charge that knows what they're doing. Otherwise it's gonna go south.

Spolsky: It depends also who's on it. I was on a software p- this is not a software conference, this is digital life and design, so this is just a very generic conference about everything, hosted by Birdo which is a big publishing company over there in Germany. And, um, my panel was called "Software", and the -- the people on it, uh, were a very very wide selection of people that really did not have -- we didn't have that much in common, I guess, and, um, and so the, uh, moderator tried very hard, Marisa Mayer, from Google, tried very hard to come up with interesting topics that would be interesting to anybody --

Atwood: Mm-hm.

Spolsky: But the trouble is, everything had to be so vague and so generalised that it was very hard to come up with anything, uh, interesting. 

Atwood: Another thing I can actually talk about is we have a bit of a {garbled}, because the way we chose to store what we call "posts", which is questions and answers, and then revisions-- so we have two tables, we have posts and revisions.  For viewing a post, the revisions we usually care about are the first revision and the current revision.  So, there's basically pointers in the post table to records in the revision table.  And this sounds fine on paper.  It has a few downsides, in that for example if you retag something, due to the way we're storing the data, we have to store duplicates of the post and everything else, so it wasn't optimal from that perspective.  What turns out to be the big killer with this approach is just the fact that we're joining, all the time.  Like, in order to talk about a question, I need to go get the current revision, right?  In any case.  'Cause I need to figure out who the author is, and stuff like that, and that's all stored in the revision, because there could be ten revisions to a question.  You could have a revision, I could have a revision, Jon Skeet could have a revision.  So, it doesn't sound like much, this is like relational databases in a nutshell, right?  Just go do a join.  But it turns out these joins are unbelievably expensive.  I mean, if you want to do things --

Spolsky: Really?  Is it just not indexed right?

Atwood: No, believe me, it's indexed --

Spolsky: Oh, it's those memo fields, that's why.

Atwood: Uh, well, that's right.  There's some large strings attached to revisions.

Spolsky: Yeah.  Those always take longer, because they're not in line with the rest of the table, they're in some big blob storage place, and even the APIs never get them out directly, they always read them out like 64kb at a time.

Atwood: Yeah.  Right.  No, there's definately some semi-large fields there, depending on the size of the post.  They're variable-sized fields, of course, by definition.  But we find that even when you're dealing with just regular tables, joins are not free, by any stretch of the imagination.  Every time that you're doing a join, that's a very real cost to the query.  So, we're going to have to do a massive refactoring to fix this.  We're going to move a lot of data up into the post table, so that like 99% of the time when we're talking about the current revision of a question or answer, we'll just have to look at that one record in posts.

Spolsky: This brings up sort of a more general problem, that probably a lot of people have been thinking about.  Because if you look at those...  What am I talking about here?  A lot of people have developed these libraries that are less than relational databases for use on very very large, highly-scalable websites.  So, for example, I think Google's thing is called BigTable.  A lot of times they just have some big, gigantic, ultra-super-duper scalable name-value pair storage.  Think of something like berkley db, where, I can store things for you, I can do it pretty fast, but I'm not going to give you full relational capabilities, and in particular you don't get joins.

Atwood: Right.  I can appreciate why they do that, because I've worked with databases for a long long time, going all the way back to db4 or whatever it was called, dbase4, and then Fox Pro... I've done this for a long time, so I really understand how databases work -- not that I don't make mistakes, believe me, I do -- but it's just interesting to me that we built up the new database server, which is now ready to go, by the way, all this stuff is ready to ship finally, and I was doing some just basic benchmarking, so I loaded the stackoverflow database on our new database server.  So this thing is basically 50% faster than our old server, roughly.  It has 50% faster CPU, a lot more memory, faster bus, more level 2 cache, all that good stuff, so you'd expect it to do about 50% better, and--

Spolsky: Well, not if it was already maxing out, sort of.

Atwood: But it's not.  I already noticed that it's not even close to maxing out.

Spolsky: Well, wait, that's not what I meant.  What I meant was, you wouldn't expect it to be any faster.  For example, if you had more memory, it would be faster, but only if that means you could keep more things in memory, whereas if you weren't already using up all your memory in the first place, then that additional memory isn't going to help.

Atwood: Right.  You're always playing a game of "where's the bottleneck."  This is, hopefully, as a programmer, this is why I like programmers to mess with hardware, because you learn to play this shell game of like "oh, now the bottleneck's the disk, now the bottleneck's the memory, now the bottleneck's the CPU."  And you're really just trading off bottlenecks, at some level, right?  You can do compression, for example, compression's a classic example of trading CPU bottleneck for memory, which is usually a good deal, based on CPU speed.  But anyway, when I loaded up the stackoverflow database, and was running some comparison queries, I'd run them on my local machine, I'd run them on the new database server, and I'd run them on our current, live database server.  It's not really a fair comparison, because the live database server is live, although, in all honestly, our load is really -- our CPU graph is almost null.  We've gotten to the point where we're not using CPU at all.  We're running out of memory a little tiny bit, but it shouldn't have a dramatic impact on our results.

[23:32]

Spolsky: And this is... er, and this is serving ten million hits a month... ten million pages a month?

Atwood: Oh yeah. Yeah yeah yeah. It's doing it no problem. What I found was, so this is the interesting... the main difference between these machines is really CPU. Like, my desktop has a 3.5 gigahertz dual-core.

Spolsky: Mm-hm.

Atwood: The new server has 2.5, and then the current database server has 1.8. So you got the continuum of 3.5, 2.5, 1.86, [garbled] CPU, and I've found that for a lot of queries, unless you're just massively exceeding the amount of memory, like you're doing some query that's pulling, like, gigabytes of data...

Spolsky: Mm-hm.

Atwood: Um, it scaled really linearly with CPU. Like, I'm talking like, you go from like 100 milliseconds, to, to, like, 60 milliseconds on the new server, and the on my machine it'd be like 40 milliseconds.

Spolsky: Huh. Wait, why don't we shove your machine in the data center then?

Atwood: [Laughs] Well the shocking thing is people, when you talk about, you know..

Spolsky: How did you... wait, how do you even have a machine that's faster than the one in the data center?

Atwood: Well, it's just CPUs. Because server CPUs are not, you know, they're not like top end, like they go with the conservative...

Spolsky: Are they?

Atwood: ... process model. So Intel... the fastest CPUs aren't necessarily the server-room CPUs. In terms of clock speed. Um. But this is contrary to what you're told. You're told, "Oh, CPU's not that important on a database server. What you really want is lots and lots of memory, and super, super-fast disks."

Spolsky: Well that's 'cos that used to be the problem. 

Atwood: But I think for our database, because, first of all we have 24 gigs of memory now which is just a ton, ah that's six times more than what we have now...

Spolsky: How big is the database? If we have 24 gigs of memory how big is the database?

Atwood: Ah, gosh. I mean, oh disk? 

Spolsky: Yeah.

Atwood: As a backup? It's like, 3.5 gig...

Spolsky: OK, so its all in RAM.

Atwood: It's not quite all in RAM any more. One thing I've noticed is that certain queries, when I run 'em, like, they'll be just slow the first time. And the second time...

Spolsky: Because they... they have to page-fault in, eventually. But eventually most operating conditions eventually... everything just sits in memory. 

Atwood: Yeah. Right. No, for the most part it's in memory although I am seeing some cases where, uh, we're definitely paging now, in some form. Um, but I just wanted to talk a little bit about that because, a, I want to make sure that new database server is faster, and the good news is it is, it's not fifty percent faster as I'd hoped, but it's more like thirty-three percent faster, like across the board. Like in general, anything you're doing, any query on this new server will be about thirty-three percent faster. But also that, you know, CPU matters. 

Spolsky: Yeah.

Atwood: If your query's in memory, it's all about the CPU...

[talking over each other]

Spolsky: Don't forget that thirty pecent... a thirty percent improvement doesn't mean the average person gets their reply thirty percent faster. Because it's possible that several people have all submitted a query at the same time and you're waiting for all of them. So it might mean that you're... you know, a bunch of people have submitted and you gotta wait like five seconds, and now you're only going to have to wait like three seconds instead of five seconds while their things'll finish. And that might mean that the computer gets to idle faster, which may mean that an increasing number of people don't have to wait online at all.

Atwood: Sure.

Spolsky: Did you ever take a, uh, queuing theory class or study queuing theory in any way?

Atwood: I don't think so.

[26:30]

Spolsky: It's kind of interesting. There's a whole mathematics of, like, people arrive at the bank and they wait in line, and how long does it take them to get served and so forth. And, um, one of the interesting rule of thumbs that I remember as being, you know, not a formal result of queuing theory but something that you can kind of keep in mind if you're ever, you know, setting up a restaurant or, like, a, uh, line of people to get coffee at your coffee shop, or whatever, any kind of situation where there's a line serving people. Um, is that there's various measurements that you use in queueing theory. One of them is called "utilisation". And that's the percentage of the time that the people serving the queue are busy. So if you've got tellers at a bank, what percentage of the time are they working, what percentage of the time are they idle, or, you know, telephone operators, people to pick up the phone at [Unknown]. And um, ah, the total amount of, of time that they spend working divided by the total amount of time that they have available, that they're sitting at their desk ready, is called the utilisation. And one of the common results in queueing theory is that at a utilisation of 80 percent all kinds of things start to go wrong. And the... the length of the lines, the average amount of time that somebody waits in lines, tends to get really really bad if the utilisation goes above approximately 80 percent. And so obviously, umm, the number will be a higher percentage if there's a smaller number of... a larger number of small transactions, and it would be a lower percentage if there's a... etc, but the point is that, uh, kinda as a rule of thumb, and this is something you can measure for your own scenario, but as a rule of thumb, um, typically if you have a bunch of bank tellers and they're working 79 percent of the time, most customers will come in and not have to wait in line at all. but if you just get up to like 81 percent utilisation that they're busy, then the average customer might come in and wait fifteen minutes on-line. 

[pause]

Atwood: Wow.

Spolsky: Because there's a certain point at which... um, you know there's a certain... you can do this sort of mathematical simulation when you say "people are arriving with a certain probability at any given time" and that's something called the Poisson distribution... the probability that people will arrive if everybody arrives independently, and, uh, they tend to distribute according to a certain curve, and so there's a certain probability that a big chunk of them will arrive all at the same time. And, um, the... the, if the utilisation goes too high, you get to the point where, maybe you're serving... imagine a utilisation of 100 percent. You are serving everybody. But, you know, the fourth person to arrive in the day is going to have to wait for the first three people to be done, and the last person in the day is probably just going to get dealt with right away, but, but during the day you're gonna have all kinds of people who arrive and have to wait for hours. To - to wait their turn, basically. 

Atwood: Hm.

Spolsky: So, uh...

Atwood: I remember reading [garbled]. This has a lot of operating system implications, 'cos isn't the way the scheduler works in the operating system really critical to overall performance? Like...

Spolsky: Kind of, yeah.

Atwood: I remember even in Vista, like they improved some things about, like, you could get priority scheduling, like if you're doing multimedia playback so you can't get kicked out of the queue, and have stuttering, and, and...

Spolsky: Right, right...

Atwood: This is really the art of divin... designing an operating system, isn't it? This making sure that... it's just playing the shell game with bottlenecks, right? IO, disk, CPU, and making sure that nobody's, like, starving, unless...

Spolsky: Right, right...

Atwood: ... something catastrophic is happening, in which case you're just... you're just hosed.

Spolsky: One thing I've noticed over the years is whenever is something's taking too long, I'll switch to another window, open a browser, and try to do something else! So you're kind of punishing the CPU for being slow or, your computer in general. You're actually making it a little bit worse, which is.. you're launching other apps to keep you entertained while your computer works on your first process and making it even more overloaded at precisely the moment where it needs some extra CPU power. 

Atwood: Well one thing I like about, you know, Vista introduced this new system performance metric -- which is a reasonable set of benchmarks, but what I liked about it that I thought they really got right -- so pay attention, something that Vista actually did correct, Joel, you may want to write this down...

Spolsky: I'm getting my fountain pen.

Atwood: ... it actually does the benchmark, and it, it actually takes the lowest number, is your score. In other words, wherever the bottleneck is, that number determines your entire score.

Spolsky: Mm-hm.

Atwood: So if you have a really slow disk, say you get a one-point-oh on your disk score...

Spolsky: Mm-hm.

Atwood: That's your overall score.

Spolsky: Oh this is the, uh, the...

Atwood: Which makes total sense because that's really how you should look at it, like, "Where is my bottleneck?" That's the thing I need to improve, so really incentivise you so... "Wow, one-point-oh on disk. I can have a four-point-three, all my other stuff is at four point three, if I just rid of that stupid slow disk." Um, and that was really smart and really the correct way, um, to look at performance.

Spolsky: Does anybody else... uh, I put Windows 7 on a laptop here, and it just freezes. The whole system just freezes. The mouse won't move, it's just like hard frozen. 

Atwood: Ooh.

Spolsky: Is that the laptop, or is that Windows 7?

Atwood: I haven't -- I have not heard of that. All I've heard is, like, people liking Windows 7. 

Spolsky: Yeah, that's what I heard.

[30:57]

Atwood: [Laughing]

Spolsky: But I don't like it 'cos it makes the whole laptop freeze. But the laptop may just suck, you know. It's a Dell.

Atwood:  Huh. Interesting. No, I dunno. Could be a hardware problem. It's beta software.

Atwood: So, I have mixed feelings about it, but again it's a balance that every programmer has to decide for themselves. And I -- quality is obviously important, I'm not going to say that quality isn't -- but there's a lot of ways to get there. I feel like there's a lot of ways to get there, and I kinda dislike some of the dogma around --

Spolsky: Right, it's the obsessive-compulsive things, where it just starts to become like a hygiene, where it starts to become, like, I'm dirty unless I do these things. And that's where the risk is that you will waste time doing something non-optimal, that you haven't thought through.

Atwood: But the other thing is I'm very uncomfortable coming out on record saying I'm anti-unit testing, because I think it is a significant advance in quality for software engineering, like across the board.

Spolsky: Mm-hm.

Atwood: So I've mixed feeling about this discussion. But on the other hand I have to be honest with people about what I actually do, I feel like I'm lying at some level. It's like, oh! You should unit test. But we're not unit testing to the degree... to a significant degree. When you talk about unit testing we probably don't even, like, add up. So I just want to be honest with people. So anyway let's get to the question.

Spolsky: Yeah. I don't even know what it's gonna be, so it may be completely off-the-wall. This is from Michael Cohen:

Caller: [Garbled]

Spolsky: Start over. Start over. 

Caller: Mike Coney. Philadelphia ---

Spolsky: Coney. Can you hear him?

Atwood: Mm-hm.

Caller: On the last Stack Overflow, Joel was talking about the S.O.L.I.D. principles of Robert Martin, and this idea of creating an interface which represents the parts of the target class which you wanna use. The idea, if I understand it, is to isolate your class from changes that it doesn't care about in the target class. This sounds awfully familiar. About ten years ago, I worked for a company doing consulting. The client I was working for had a system architect that wanted us to create interface classes that would go inbetween two components that they'd purchased from the same vendor. 

Spolsky: Yes!

Caller: The components were designed to be plugged together and work. The stated reason for this idiocy was to make it easier to change out one of the components if that ever became necessary. Thankfully, my boss talked them out of doing this work, as we weren't looking forward to debugging the messes it was bound to create. I think that this part of what Martin is advocating is the same over-isolation that architectures have been prone to for years and years and years.

Spolsky: Yes! Yes.

Caller: In the real world, when you need to change out a component, you simply write the new component to the old interface. And if it turns out that the old interface wasn't sufficient, then you end up having to rewrite both sides anyway, and having an interface in between doesn't buy you anything. 

Spolsky: Yep.

Caller: Thanks for listening.

Spolsky: I think he just agreed with us.

Atwood: Yes.

Spolsky: And that's a good story, actually... it's... it is something about, you know the interesting thing is to look at, you know, when an engineer describes engineering principles, and you actually look at the things that engineers actually built, those principles make sense in very very limited frameworks. So for example if it's the thing about the interface so you could swap out or whatever, it's like, yeah, we have standard double-A batteries and they have to be a certain size and shape and they have to have the positive electrons flowing to the negative electrons and the north side and the south side in the right order.

Atwood: Huh.

Spolsky: And if you did it the other way then people would be ... upset. And, but once you've defined that specification -- and you can work really really hard in making those protocols of the specifications work -- but that doesn't mean that every single thing that every single engineer builds is... consists of individual components all of which are sort of the ultimate in perfect engineering. Because that would be costly. It's not just a matter of wasting the effort to write all that code that creates that arbitrary interface that isolates components from one another in a way that they weren't even meant to need to need to be isolated from one another. Um, but if you think about that in the real world, it, uh, it's guaranteed to be inefficient. It's guaranteed to create inefficiencies. So for example, you know, I have a cellphone, and the cellphone has a GSM chip in it somewhere, but I can't swap out that GSM chip with another GSM chip. And a good engineer might have built a cellphone with pop-out user-replaceable GSM chips, but that would have required another little slot, a little thing that I could open, and it would have required that the interface between the GSM chip and the rest of the cellphone be, ah, a plug-and-play kind of thing rather than something soldered on, so it would have been bigger, and heavier, and more expensive, and who knows if any manufacturers would ever come along with a superior GSM chip that it was worth installing on there, and if it would really be truly compatible. So yes, you do this things for the headphone jack. But you don't have to do it for every single little component internally in the application. And maybe, maybe um, maybe we're not given Uncle Bob enough credit. Maybe the idea of these S.O.L.I.D. principles is like, "Well these are the principles, doesn't mean you always apply them, you have to use your brains as to whether to apply them or not."

Atwood: Yeah. I -- I have mixed feelings about a lot of the architecture stuff. I feel like the only metric that really matters is whether you're shipping software that really solves some problem for somebody.

Spolsky: Mm-hm.

Atwood: And I've become more and more reluctant over time to really judge anybody's methods that they use to be successful with this. Now I will say that the teams that tend to be successful do follow some patterns, and you should look at teams that have been successful and the things that they do --

Spolsky: They hire smart people. 

Atwood: But yeah! You can't switch -- it doesn't generalise well. It kinda breaks down into "just have a really good team" that really, like, gels. Um. And it becomes less about even the technology because there's been all these really great products released in PHP, which I continue to believe is a really horrible, horrible broken language -- 

Spolsky: Yeah. 

Atwood: People talked about BASIC sort of, what is it, corrupting the minds of programmers? 

Spolsky: Oh yeah, PHP is a direct rip-off of Visual Basic that didn't even -- or VBScript that doesn't even take advantage of -- that puts those stupid dollar signs and they're [garbled] correctly it's not even a well-designed rip-off of VBScript. 

Atwood: Yeah. And so it tends to make people write really bad code. But at the same time you'll have these applications which are -- like Wordpress -- which are great. And they might be written horribly internally, or maybe they're written really great --

Spolsky: I think Wordpress is kind of a mess internally.

Atwood: Yeah, but it's possible to create these really great things that really help a lot of people --

Spolsky: Yeah. 

Atwood: -- and solve the goal that they intended to... create.

Spolsky: Absolutely and in fact ...

[talking over each other]

Spolsky: To be fair, as the price of an application goes up the likelihood that it was written with a good tool goes down. So for example, those really really expensive enterprise software that does some big humongous complicated thing ... this software is selling for a million dollars a seat. It looks terrible, the user interface is horrible and hobbled, but, ah, and it's written in Visual Basic 3.0. But it does actually solve a business need that makes it worth a million dollars as seat. And that's why somebody's getting very very rich off of it. So. So there.

Atwood: Yeah. 

Spolsky: Do you want me to, ah --

Atwood: I can kind of see where you're coming from when you were talking about writing newer articles you felt like you had said everything that you had, kinda, wanted to say. Like at some point this stuff all repeats, right, and we talked about it before: there's a whole new generation of programmers coming up and I do hope that they learn a better way and are able to avoid some of the mistakes that we made, but beyond that... I dunno. I don't think there's a lot of specific stuff you can point to, like S.O.L.I.D. principles or whatever, that's really going to save the day. Like, just because I learned S.O.L.I.D. principles I'm shipping an app that, you know...

Spolsky: And what you're saying here is itself not new. It's a Frederick Brooks essay called No Silver Bullet -- remember that essay?

Atwood: That's true! I am myself repeating No Silver Bullet.

Spolsky: So -- yeah, if we'd all just frigging read No Silver Bullet we wouldn't be having these, uh, these conversations. And I don't think anybody claims to have silver bullets any more. I think they just claim to have, ah, certain advancements, incremental advancements in the state of the art of programming technology. But I've found, for example, in an advancement -- add another feature to a compiler, ah, for example, adding good garbage collection is the easiest one, but even some of all the cool new features in C# that are starting to come down the pike, and new features in Python 3.0 -- and all those things are probably going to advance the state of the art in programming a lot more than any of these Object Oriented design principles possibly can. So there. Alright, enough about Object Oriented design. You wanna take, ah, some Stack Overflow questions?

Atwood: Er, sure. I have one that I looked up.

Spolsky: Okay.

Atwood: On my "Favourites" list. And it's something we've kinda touched on before and you know what my answer to this is gonna be before, like, before I'm even done saying the titles. So save all your laughter until the end. But it's called, ah, "Is mathematics necessary for programming?"

Spolsky: Oh yeah.

Atwood: Yeah. And the guy... I'm just going to summarise. This is 157354, so this has been around a while, this is a four-month-old question. He was having a debate with his friend about whether advanced mathematics was necessary for a veteran programmer. Ah, so one guy was saying, "Ah, you only need basic mathematics, from, like, high school". Um, and the other guy was saying that ah, you do need, ah, advanced math, in order to be a good programmer you should understand, sort of, calculus and beyond, because it ties into certain how you design algorithms and things like that. So it was really positioned as a debate, like, you know, how much math do you really need?

Spolsky: That's a good question because math... I think that the reason that this might become a controversial question is that the curriculum of math as it is taught pretty much universally throughout the world is, um, is a very bad curriculum, I think. A-and a lot of people, um, who I trust who are math professors, have told me this. The stuff that is taught -- the fact that we teach a pretty standard curriculum pretty much around the world starting in high school -- algebra, geometry, trigonometry, maybe pre-calculus and calculus -- um, that's a very bizarre way of teaching mathematics, and it usually burns people out when there's so much more interesting stuff going on in math that doesn't happen 'till the advanced levels of math. So for example things like graph theory, algebra theory, sets, linear algebra, all that stuff is awesome and you don't need calculus for any of it. And calculus --

Atwood: What about that -- wait, what about that queueing stuff you were talking about, wouldn't that --

Spolsky: That'd be awesome! That'd be frigging cool if people were taught that. That's not really math, it's too practical to be math. Um.

Atwood: [Laughing] It's too practical too be math. That's too useful. Throw that out.

Spolsky: Yeah. But like optimisation techniques, linear algebra, that kind of stuff, and that's as far as I got, unfortunately, and I have sort of peered at the textbooks that I never got to, and I recognised that first of all the more advanced stuff would be a lot easier, would be a lot easier for high school kinds to learn linear algebra than integrals, for example, or it would be a lot easier for them to do set theory, um, or, um, you know basic algebra, algebra in the advanced mathematical sense of the sort of theory of numbers, um, or, ah, that kind of stuff -- different -- I don't even know what it is. But, ah, I, everyone who's studied math has told me that it gets way easier after calculus, and yet everybody sort of does c-- the advanced students who I know get to calculus, and then just kind of blow up and be like "oh god, if it's going to be any harder than this I quit." And it's a shame that they're burned out of mathematics being forced to learn a particular topic that is actually not very useful. So I would expect that if you asked the question "Is mathematics necessary for programming?" and we think of mathematics as geometry, in which the answer's "no", trigonometry, for which the answer's "yes, for some 3D programming", calculus, "no, not necessary for programming", linear algebra, "yes, for certain optimisation problems", you know, it really depends on what the field is. So much of computer programming is discrete as opposed to, er, uh, you know it's in the integers, so to speak. I don't remember what the first version of Windows is that used a floating-point function, but I'm pretty sure that Windows 3.1 did not have a single floating-point call anywhere in the operating system.

Atwood: Yeah. Just to add to that, I, a lot of Universities, I know where I went to school, the University of Virginia, the, uh, the computer science was tied to the, uh, engineering school, not the math department, which I appreciated because, I dunno, I just have a hard time -- maybe it's because, first of all, I suck at the math, so let's just throw that out there, but I never viewed these two topics as having a heck of a lot in common. I mean, logic, and stuff like that yes, but pure math, like calculus and beyond? No. I really struggled to see the connection. And I think in a practical viewpoint you have lots and lots of programmers who have never really even gone to college. And they may be really good programmers. To say that, oh, you have to know math to be a programmer --

Spolsky: No, that's obviously not the case. 

Atwood: Yeah, you can disprove that, like, instantly.

Spolsky: Now there's a go-- yeah, if I had to decide between a programmer, you know, all else being equal, if I had to choose between a programmer who knew calculus and a programmer who had taken a class in anthropology, cultural anthropology, I would go for the anthropologist, for sure. It would be more likely he's going to learn something about human beings that would be useful to him to design good software, than there's going to be a need to do an integral in code. Except for obviously very specified mathematical -- very specific mathematical code. On the other hand, on the other hand -- you can think of lots of examples, like for example if the Google guys had come up with their idea for PageRank but they didn't have the linear algebra to actually implement it, they may have never made Google. Just the fact that -- and linear algebra was a part of it, the CS curriculum when I was at Yale, so that was one thing, a very specifically mathematical thing, that I was forced to learn in order to be a computer scientist that has once in the history of the world actually demonstrably made billionaires.

Atwood: Right. No, that's a good point, that's a classic paper. Although to me that doesn't even really feel like math. It just feels like logic, where you're proving, okay, if you have these links, and you ca-- I dunno, it just doesn't f--

Spolsky: No no no no, that's true, but then, but if you don't know how to solve it. You're talking about the PageRank paper, the original Google paper?

Atwood: Yeah.

Spolsky: Yeah but if you don't know the math necessary to solve all those equations simultaneously, ah, and realise that there is such a capability and that it's actually not so hard, then you may have just given up on your wonderful idea of using citations, or incoming links. 

Atwood: Yeah, that's fair. I mean we can certainly point to some famous successful people that have a math background, and I --

Spolsky: Right. Sure, and we can point to some that don't so, it's... *laughs* these are just anecdotes.

Atwood: Yeah. So do you have any vague Stack Overflow question?

Spolsky: Let's see, let me look on my favourites here... Did we do significant innovations in computing since 1980? We did that, right? 

Atwood: I did, we just, we talked to Allan [Garbled]

Spolsky: Excuse me. I just forgot to un-favourite it, I guess. Um. Why hasn't it disappeared from my favourites list? Oh. Now it did. How about "Does it matter where you get your CS degree"? And this has become a community question. The question is, "Does going to a less-famous university that might not be terribly selective necessarily precluuude somebody from the elite software companies?"

Atwood: What question number is this so I can...

Spolsky: Uhh... 191302.

Atwood: K.

Spolsky: Google or Microsoft, regardless of my actual ... oh, er, elite software companies which he's considering, like Google or Microsoft, [garbled] "some actual abilities. Furthermore, how often do you find your alumni places a factor when looking for a job?"

Atwood: Well how do you guys look at it at Fog Creek?

Spolsky: We, ah, yeah, er, I --

Atwood: I'm putting you on the spot--

Spolsky: No, because Fog Creek is just one small example, and I think we're a relatively enlightened company, and even then we're not so enlightened. And the answer is that if i had two resumes that were otherwise identical, and one of them had evidence on it  that the person in that resume has gone through some kind of screening process and passed, then I would rather talk to that person first. To save time, I would interview that person first. And that's what I call the selectivity principle. I look for things on the resume that indicate some selection process, of some sort. And that could even be getting into, um, the Coast Guard, or getting into the Marines. Ah, anything where somebody, ah, went through some kind of rigorous selection process and made it through there. All else being equal, and I really literally mean that, all else being equal, then I will talk to that person first. I'll call that person first, and put them through the interview process first, because I think that in the long run that's going to save me a little bit of time, because those people are a little tiny bit more likely to succeed in the rest of our interview process, because they have succeeded somebody else's process, whatever that may be. And so, when it comes to selective schools, we have a very specific principle, where we say ah, we have a detailed list of all the schools in the United States that accept fewer than thirty percent of their applicants. So I think it's a hundred -- I don't know how many schools there are on that list, but schools that take less than thirty percent of people that apply to them, and that's our criterion, and if you get that then you're going to, all else being equal, you're going to get called first. And, um, so you are actually a little bit more likely to get a job at Fog Creek. 

Atwood: It does say right here, the second response, by mattj, "Joel is bashful about being a college snob." 

Spolsky: Yeah. Well you can put it that way...

Atwood: [Laughing] Is that what you're saying now?

Spolsky: Ah, well the truth is that people that didn't go to Yale are just so lower-class... do you really want to have lunch with somebody that went to Harvard, or...

Atwood: The riff-raff, that you really don't want.

Spolsky: Yeah, they're probably going to be mugging you in the elevator to take your money or something. I don't want to have to sit with people like that at lunch.

Atwood: Yes. So really just to summarise what you're saying and make sure I understood it: ah, really the breadth of experience on the resume trumps any one college reference.

Spolsky: There's lots of other things that trump that, lots of ways that you can... for example actually, for us, having experience at Microsoft, because we know that Microsoft has a pretty rigorous interview process, so having an internship at Microsoft to us is just as valuable as getting into an Ivy League school. Both of those get you one point on the six-point scale we have for deciding whether... when to call you. At what point to call you.

Atwood: So really it's just like getting a, uh, a point. It's kind of like when you're in the Boy Scouts when you get a merit badge. Like you get enough of these merit badges, and ...

Spolsky: Yep. And it's never used to make a quote-unquote hiring decision, it's simply used to sort out all the huge quantities of resumes we receive, and call people in the order in which they're most likely to get a job. Just to save time in the interviewing process.

Atwood: So I guess the way to look at this then would be okay, if you're deciding whether to go to Stanford versus some local college that's not maybe as well known --

Spolsky: Go to Stanford. Just go. Do it. Yes.

Atwood: So really you're really coming out in favour of these big-name schools--

Spolsky: Yeah. And it's not because of the big name, it's because of the selectivity. If you look at a list of selective schools, there's a bunch of schools in there you've never heard of, that are small and selective. Um, you know, there's this little school in Santa Fe, New Mexico, where you sit and read books in Latin and Greek for four years. And, you know, 700 people apply and they take twelve. So they're on the list! Because they are...

Atwood: What school?

Spolsky: Somebody write in and remind me of that school. We'll put it in the show notes. Um. Yeah, it's like, their entire curriculum is reading great books, and they just have a curriculum of what they consider to be great books, ah, that they read. And that's the whole curriculum. But anyway, the point is that they're selective, and, similarly, we know certain companies that we know put people through a very rigourous interviewing process, and that is just as good as going to a good school. But wait -- I actually feel, to be honest, that Fog Creek is relatively enlightened. In other words, if I was interviewing somebody, and they dropped out of elementary school in fourth grade, but they were brilliant and they just wowed me, um, there's no question I wouldn't hire that person. I certainly would hire them, I don't really care where you went to school, we have a lot of people here who didn't go to the most elite schools in the universe, and it's not a question of how big the name is, it's just a question of what I can tell from your resume about whether or not I should, ah, interview you first, or I should maybe wait 'till I'm running out of choices and then interview you. And, um, yeah. I feel like there's also a lot of other companies, um, not Fog Creek, that, um, that aren't quite as enlightened and they will literally just hire you. Because there's a name brand on your resume. There are definitely a lot of companies that will automatically hire people from certain schools. Um, so, there's no question that you're going to open a lot more doors if you go to the school that's well-known and that has a brand name. I don't wanna say big name, but really, a -- a name as being either a quality school or a school where quality kids go or maybe just a school with a reputation, or a school that doesn't sound like a state school, or anything like that.

Atwood: Right. So if you can't go to a school like that, then you definitely have to make up for it by doing some extraordinary, or semi-notable thing. 

Spolsky: Probably.

Atwood: That's the way you'd balance that out.

Spolsky: What can I say? You're really going to get your money's worth in your career by going to the better school and you may -- I hate to say this. I used to try to be really, um, a little more idealistic about it in some ways. You know, my parents are professors -- were professors -- and they -- I grew up in New Mexico because at some point they both wanted to be professors in the same city and live in the same city, and that left them with not very many choices because they had to get jobs for the both of them. And then University of New Mexico was on a hiring spree, and so they got jobs at the University of New Mexico, and went there. And, um, ah, you know I always sort of assumed, okay there's going to be lots of high-quality smart kids, all over there place, and there's going to be a whole class of kids that want to go to the University of New Mexico because they grew up in Albuquerque, you know, it's just where they are, and maybe they have family reasons, or financial reasons, to stay at home, and they're going to go to go to the University of New Mexico no matter how smart they are, and how likely it was that they could have gotten into Harvard, and, um, and so I always assumed that the big, I call these sort of regional schools, a regional school is a school that's going to attract people from the region, that could very well have gotten into Yale or Harvard, but just didn't want to leave the region, and my mum said to me: don't over-estimate the kids at the University of New Mexico. [laughing] I could count on one hand the number of students I had that were smart. Anyway. Now I'm going to get in trouble, going to get all these letters from Albuquerque. 

Atwood: Well I think the point is that you should try to challenge yourself, and go where -- universities that are really challenging their students.

Spolsky: It's gonna pay off. It really is. It's going to pay off in your career. That's just the way the world is. And--

Atwood: But the good news is, I think, that there's not just like Yale and Harvard, there is a class. If you look at the top 10% of universities, there's really a wide range. So I don't think you have to go to Stanford. You just have to go to a school that's selective. That's known to be selective and challenge their students.

Spolsky: Absolutely.

Atwood: That's reasonable. 

Spolsky: There are schools -- you could go to like Olin in Boston, and this is something that nobody would ever, has ever heard of, but it's very, uh, selective, and you'll get a better education in computer science, computer engineering, than you would get at Harvard. Definitely. So, go there if you want to go to a small school. And the good news is, I think a lot of people that kind of complain about my snobbism towards schools are working on the assumption that Yale and Harvard are these elite places that the children of Yalies and Harvarites or whatever it is get to go to because their legacies, and it's really just a way of the WASP classes protecting their social -- what's the world I'm looking for -- benefits. 

Atwood: Standing?

Spolsky: Standing, exactly, by letting their children go to these. And, uh, that hasn't been true since 1969. None of these schools -- all of these schools are almost entirely -- their applications are almost entirely based on merit. They've been meritocracies for the last forty, fifty years. So, if you think it's the Great Gatsby at Yale, that's just not realistic anymore. Just not true. So, yah, always stretch. I always encourage people to stretch. Occasionally I get email from like high school kids asking for advice, and I hate to give advice because I don't know anything about them. But if the advice is of the form "Well, I have..." one kid wrote to me and he actually said "I have a scholarship, a complete scholarship to Carnegie Mellon, or I could go to some local school" that I've never heard of. I was like, "are you kidding? Go to Carnegie Mellon." "Nah, but I think it's gonna be kinda hard..." "Just go to Carnegie Mellon! Stop it! You got a scholarship! Off you go!" And then he wrote me after a year and said he was getting straight As and he was really happy there, and a lot of times kids get to -- especially kids from smaller towns and stuff like that -- they get to these big universities where everybody's smart. And they probably got there because they were the only smart kid in their high school and everybody picked on them for being smart, reading Thucydides and stuff like that, and then they get there and everybody's talking about Thucydides, you know? And it's just, all of a sudden it's like, woah, this is like a whole school full of people just like me, that one freak in my high school. And everybody here is like that and it's just awesome.

Atwood: Right, that's how I think of California, actually. We have the Internet that does that as well.

Spolsky: That's true, you could find the other weird people that read Greek literature in the original.

Atwood: Oh, I love that. It's all about finding the weird people that are into the same stuff as you. That's what we're doing right now. Which is why it's so beautiful.

Spolsky: That -- Yeah, we've got 11 listeners now. I think we're in double digits on the Stack Overflow podcast.

Atwood: On that note, we should probably end before we go over too long.

Spolsky: Absolutely.

Atwood: Our closing notes, so we do have a telephone number you can call if you'd like to submit an audio question to us. That is 646-826-3879. You can also email us an audio file, 90 seconds or less, please, at podcast@stackoverflow.com, and we try to get to some of those. We also have a Wiki, for people that can't listen to the audio or don't want to listen to the audio, that'll be in the show notes. If you would like to help us do a transcription, that is very much appreciated. Anything else, Joel?

Spolsky: That's about it. We have a special guest host on next week, Michael Lopp, from the Web site Rands in Repose. 

Atwood: Yeah, that'll be a good one, next week. Rands. 

Spolsky: Rands rands.

Atwood: It's like the --

Spolsky: Rands rands rands!

Atwood: That's it for this week!

Spolsky: See you next week!

[Outro, end]