A: So, we have a minute – I want to talk about one Stack Overflow question which I love. It's called "What is the most ridiculous pessimization you've seen?". And it's 312003. And what I like about this that pessimization is the opposite of optimization [laughter]. Where you think you are optimizing, when you are actually pessimizing [laughter].

S: Right, right, because of some crazy idea to make something bad, like you gonna [??] some loop - yeah.

A: Le dorfier has some good examples, let me give you some examples that I think were good, with 12 upvotes. [reading] "Databases are pessimization playland. Favorites include: Split a table into multiples (by date range, alphabetic range, etc.) because it's "too big"; Create an archive table for retired records, but continue to UNION it with the production table." [laughter]

S: [sarcastically] Oh, we've all been there! [laughter].

A: "...Duplicate entire databases by (division/customer/product/etc.)"

S: You've got the one for the IT questions, and one for the programming questions, and you've just duplicated the whole database.

A: Yep yep. [still reading] "Resist adding columns to an index because it makes it too big; Create lots of summary tables because recalculating from raw data is too slow; Create columns with subfields..." ...and there's a bunch of other stuff. But, yeah, it's great. And there is the downside of "optimization". 

But there is a dissenting opinion, in fact the highest upvoted answer. He says "premature optimization is the root of all evil" is actually overused. And he thinks it's an excuse to not to take performance into account until late in the project. And I think I can see this.

S: First of all, I have to agree with that. It's not just that you do have to take optimization into account when you design your architecture – I hate to use that word – but when you figure out basically how your program is going to work. You have to think hard about whether the basic layout of your code even has the ability to ever be fast enough for what it needs to be. That's not "premature optimization" – there's a lot of stuff to do initially, like deciding what major data structures to use, even what programming language to use; what your general strategy will be. 

Think about if you sat down to make an email program and you, just for any reason decide "you know, I'm not going to prematurely optimize everything, I'm gonna just put every email into its own file."

A: [laughter]

S: "Later I can optimize to try to make it fast." And there were a lot of email clients that did exactly this. [as a passing remark] *UNIX*. And that means that to show the inbox, means you have to open every one of these files and parse them, and find all the headers and then you can show the inbox. And people will a thousand, two thousand things in their inbox, at least. That's going to be slow, to open a thousand files. And you should never have built it that way. That's something where premature optimization would not have been premature. On the other hand it is certainly true, that there's entire categories of optimization that you shouldn't bother making until you actually know that are going to cause you to be slow, and that they are actually going to be problems. There are some things that you can foresee and some things that you can't foresee.

A: No, I agree. I think I would characterize this as "try not to do anything too dumb upfront." It's those dumb decisions that, in retrospect, make you say "man that was dumb", you know? There's stuff that are borderline. "We could do it this way, we could do it that way..." As long as it's not a way that's going to be the dumb way later. I really think that that's important. Because you can kinda optimize stuff later but I think you're right, if you've come to the fork in the road, and you make the left turn into Albuquerque, then you're kinda screwed.

S: Give me an example where you thought you were optimizing, and you weren't. Do you have an example from your career?

A: Oh, this happens all the time. Well, I wrote the Shlemiel the painter -post, right? That was about that. I thought I was all clever...

S: ...wait, I wrote that!

A: Well, you wrote that, but I then took it, because I was trying to figure out if lots of small string concatenations are as slow as...

S: ...I'm docking your salary. [laughter]

A: But I spent all this time ... cause we spend all the time dealing with strings, that how websites work. For example, the user signature block, it could be called a hundred times per page.

S: Oh, I see. Uh-huh.

A: That's a lot, for me. I think, a hundred times per page, times number of pages we serve up. But after I ended optimizing, and I did optimize so that it got quite a bit faster. Then I realized that it really didn't matter, because the total time saved from my "optimization" was 3-4 seconds per day.

S: So that's almost a failed optimization, not really a... negative one.

A: Oh, I made it worse? It's not really a pessimization, where you go and optimize, you actually destroy something... [laughter] I'm sorry... I don't think I really have done that.

[62:28]