Episode 3: Ways to Be Less Clever

Introduction

Tom: Hi, welcome to Why Are Computers, which is the name of a podcast.

I’m Tom Stuart. This is episode three; I apologise for the long wait since episode two. I’m joined today by Sandi Metz and Katrina Owen. Hi Sandi and Katrina!

Sandi: Hi Tom.

Katrina: Hi Tom.

Tom: Sandi, would you please introduce yourself briefly?

Sandi: I’m Sandi Metz. I still think of myself as a programmer, even though I don’t write that much code these days. I’m the author of Practical Object-Oriented Design in Ruby and now 99 Bottles of OOP. Having written books changed my life dramatically, and so now I teach and travel and consult and give conference talks. And I’m a big bicyclist.

Tom: And Katrina, could you please tell us a little about yourself?

Katrina: Yeah. I’m Katrina Owen. I am a programmer — again, I also do other things, like lots of open source, which means support and helping people figure things out on GitHub, and making sure that people are successful contributing to my project. I also work at GitHub on the open source team as an advocate.

Tom: Great.

Rules and Refactoring

Tom: So the two of you recently released, I believe, a beta of a new book. Could you tell me a little bit about that? What is the book, and what’s the beta, and what’s the deal?

Sandi: So first of all I want to discuss that term “beta”.

Tom: Yes. Am I saying it wrong?

Sandi: No no no. I’m sure you’re saying it right for that country you live in.

Tom: That’s right.

Sandi: Yeah, okay. So the beta book that we just released is… okay, I’m going to back up for a minute. There’s a problem with academic books about tech. And Katrina and I actually taught a lot together — we don’t teach so much together now, since we have other avenues where we get to hang out together. But when I first started teaching courses after POODR was released, it became really clear to us that there’s such a difference between knowing things about what object-oriented design is, and being able to write code that’s well designed.

There’s a huge chasm there. And there’s lots and lots of books about how code looks when it’s done, but there’s not many books about the step-by-step decisions you have to make line-by-line when you write code that will let you achieve good design. And we were really interested in teaching people how to do it, not showing them how what they wrote ought to look like. And so this book is an attempt at a new kind of tech book which basically records every decision point about an extended refactoring of code.

Tom: You start out by writing some code to generate the lyrics for the “99 bottles of beer” song, and then over the course of the book you refactor it and make the code better. Katrina, what was the process of writing this book? How did you end up writing a book together, and what was the division of labour about who did what?

Katrina: I don’t think we realised that we would write a book at first. We had discovered this 99 bottles of beer problem that people were writing kind of awful solutions to, and I personally thought that maybe there was no good solution to this problem. Like, everyone was going to sweep something under the rug, and they would sweep some different thing under the rug. And it was kind of fascinating to me the many ways in which this problem could be solved in a terrible manner.

It was on exercism.io, which is the open source project that I’ve been running for about three years. And this was very very early, so this was probably almost three years ago. And I probably said something to Sandi about this, like “I don’t think there is a good solution”, and Sandi probably took that as a challenge. And we ended up sitting down and writing versions of this problem, getting together for hours sometimes, looking at it line by line, trying to figure out “what decision did you make here?”, “what decision did you make there?”. This was my big thing: Sandi would make a decision, and I would say “how did you know? how did you know to do that?”.

Sandi: Well, so I remember exactly what happened. I got up on a Saturday morning and I was looking at the Exercism problems and I did the 99 bottles of beer problem, and I submitted three different solutions, all in one thing. And when I talked to you later about it, you said two things to me: you said, first of all, “no-one ever does that”, because I’d written these three solutions and above each one I had written a comment about “if you think of the problem like this, here’s a solution that would work”.

So I did three completely different things, but they all had a number of objects in them. So we had a conversation later and you told me, “no-one ever submits more than one solution”, and I’m like, “you know, the answer to every design problem is ‘it depends’”. So it depends on all those things. And the other thing you asked me was, “how did you know those objects were in there? how did you make those objects?”, and I said “can’t you just see?”.

Katrina: “It’s obvious!”

Sandi: Right? And so that led to a whole bunch of really interesting discussions between us.

Katrina: I refactor differently from a lot of people, it turns out. I didn’t realise this. I thought I had read Martin Fowler’s refactoring book back in the day, and followed his rules to a tee, and just done that over and over and over again. Now, it turns out I had made some perhaps somewhat stricter interpretations of Martin Fowler’s rules than he meant, and then in the doing I had made a lot of little rules for myself, and a lot of processes that worked for me. And sometime during this process of solving this problem over and over and over again, Sandi saw me refactor and was kind of horrified. Is that correct, Sandi?

Sandi: Well, it seemed appallingly pedantic and detailed. The increments were so small that I imagined that they would be a waste of time.

Katrina: Yeah.

Sandi: For somebody like me, I was saying “well look, I can just look at this code and see the answer, can’t you?”. And you were like, “well how do you know the objects are in there?”, and I’m like, “well can’t you just see them?”, and then you’re doing this refactoring which is killing me, right? When we used to teach together I would do a description while Katrina was demonstrating this process, and I would just reflect all the appalled looks on the faces of the students. I wasn’t really convinced at the beginning, but it was really clear that the process was fascinating.

And we discovered this thing, I think, about the two of us, over time spent together, which we can sort of articulate now, and it was that I am a top-down thinker. I look at things and I see the abstraction. I want to derive the big global rules. I mean, I wouldn’t describe myself as “hand-wavey” about the details, but I have a lot of confidence that the small details will work out if I understand the big picture. I’m way at one end of that scale, I’m extremely comfortable with that ambiguity about how the details are going to work, whereas you’re at the opposite end of the scale. I’ll let you describe yourself instead of me describing you.

Katrina: It’s always fun when you describe me, but I’ll go ahead and do that. Yes, I am at the extreme other end. I don’t even realise there is a big picture until someone tells me that it’s there. Sometimes I might be able to — if someone mentions the fact that there is a big picture, I might be able to sort of slowly back up and find this big picture.

But I have to go from the weeds; I have to start at the smallest details and build my understanding up, step by step by step by step, until I can get to the big picture. And once I’m there, I can be comfortable with some — some! — a little bit of ambiguity, not so much, but a little bit.

Sandi: There’s this thing when people are really really different: the question of how one deals with difference. And it’s been really fun, I think — it’s certainly been fun for me, and I don’t want to put this on you. but…

Katrina: Oh yeah, it’s been tremendous fun.

Sandi: …to have this enormous respect for one another, and have completely different ways of viewing the world. What we found in our extended discussions was that I could watch Katrina work and eventually start seeing patterns. And then the patterns I could explain. I could say, “oh, you’re doing this!”, and she would be like, “oh! yeah! you’re right, I am!”.

We were trying to find a path using her rules that would lead to my objects. Because if we could find that path, it meant we could teach OO without having our teaching technique be “can’t you just see?”.

There are lots and lots of refactoring paths that lead to places maybe that you don’t want to go, that are not fruitful, but the great thing about having a set of clear rules about how to do the refactorings is that they give you safety to retreat. And so you can do these exploratory refactorings, and so what we did was we worked and worked and worked and worked and worked and it wasn’t, you know, the first thing we did is not what’s in the book, the second thing we did, the twentieth thing we did, was not what’s in the book. We’re careful to tell people that following the rules doesn’t make you make the right OO decisions, but what it does is it gives this sort of cloak of safety.

After many many many years of writing code, and just thinking I knew best and flinging code out there, like deciding in advance, and writing code, my style of coding has been very much influenced by the techniques that I learned from Katrina, and then that we sort of evolved… like, I need the abstractions. So, she knew the details, and from that we sort of jointly derived the abstractions. And it really has improved the way I write code. It’s a cool thing.

Katrina: Same here, really. I mean, it has improved the way I write code, because now there are things that I understand that were very very unclear to me. I was perhaps doing some of the right things before, but now I know why, and I can articulate why, and that makes a huge change because now I’m not accidentally doing the right things occasionally, I’m deliberately doing the right things quite a lot more often. I have actually gained the ability to tolerate ambiguity quite a lot throughout this process, and I know that there have been a couple of times where Sandi has told me, “oh, I didn’t think that you would be okay with this”…

Sandi: Mm-hmm.

Katrina: …and now I really truly am.

Sandi: Sometimes you would just be like, “speak to the hand! no! we’re not going to do that! no, it breaks some rule!”.

Katrina: It breaks the rule!

Sandi: And I would be like, well, there’s the spirit of the rule and the letter of the rule… like, many of our best insights were where you didn’t want to break a rule and I wanted us to break it, and we had to reconcile that.

Katrina: And what we discovered was there was a better rule, usually. Or we were able to better articulate the boundaries of the rule.

Sandi: Mm-hmm. That’s it. Yeah. I would say you forced me into better explanations because you were unwilling to be convinced unless I had a good one.

Katrina: That’s true.

Sandi: And you were so stubborn about that.

Katrina: I know it. I feel like apologising.

Sandi: Nah, you shouldn’t! It means the explanation was insufficient if the people who are very serious about rules can’t be convinced.

Katrina: Yeah.

Sandi: And so it’s improved. The explanations were definitely improved by your… I’m trying to avoid the use of the word “bullheadedness”.

Tom: Determination.

Sandi: Determination! Yeah. Your commitment.

Katrina: Thank you. Oh, these virtues, they sound so much better…

Sandi: That’s right. Every coin has two sides.

Tom: Katrina, the refactorings in the book are, as Sandi said, really surprisingly fine-grained and detailed. Do you really refactor that meticulously in real life, or is this taking the way that you normally work to… not a ridiculous extreme, but to an extreme that illustrates the process?

Katrina: Yes and no. If there’s a lot of uncertainty and I don’t feel safe, I retreat to pedantic rules. If I have a level of comfort with what I’m doing and I know that, oh, I can just try it and git reset --hard and I’ll be fine, and I could do it again if I had to, I will skip steps if I feel comfortable with where I’m going.

Making It Easier to Write Software

Tom: For the people who are listening to this who haven’t read the book yet, I thought I would try and very briefly summarise what I thought the content of the book was, and then get the two of you to argue back against that and tell me that I’ve got the wrong end of the stick.

Sandi: That would be interesting, yeah.

Tom: It doesn’t come out and say it directly, but to me the book is advocating for a specific way of writing software, which is firstly to go out and gather your raw material about the problem. This is this “shameless green” of trying to write the quote-unquote “simplest” code which captures all the information about the problem, but doesn’t necessarily contain any useful abstractions or represent its underlying structure. And then there’s this separate phase of what Eric Evans calls “refactoring towards deeper insight” — this phase where you move the code around in a way that uncovers the structure and uncovers the appropriate abstractions.

Although it’s very focused on this particular problem, and a lot of the discussion is about the particular qualities of the 99 bottles problem, it feels like the larger picture is that you’re illustrating a particular way to approach writing software. So my question is firstly: is that your goal, or have I got the wrong end of the stick? And secondly: is that the one true way to write software, or is that just one of the possible ways that you are somehow endorsing or recommending?

Sandi: You’re right, first of all, about the point of the book. Now I travel and look at people’s code. For many years I was isolated in a little bubble where I only looked at my own code, and I, you know, my own code, it seemed pretty good, at least to me. And what that means is, things you’re familiar with seem obvious.

Now that I go and look at other code a lot, what I find is that there’s a real pervasive problem in OO. Like, businesses that succeed have huge messes of code and they all feel terrible about it. They think they’re the only ones. But the truth is, as far as I can tell, that successful businesses all have messes — that’s part of what it means to be in business for a long time, and have apps that succeed. And in those apps they have two pervasive problems.

One is, they’re writing big nested long conditionals — procedures — in OO code, and that code’s really hard to change. The other is that they’ve tried to do OO and gotten the abstractions wrong, and those abstractions are very costly. And they feel trapped by both those problems. The big nested conditionals tend to be a shape that repeats all over their code, so that they end up having to make lots of changes every time a single idea changes; the abstractions that promise you that they’re correct, that are actually wrong, are usually way more complicated and harder to change.

And so part of the goal of this book is to try to make it easier to write software. One, it lets you fix the conditionals in a real mechanistic way, but the other is, it sort of pushes back against that idea that existing abstractions are right, and a way to save money. So there’s a way in which we’re trying to tell people “don’t be so clever”, “you can do less”, “your code could be easier to understand and cheaper, and maybe some of the guesses you’re making about the future are not really correct”.

It feels like as programmers we put huge amounts of pressure on one another to be clever, and we kind of want to back off from that.

Is this the one true way to write software? I don’t think so. The things you said at the first, like, writing it and then using sort of the Eric Evans kind of thing to refactor to deeper abstraction, I think that’s a good idea, we would really advocate that. Is this mechanism the only way? I would say, really it’s just one example of ways to be less clever and more straightforward, and there’s probably a whole set of techniques in that category, and that any of them are probably appropriate.

Katrina: I think one of the reasons that the term “shameless green” resonates so well is that when we write a simple solution that makes no guesses about the future, we often feel a little bit ashamed. It’s kind of embarrassing to show people that code. And I think one of the things is to allow people to write that simple code, to start at a place where everything that we understand has gone into the code, because you don’t actually need to understand very much to write the version that doesn’t have abstractions in it.

Another part of this is that a lot of people have taken the idea of red/green/refactor very very seriously, and so while Kent Beck in his book said — what was it? “Quick green…”

Sandi: Excuses all sins.

Katrina: “…excuses all sins”. He suggests that you should get to “all the tests are passing” quite quickly, and then make better decisions later. But the red/green/refactor mantra has been taken to mean that you should write a test, and then make it pass, and then immediately remove all duplication. And there’s something about that that tends to lead you astray, because you don’t have enough information immediately to be able to make decisions about those abstractions.

Sandi: Yeah! We were trying to figure out: where did red/green/refactor come from, and what is the exact rule?

Katrina: Yeah.

Sandi: And so, Katrina, you went on some sort of archaeological expedition about that, didn’t you? Tell us about that.

Katrina: I was searching around the internet trying to find, what’s the earliest occurrence, when was the first time someone referred to this, and for some reason I had read Kent’s book and not realised that it was red/green/refactor, I think because he used slashes and I wasn’t using “red slash green slash refactor” in my searches. It didn’t come up. And so I was asking a number of people, and finally someone pointed to Kent and then Kent said “yeah, I think it was me”. And then I went back to his book and read more slowly, and realised that it was “red slash green slash refactor”, and that I’d just done the search wrong.

Sandi: And so this is his test-driven development book, which I think is out of print. I had to buy a used copy of it.

Katrina: I can’t remember how I got my hands on one of those.

Sandi: I have it. Everyone should have this book, but it’s a little bit hard to come by.

Tom: There’s something very appealing about trying to persuade people to be less clever. There’s a really good quote in the book. One of the bits I really liked that I wrote down here is when you say: “Programmers love clever code. It’s like a neat card trick that uses sleight of hand and misdirection to make magic. Writing it, or suddenly understanding it, supplies a little burst of appreciative pleasure. However, this very pleasure distracts the eye and seduces the mind, and allows cleverness to worm its way into inappropriate places.”

I think that’s really good, because it is that little surge of enjoyment that you get, that little mental reward from writing something that feels very clever or you’ve done some kind of refactoring where you’ve managed to make — something that was long and had a lot of duplication — you’ve suddenly managed to fold it up into some tiny little package. And you feel very smart and very proud of yourself, and you can tell yourself that it’s DRY and that you’ve got an abstraction. But as you say, it’s quite often that case that you’ve done that too soon, and that that premature refactoring that you’ve done — that premature condensation of the current knowledge into one tiny little packet — is actually going to make your life more difficult later, because there’s more information to come. Which I guess is the point of getting to that shameless green state before you try to do anything clever at all.

Sandi: One thing that happens when you write a book — Tom, you’re certainly familiar with this — it’s like, in class, I never really had to explain exactly how to do that or what that meant or what the point was. But when I started writing it down in the book, I had to figure out what we were really saying. And I realise now that I’ve had to write it down that what we’re saying is you should maximise the number of concrete examples you can easily get before you try to reduce them to the abstraction.

And it goes back to what Katrina was saying a minute ago about red/green/refactor; if every time you write a test you then squeeze the code DRY, what happens is, there might be another concrete example of the thing you’re working on as readily available as the next test, and you should wait and accumulate a handful. If you can get a bunch of concrete examples in a very short period of time, that can do nothing but improve your understanding of the abstraction. But if you DRY things out after every test, what happens is, it makes things that are alike actually look different, such that later it’s difficult to understand what the abstractions actually are.

Simplicity’s Guiding Principle

Tom: I’d like to pull on that thread a little bit, and ask about what the underlying principles are. Whenever someone gives me tips or advice or something like that, I always want to get to the bottom of that advice, rather than just taking it at face value. So I’d be interested to hear: what do you feel are the guiding principles of the things that we’re shown how to do in this book, so the shameless green and the refactorings? Is it something as straightforward as just pursuing simplicity? Or is that simplicity itself a symptom of some deeper principle that you’re trying to follow?

Katrina: I think the simplicity is a result of following a deeper principle. Simplicity is so rarely straightforward! In the book, as we go through the refactorings, the code becomes more and more complex, before it suddenly becomes simpler. And, this is the beta, and we haven’t shown you the part where it gets simpler yet. Right now we’ve gotten to the point where we’ve discovered some abstractions that are very very stable, and that information is interesting and that says a lot about the problem. However, the complexity is still building at this point.

I think that reaching for simplicity is an important idea, but saying that you should reach for simplicity doesn’t tell you how to do it. And so, again, if you ask me I’m going to go to the details, and if you ask Sandi she’s going to go to the big picture, so we should probably both answer this question.

One of the underlying ideas here is: look for things that are similar and make them more alike. And if they’re similar but they’re really not the same idea, you can’t make them alike. And that’s how you’ll discover that they’re not the same thing.

Sandi: I’ll approach this from the the other end of the spectrum. What is the guiding principle, right? Is it simplicity? Gah. Tom, it’s really hard to say! And you have a knack for really… I mean, that’s your thing, right? That you want to really understand the deep roots of the meaning of something. I wasn’t really prepared to answer this question when you ask it, so I’m kind of winging it here too.

And all I can do… I don’t know that I can really answer it, but I can talk about the effect on me of what we’re suggesting. I used to write code by looking at a problem, making a bunch of sequence diagrams, deciding what was true, and then, in the last ten years or so, trying to TDD my way to that truth. And being willing along the way to have my ideas about truth be altered by the results of TDD. But still, I have had a pretty clear notion in every case — the TDD case or the pre-TDD case — I’ve had a mental model in my head already about what the goal was.

There’s a way in which this new technique — like it says, you don’t really know where you’re going, you’re going to collect all the examples, the abstractions. When you need to alter code because a new requirement comes in, if you can’t see a clear path, the exact transformation from the code you have now to the code that will be open to the new requirement, that you look at code smells and just start improving the code. When I draw the picture of this technique in my mind, there’s a place right there where I am making transformations on faith that the code I end up with will be closer to my goal without actually knowing what I intend to do.

And even for someone like me that’s hugely tolerant of ambiguity, I find that step a little scary. What I do find is I use this technique on code, I’m just improving the shape of the code in very small discrete bits, and it works. And so, I have a hard time telling people, “oh, this is really always going to be right”. It feels like a mathematical problem, Tom, and so you can maybe help me with this. There’s this intermediate step where I have a series of rules that, as far as I can tell, empirically from my personal experience, if we follow these rules the code does get better. But I don’t really know whether there are cases where, if we did it often enough, if we would get in situations where following the intermediate set of rules would not lead you to good places, or it would be pointless. Is it even clear what I’m saying?

Tom: I think I understand what you’re saying, and it’s something that came into my mind as I was reading the book. Where you were talking about… in the book you call them “flocking rules”, where you have these very simple rules that you ask people to just step through — not mindlessly, but…

Sandi: Obediently.

Tom: Exactly, “obediently”. And it’s interesting that you call them flocking rules, and you’re making an analogy there to the way that birds flock and insects swarm and things like that. And that makes me think of a lot of things connected with emergent behaviour and chaos theory and complex systems.

It also makes me think of cellular automata and simple computers. Stephen Wolfram has this principle of computational equivalence, that’s about saying that systems that are found in the natural world can do computation in a way that is comparable to all other computers. So even systems that have very simple rules in them can exhibit complex and unpredictable and chaotic behaviour. And I think there is a bit of a tension there between, on the one hand, wanting to give people simple rules, but on the other hand, calling them flocking rules is almost a gesture of recognition that small simple rules don’t necessarily lead to predictable or well-structured behaviour. They can actually, in some cases, lead to chaotic behaviour.

And so I was sort of curious whether you feel that the rules that are in the book are guaranteed to give you the same result every time, or if you feel like those rules need to be applied judiciously and that they have the potential to send you off in the wrong direction if you don’t apply them correctly. How much of a cast-iron guarantee do these flocking rules come with?

Sandi: Cast-iron guarantee. I wish we had one! Here’s what I like about the flocking rules. Again, I am completely aware that mathematically I don’t have proof of this, but I believe this to be true. I believe that if you and another programmer are working on a problem, and you choose the same next direction — like you want to transform your code to extract a class or something, right, you choose the same code smell to remove — I believe that you and the other programmer will write the same code.

Now, I’ve taught this class a lot; it’s not a good example in class because we push people into doing things, but I’ve also done this problem in a couple of big workshops at conferences, where I just let people go. And everybody who had the flocking rules wrote the same code.

I don’t think they have a chance of going badly wrong; however, what they don’t guarantee is that the direction you choose to move following the flocking rules is the right direction, the direction you ultimately wish you’d gone in. What they guarantee is, whatever direction you choose, when you go there, it will either be right or you can safely get back.

Tom: How much of the stuff in this book is hindsight? Are you showing us how to divine the one true path by following the rules, or is it more about giving us the tools to safely explore one possible path out of many, and then be able to backtrack and explore another one? You mentioned earlier about feeling safe, or when you’re not feeling safe, being able to retreat to these simple, small refactorings. So is this more about having a toolset to do safer exploration of the possible design space, rather than necessarily “here is how you are guaranteed to hit upon the correct design first time”?

Katrina: Very much so. It really is tools for the process, not for the outcome. We did learn a lot, and what you see in the book is a much straighter path than the path that we took; however, we also learned more about the rules along the way. And so we realised there were places where we hadn’t followed the rules — the first, I don’t know, half-year? year? And when we finally did follow the rules, we ended up much more quickly at a simpler, easier-to-understand, stable place.

A lot of people want a simple silver bullet, they want the magic pill that will take them to the perfect code, and I think that one of the things that we’re trying to say in this book is that everything is a trade-off, and you have to make decisions. I know they say that tests drive your code, they drive your design et cetera, but you’re writing the test, and so you’re making decisions. And at every single point you’re making decisions. And I think that the rules are a tool in that process of decision-making to allow you to take smaller, safer steps, which will give you more information, that you can then make decisions about.

Sandi: One of the things that happened was there was a feedback loop. We had some rules, we tried to write code according to those rules; we didn’t understand the rules well enough, so we broke the rules in our code without knowing it. And so we did this over and over again, and then we would have this revelation, we would be like, “oh, we’re not actually following the rules here! what would happen if we did?”. And we would circle back, and circle back.

So every time we wrote code, the errors that we made in following the rules gave us a deeper understanding of the rules, and then when we iterated and really did follow the rules, we got better code. And so the process of doing this over and over and over again until we found the path that’s in the book, it gave us not only a deeper understanding of the rules, but it gave us more confidence in the rules. As we got better and better at following the rules, the code got better and better. And that was a real revelation.

Katrina: I’ve used these rules on other problems, and they really are a tool for safe exploration. Sometimes you’ll end up in the same place, doing the same refactoring, in seven or eight different ways, and over time you’ll start to understand the problem better and be able to go take a more direct path to the problem.

Tom: Without getting too recursive about it, one of the most fascinating things I found about this book was almost watching the process of the two of you triangulating your own intuition. You talk about using triangulation in testing and in implementation, to try and figure stuff out, but one level above that, the prose in this book is constantly justifying and explaining various trade-offs, and for me I very much got the feel that this was the result of you two working together — you both have different styles of working, and you both have different sets of intuition, and you’re constantly asking the other to explain and justify those decisions, and then this feedback loop is giving you a better and better sense of how to justify the decisions that you make in some way other than “well, can’t you just see the objects?”, or “can’t you just see the obvious answer?”. And that’s kind of a fascinating thing to watch from the outside.

Sandi: It’s cool that you get that sense from the book. Because that’s exactly what happened.

Katrina: Yeah.

Tom: I think most programmers don’t have enough of a sense of their own decision process to be able to write down that kind of explanation. It’s not a common skill among developers to be able to not only make a correct-looking judgement in the moment, but also be able to justify it with reference to anything more concrete than just what they’re feeling right now.

So it’s very interesting to see, for example in the first chapter of the book, I think, where you’re going through showing several examples of: is this shameless green? or is this shameless green? I think there are three or four different ways of making the tests pass, and for every one of them there’s quite a detailed discussion of “well, is this what we’re looking for? is this simple? is this as straightforward as it could be?”. It has the character of having been the result of reflection; rather than necessarily “I can tell you this immediately about this code”, it feels more like the result of conversation or having to argue back and forth about what the right thing is and why it’s right.

It’s a really interesting thing for any developer to be able to see, that it’s something that it is possible to do. That it is possible to speak about code in a relatively objective way, and to be able to justify things based on… whether it’s metrics, or code smells, or some kind of other, objective-ish measure of whether this code is good or not, I think is a very healthy thing for people to see.

Katrina: I think it’s a lot like doing a code review with people that you respect and disagree with and who are very very smart and good programmers, and then having to articulate those gut feelings, and really start to understand why you make the decisions you make. And you realise that people have very different experiences, and those experiences lead them to make very different decisions.

Sandi: One of the things we talk about over and over again in the book is how sameness — two pieces of code that are the same — that tells you something, but two pieces of code that are different… difference tells you more. Sameness is easy, to see that they probably are the same thing, or they share an underlying abstraction. But sometimes things look different in the code, but really represent the same abstraction.

Difference is much more useful to understand in programming than sameness. And it turns out it’s the same with people. Katrina and I are really really different. My respect for her is so great that I was very motivated to understand why she dislikes some of my code. Her reasons had to be good, and if I could just understand them then it would improve my ability to write code, and I think that conversation, I think, Tom, maybe, is what you’re talking about, that you see in the book.

The Ebook Business

Tom: So how long as you been working on this in total? Is it two years? Is that right?

Sandi: I don’t even want to think. Katrina, when did we start?

Katrina: We started going back and forth almost three years ago.

Sandi: Yeah, it’s been a while.

Katrina: And then we probably started on the book two years ago.

Sandi: I fear a little bit when I say that, because I don’t want to give people the impression that it will be many years before it’s done. It’s going to be done soon.

Katrina: Let’s talk about that. Tom, at the very beginning you asked “what was the process for making this book?”. One of the things is that we know the contents of the entire book. We have the code examples step by step by step for the two final chapters. We know what the ideas are that we need to explain, and we know pretty much in which order we need to explain them.

And so a lot of the work in these two years has been discovering what those ideas are, discovering what order we need to explain them in, discovering what the step by step process of the refactoring needs to be in order to be able to convey those ideas. And so I feel like a lot of the groundwork has been laid, and what you’re seeing now is, in addition to all of the right ideas in the right place and the right refactoring and the code examples, also this enormous process of shaping the language to avoid unnecessary jargon, to not make assumptions about what people already know, and also all of the editing that goes on top of that.

A lot of the work for chapters five and six is done, so it won’t take two more years, I promise. Can I promise that?

Sandi: Yeah, you can promise that. Getting the writing right… like, I have been the final writer, in an attempt to keep the tone even in the whole book, and writing is painful for me, but that’s all that’s left. All the ideas are complete; it’s a matter of getting the English so that it’s straightforward.

I will say, Tom, you’re at fault for the beta being released at this point in time.

Tom: Oh no!

Sandi: You know that, right?

Tom: No! What did I do?

Sandi: Well, because I asked you a year ago. The word was out, and people wanted what we had written up to that point, and I had a conversation with you about: when is it morally acceptable to release a beta? Because releasing a beta means taking people’s money. And you gave me a heuristic that was very useful. You said: it has to be a complete thought that would be useful to people — valuable to people — even if they never got anything else.

Tom: Right.

Sandi: Now, there’s no worry that they’re not going to get anything else, but it gave us a place to put a marker in the sand. I would be interested, actually, in your opinion. Because, have you looked through the whole book? Have you read all the way through chapter four?

Tom: I have.

Sandi: Okay, so does it feel like a complete thought to you? I mean, a cliffhanger perhaps, but a complete useful thought at this point?

Tom: Yeah. I think it’s obviously at that point. If you both got hit by a bus tomorrow — I hope you don’t, but if you did — and this book was left in its current state, I think that there is a lot of useful and interesting stuff in there. I think it will be even better when you get all the way through to the punchline, if only because that completes the motivating example.

Early on in the book, you talk about: we have this new requirement and we need to implement it, and ultimately the whole story arc of the book is implementing this requirement.

Sandi: Yeah, exactly.

Tom: And so it would be a shame if you never got to the point where you show that requirement being implemented, and obviously you will.

Sandi: It would be a disappointment, wouldn’t it? Up til this point it is basically an exposition of those refactoring rules, and a complete long example of what happens if you follow them. And so the first half of the book, there’s a fair amount of OO in there, but it’s a lot refactoring, it’s like the combination of refactoring and OO principles. It gets more increasingly OO-ish, because now that people know those principles they’re just used sort of reflexively, like, they’re used without much talk, and the rest of the book gets increasingly like, many more objects get created, and many amazing things happen that are more in the OO term.

The book was released at this point because it felt like enough content to be morally acceptable for us to take people’s money, frankly.

Tom: So that’s a good opportunity to take a slight detour away from the content of the book and just ask you about: how has the experience of self-publishing an ebook been? Because I know, Sandi, POODR was published by a… let’s call them a “real publishing company”. And it was on paper and everything. But with this one, as far as I can tell, the two of you have taken on at least most of the burden of marketing it, and actually putting the file together, and really doing all of the work, and deciding on the pricing and promoting it.

How has that whole process been? As just two people, have you found it easy to launch a beta of a programming book, or has it been completely different to what you expected?

Katrina: It’s a pain.

Sandi: Okay, it is a pain. But it’s like… what business are we all going to be in? The first thing I feel the need to say is that I have people, and most of the burden of managing the logistics of selling and the marketing of selling fall upon them. If it were not for them, this would, I think, be an almost unbearable burden.

If somebody who was primarily a programmer, like me, asked me for advice about writing and self-publishing a book, I would tell them not to try to do it themselves. There’s a lot. There’s web sites, there’s money, there’s Stripe, there’s DPD cart, there blah blah blah, right, that goes on and on. So that’s one thing. You just have to have help, and without help we would just be overwhelmed.

The book launched last Wednesday. I’ve been been very superstitious. Like, I have been unable to think about whether or not people would actually buy it.

Katrina: It’s the same for me, yeah. These ideas are important. We’ve stumbled upon some simple ways of understanding something that are truly helpful, and that feels a little bit rare in programming books. I feel like a lot of programming books are not talking to me, they’re talking to someone who’s way smarter than me. And I’m pretty smart, but I don’t have all of the background knowledge, right. And so I feel like this book is important in some ways, in that it doesn’t require you to know all the things, it just requires you to know enough to be able to read it, and think about it. But, like, we’re self publishing it! There’s no guarantee that anyone would buy it. And it didn’t feel like we could make any sort of guess about the future in that regard.

Sandi: Our initial impetus to have lots and lots of conversation about these ideas came about because we were teaching together. I have a class on practical object-oriented design — it’s incredibly expensive. Like, private companies buy it. There’s so much demand for the course that I’m booking way off into the future. I keep trying to work this balance between what do I charge for the course and how often I teach it, so that I can be home at my house sometime.

I have a real blue-collar background. I’m the first person in my family to go to college, I learned to write code at a vocational technical school, at a trade school in the US. I found myself at this remove of my life, feeling like I had tons and tons of information that would be really useful to people like me, with my background, and absolutely no way to get it to them, because I had priced myself out of the market. So we were serving this really narrow niche of people that had tons of money, and not being able to get this information to a whole broad swath of programmers who might find it really useful, who are self-taught like I was…

Katrina: And I am.

Sandi: …and like Katrina is. And so that’s what caused us to decide to write a book. And when we first started working on it years ago, we were like… we were so naive. So we would be like, “programming books! they’re too expensive! when we write this we’re going to sell it for ten dollars! ten dollars would be awesome!” And time has passed, over the years… well, Katrina, did you want to jump in and say something about that?

Katrina: Oh my goodness. At some point, we floated the idea of forty dollars. And I was like, “no way, no way, it’s not possible, it’s horrible, I hate when people do that…”

Sandi: “It’s a rip-off to the community.”

Katrina: “…it’s a rip-off to the community, and it’s not going to happen”. And I was bullheaded about it, very very much. And so that was like, no, absolutely not.

Sandi: Well, we were in total agreement. And then time passed — years passed! — while we worked on it. So in the end, we were really struggling with, what is a fair price relative to its value, relative to what other people would pay. We really wanted to get it out to people, but then we stumbled across Avdi’s postcard scheme. And that really solved all of the pricing problems.

You can imagine, it is a huge — marketing, selling and all that — but picking the amount of money you’re going to ask people for a piece of content, it’s a marketing problem, but it’s a moral problem, more than a marketing problem. And so now we finally stumbled onto a way to give the book away, while having people get it for no money but for a little effort. And then that freed us up to price it right where other technical books are priced. That seemed to really totally solve the whole dilemma.

Let me tell you a thing that’s happened last week, Tom, since the book was published. People from countries whose exchange rate is bad keep sending me emails saying: other self-published authors have a program where they give people from Brazil, for example, they sell it for fifty percent. And I just don’t know how to navigate that moral landscape. It’s like, well, is that fair to people who paid full price in other countries? Does every country that has a different exchange rate pay a different percentage of the list price? If someone from that country pays full price, does it have an effect on other people’s prices? Like, I don’t want to make those decisions about other people’s finances. And so now we’re doing a thing where there’s two prices for the book: $49 and free, and for free you have to mail us a postcard.

We’re also doing a thing… we just talked about it this weekend, so we’re getting ready to put it up. We’re going to put some good postcard karma products on the web site. So if you get a free book and you want to pay something, you can just come to the web site and buy karma, and that lets you contribute some money to the postcard program without us deciding what discount you get or deserve.

And so, I don’t know. Tom, I would be interested in your… now I’m going to put you on the spot publicly. What should things cost, and how do we navigate those waters where some people can’t pay, but we want to give them the content?

Tom: It’s really difficult, and like you say, I think the postcard situation is a good one. If I was going to buy a book that was going to make me better at my job, then I think I would be prepared to pay a significant amount of money for that. It’s always been confusing to me how these things are priced, and especially when people expected ebooks to be cheaper than a physical printed book. In many respects an ebook is a more convenient form factor — I would always rather have an ebook, because I can put it on my phone, and on my iPad, and on my Kindle, and I can read it on my computer screen.

I read your book on my iPad, and I made notes in it in iBooks, and then when I sat in front of my Mac all of those notes were right there, and I could put them in an email and send them to you. So that’s very convenient for me. I don’t know how much ebooks should cost, but obviously if you’ve set a price for it and people are paying that price — which I assume they are, I assume you have customers —

Sandi: Yeah. Lots.

Tom: — well, if it’s “lots” then I guess you should have charged a hundred dollars for it or something.

Sandi: The price is sort of based on what other people are charging: Avdi’s books, Jim Gay’s books, other self-published authors. Now I know there are definitely self-published authors who charge less, but there’s also self-published authors who charge more. So, I don’t know, I don’t know. I wish I knew what was right. We would have just done it.

Tom: In some respects it’s misleading to call it a book, because what you’ve really done here is produce a distilled version of the course that you give.

Sandi: The course. Yeah.

Tom: And so the choice that people have is that if they want to come on your course, then they have to persuade someone at their company to find enough money to be able to pay for you to come and give that course in their company in 2018…

Sandi: Yes. Exactly.

Tom: …or they can pay $49 to get… not all of that benefit, obviously there’s no substitute for spending three days in a room with the two of you, learning about programming…

Sandi: Or you.

Tom: Or me. Or Avdi, or whoever else. But there’s definitely some benefit in reading the book. They’re going to get some of the content from the training course distilled down into a little concentrated nugget of information, and the fact that we call that an “ebook” is a little bit of a cultural accident, really. You could just as well call it an online training course.

Sandi: That’s an interesting thing. Had we called it online training, people would find it cheap.

Tom: Right.

Sandi: And it is exactly that. It feels not like a tech book to me. I mean, it’s not exactly a workbook, but it’s much closer.

Tom: You only made the book available in one file format, which was EPUB. Was that a problem for people, or has EPUB won the ebook format wars now?

Katrina: Uh, was that a problem? Yes. Let’s talk about file formats. A few people wanted PDFs. Many many many more people wanted PDFs than we anticipated. PDF is a really problematic format for a number of reasons, and so we have created a PDF now and it’s not awesome but it will get you the content and it will even print, if I recall correctly, per my latest experiments.

Sandi: I believe you fixed that.

Katrina: Alright, so PDF, we have that. Now, in the PDF, the pages are letter-sized for technical reasons, which means that there are fewer pages in the PDF than there are in the EPUB, but it’s the same content, don’t be fooled.

Let’s talk about Kindle.

Sandi: Kindle!

Katrina: Do you want to talk about Kindle?

Sandi: No, no.

Katrina: Alright, okay. Kindle is owned by Amazon, and they have a command-line tool, that is quite good, that will convert an EPUB into a pretty decent Kindle format. However, it is illegal for us to use that tool to generate a Kindle formatted file that we then sell somewhere other than on Amazon.

So we can tell you how to download that tool and do it yourself; not everyone is comfortable on the command line. There is an online service where you can upload your EPUB and they will probably use that tool and then send that to you, and that’s fine because at that point you own the EPUB, and so it’s on you. Or — and this is what we also do — we can generate a kind of terrible Kindle formatted file using a free and open tool, but since the format is proprietary, the tool creator can’t actually know all the details of how to turn an EPUB into a Kindle.

Sandi: Last Wednesday, at 10:30 in the morning, we released this book, and it was only available in EPUB format at that point. Part of that was based on the fact that, in the entire history of Practical Object-Oriented Design in Ruby, only five people had ever bought the PDF. I have access to that data.

Katrina: And that’s in how many years?

Sandi: That’s twenty-five thousand copies of that book have sold, since September 2012. Five people have asked for a PDF. So we’re like, “well no-one’s going to want the PDF, so we don’t have to do that”. And then, we can only make a crappy Mobi. And surely everyone who owns a Kindle will have no trouble converting the EPUB to Mobi. So we’re like, “great, we’ll just do EPUBs, that’s awesome”.

And so then, you know, people started buying it and… turns out we were wrong about those things. People wanted us to give them a Mobi version even if it was bad; nobody really understands about Amazon’s proprietary licensing of that format; and I’ll bet ten or fifteen or twenty people sent me email that first morning, Wednesday morning, wanting the PDF. So now we’re doing the self-published author scramble around thing, trying to make a ZIP file and upload it to DPD cart so that when people buy the book, they get more than one format.

And that was definitely a mistake on our part. Our assumptions about what people would do, could do, would be happy with, were really wrong. As a storefront owner, I definitely disappointed some people on Wednesday; by Sunday afternoon we had a ZIP file up there where we’re producing all three formats, and in the ZIP file is instructions about how to make a better Kindle format, but if anybody out there’s getting ready to self-publish a book, I would tell you: all three formats have to be available from day one, when people start buying the product.

Katrina: And try printing your PDF, a page that includes code in it. Just saying.

Sandi: And I don’t really know… Katrina did something. At one point I accidentally printed out the PDF. I was trying to compare it to the page size in POODR to see how much content is on that 8.5″ × 11″ sheet of paper that the PDF is on. Most books are that 7″ × 9″ format. And so, we were trying to figure out, now the book looks smaller because it’s on these big pages, is it really the amount of content I told people? Are they going to be freaked out by that? But I printed one and it turned out none of the code showed up on the page. It was like, missing code.

Katrina: Including inline. If we refer to a variable inline in the text, that was also gone.

Sandi: It was just missing. Fortunately that got fixed by Katrina, and I don’t even know how.

Katrina: You don’t want to know.

Sandi: So now we fully support all electronic formats. People want print, and they’re insistent on having it now. But it’s a beta, of course, it’s not done, so we would not have a print version at this point anyway. It was interesting to me how many requests for print… I mean, so many people want print that we’re actively trying to figure out how to get dead-tree versions of this when it’s done. And then that becomes a whole new pricing boondoggle, because, certainly for a small publisher like me, I don’t think print is cheap, and I don’t really know how to be fair to the people.

A number of people have asked for it, and I’ve told them it’ll be available at a massive discount. One of the ideas that we’ve been bandying about is selling the print book at cost. You always have to buy the ebook, and then, if you want, you can get the printed copy in addition at cost. I don’t know what it costs to self-publish your own print book, but it wouldn’t surprise me if the book itself plus shipping and handling was $20 or more. I don’t know, to get small quantities of your own book printed, I suspect that’s not cheap. But we’re going to try to satisfy the people who want that.

Katrina: We don’t know how yet.

Sandi: We don’t know how. We don’t know how.

Tom: It sounds like it’ll be a lot of hard work.

Sandi: Well. Yes. It’s a pain. You know, I’m a reader of print books too, so I’m motivated to solve that. But we don’t know what we’re going to do right now.

Tom: Right.

What We’ve Learned

Tom: So I’d like to just cycle back to the content briefly. You’ve been working on this for two to three years. It sounds like you’ve both learned a lot over the period of writing this book, and teaching the material, and having these conversations between the two of you. If you were able to send a tweet or a small piece of paper back through time to yourself, two or three years ago, before you learned all this stuff, what would you say? What is the most condensed version of what you have learned over the period of making this thing that you would like to communicate to yourself in the past so that you can avoid all of the pain of learning that thing?

Katrina: Don’t write a book.

Tom: What about technically?

Sandi: If it’s restricted to a tweet about technical things. What do I know now about programming that I didn’t know three years ago? I would say… I would’ve told you three years ago that I wrote simple code, but I can promise you that now, when I look back on that code, it was too complex. And so despite the fact that we said simplicity was not the guiding principle of the whole thing, I think the message I would send to myself in the past was, “be braver about simplicity”.

Tom: How about you, Katrina?

Katrina: I kind of want to say: believe Sandi when she says that it’s obvious. It is obvious if you can go from where I was to where she is with a known process. And obvious might be an exaggeration. It’s not impossible, it’s not obscure, even though the gulf can seem very very large. But just trust that this is possible.

Tom: Good answers! Katrina, you get exposed to all sorts of different programming languages as part of working on Exercism, but you’ve just spent the last couple of years working on this book which is about Ruby, and object-oriented programming more generally. Do you have a sense of what the big benefit of object-oriented programming is? Now that you’ve spent a couple of years arguing back and forth with Sandi about this, do you have a clarified sense of what is the point of object-oriented programming, and what benefits does it bring above simple imperative programming, or functional programming, or any of these other kind of paradigms? Has working on this book given you some kind of sense of what is the big sell of OO design?

Katrina: I do feel like I have a better grasp of that now than I did, for sure. A lot of people say that “oh, inheritance is bad, and therefore OO is bad”. Working on this book, I’ve discovered that inheritance should be used very very carefully, and in a few situations where it works out quite well, but often that should not be the first thing that you reach for.

The thing that I’ve learned about OO is that abstractions should be discovered, not made up. When you do that, it frees up a lot of your working memory, because it lets you hold bigger ideas in a smaller part of your mind.

Tom: Has working on this book given you a better idea of what it is that’s good about object-oriented programming, or has it not moved the needle at all for you? Do you still feel the same way about OO as you did two or three years ago, or has the resolution been dialed in any?

Sandi: It didn’t change how I feel about OO. I so long ago drank the OO Kool-Aid, that it’s hard to make me happier with it than I am now. I’m already very happy with it.

I don’t mean to say that other programming paradigms are bad in relation to it. Again, the answer to everything is “it depends”. Some problems are really suited for OO, and I tend to work on those problems, and I like OO a lot. I do feel a little bit of lack in my life that I… I’m going to learn some more about functional programming. It’s definitely on my to-do list right now, and it’s hard to know whether that will change. You can argue that my experience is insufficiently broad to make a good judgement, and so I’m trying to broaden it, and see if it changes my mind any.

Tom: But if a beginner came to you and said, “Sandi, I want to learn object-oriented programming, what’s the big deal with objects?”, what would you say to them?

Sandi: I would say: writing code that lets you create objects lets you create a model of your problem in a way that lets you do the thing that Katrina talked about. It lets you give abstractions names so that you can write a program using a language that talks about what your intentions are rather than what the implementations are. You can make objects that hide details behind them. And when you do that, in an object-oriented language, what it means is it gives you infinite freedom to change those details without having to change very much about your code. We talked earlier in this conversation about how I’m a big-picture sort of person, and so it really suits me to have a language in which I can name a bunch of abstractions and then write code in terms of those names with confidence that the details behind the abstractions will just work.

Tom: That’s a very good answer. And it makes me realise that the point you’ve got to in the book with this first beta is almost the pre-OO point. The few chapters that are there are about extracting methods and refactoring conditionals. You haven’t yet got to the point where you’ve actually created any new classes. There’s no instantiation of new objects; there’s certainly no inheritance or composition or anything like that; it could just as well be a Ruby script with a bunch of method definitions in it. It doesn’t even need to be in a class.

When you’re working on the next two chapters to finish the book, I guess that’s when you’re really going to kick into the explaining and justifying and demonstrating the benefits of having multiple coordinating objects rather than just one long script that’s got a bunch of methods in it.

Sandi: Yeah. Exactly. We’ve introduced the refactoring rules, and we’ve used them to identify small concepts, but those concepts are really at this point methods — functions — that could’ve been in a script. Despite the fact that that’s true, it’s not very OO, it’s still pretty procedural — those refactoring rules have been enormously beneficial in isolating those abstractions in a very mechanistic way. I didn’t have to make the abstractions up, I just found them.

It’s like a quart jar full of different coloured marbles. What we’ve done is, now the marbles are clustered by colour within that jar. And so it’s still a procedure because of that. The process that’s going to happen in the next few chapters is the colours are going to move out of that jar into containers of their own, such that then I can substitute three different kinds of red marbles in for the red marble jar, three different little jars of blue marbles for the one jar of blue marbles I have.

OO is what gives me seams and substitutability of new objects, whereas right now I don’t have that. Right now I’ve isolated concepts within a procedure, but when we pull those concepts out into objects of their own, I can make other objects that play the same roles, and get substitutability. I’m sorry, I hope that was not too big a leap in abstraction for this conversation.

Tom: No, that made total sense. Thank you very much to both of you for coming and talking to me about this. It’s been absolutely fascinating to hear about you working on the book. It’s always a real pleasure to speak to you both.

Sandi: It was my treat Tom, thank you.

Katrina: It has been such a pleasure. Thank you so much.