TW Hangouts | Is TDD dead?

hey everyone I am David Henry Hanson I’m here with Martin Fowler and Kent Beck to talk about is TDD debt as a topic coming off keynote I did at railsconf in some subsequent articles I posted on the blog I had a chance to talk with both Kent and Martin sort of one on one in the days that follow and we got to discuss a bunch of these issues and I got to clarify some thoughts and definitions that I had in my head and I got a lot wiser about where both Martin and Kent are on the issue and Kent suggested that there hey maybe we should share that with the world so that’s what we’re doing now I know a bunch of people have been expecting that this was going to be a big Duke out there will be no boxing will explore bunch of definitions and problems and patents we’ve seen and what’s worked for for us individually and go from there but to get started I thought I just run over a brief recap of the problems that I’ve seen with CVD and use that to kick off the discussion and really three major point that I’ll focus on for this discussion the first is the definition of of TDD and its components the specifically unit testing I know the T to the anew testing is certainly not the same thing but TDD is often used to drag you in tests and as I was looking through sort of issues I’ve been seeing in the rails world around T to be a lot of that came down to the definition of unit testing that people were defining unit testing as it can’t have collaborators you can’t touch the database you can’t touch the file system the goal was to have a suite of unit tests it was so fast that they could complete in a blink of an eye I’ve seen arguments of oh if it takes more than 300 milliseconds to complete your entire suite then it’s way too slow I can’t use it for anything and that just wasn’t matching my useful definition of unit testing and wasn’t matching what I was trying to get out of it and frankly I wasn’t getting anything useful out of unit tests that were talking to any collaborators second which quarter follows from that is is driving your architecture through mocks if you want everything in your test suite to be so super fast well obviously you you have to talk to mocks you can’t talk to anything slow and you can’t talk to too many layers at once which leads to a mock sort of heavy and enabling architecture repositories and and a hexagon patterns and and all these other things that makes it very easy to to stop and mock your way out of everything and turn every test you could imagine into a unit test and VC frameworks as I work with in rails the controller oh no problem we’ll just stuff about everything talking to to the viewer talking to the model and then that’s going to be great I was not seeing that being very great at all this was the the key point where I thought this is test induced damage that the architecture is actually suffering damage from this drive towards unit testing all the things and that the unit testing all the things is coming through the TDD type and then finally the sort of ephemeral point of TVD is just red green refactor loop and that was more of a sort of a personal thing like we can all have our own personal preferences and how we develop things in TDD red green refactor seemed less optional more of a mandate and I never dealt with that for for the bulk of the work that I do at certain patients had worked them at certain locations that flow was good but I felt like there’s tons of code that I was writing more a red green refraction she’s not an enjoyable way to do it and I didn’t feel like I was getting to a better place by going through that so those are sort of the three top line problems that I’d seen with with TDD let’s sort of open the conversation up and and have Martin and Kent chime in perhaps a detective one in time what is what is a unit test what’s the definition what’s your useful definition of a unit test perhaps both what’s a useful definition of unit test and how are most people currently defining it in in your guys’s view well actually I propose that in order to understand something I have to find it’s useful to know how it came about and that the style of unit testing that we’re familiar with was is basically all Kent’s fault

right because Kent did the frameworks the xunit frameworks did the whole push for extreme programming so maybe we start with that where where does the source of this style of unit testing come from you would a guy you explain it there yeah can you hear me yep so when I was a when I was a kid I would read books about programming I was just fascinated by programming I’m talking about nine ten eleven years old and my dad was a programmer so there were lots of books about programming around and he had a book and I would read I didn’t understand anything but I was just sort of compulsively reading these books because I found the whole topic fascinating and there was a book that said here’s how you program you type in the output tape that you expect your program to produce given some input tape and then programming is you just write the program until it produces the output tape that that you expected so that was that was kind of fixed way back in my brain as the flow of programming and it wasn’t until after I had been testing for a while had already come up with the first version of s unit the the small testing framework that was a precursor to J unit and I remembered this idea of writing the test before writing the code which makes no sense right the test is going to fail and you want the test to succeed so why would you write it when you when you knew what’s going to fail so I just tried it you know one of my heuristics is if an idea is obviously bad find a cheap way to try it because if it turns out it’s not bad then it’s really interesting if you have a good idea that’s obviously good somebody else has probably tried it before so I just figured I’d try it and I think I built a stack the first time and and it worked really really well for me I think there’s aspects of my personality that make this this kind of piecemeal style style work really well I have a lot of anxiety I tend to get overwhelmed by big problems and so for TDD so for me TDD solves those problems I can always work even if I don’t know how to implement something I can almost always figure out how to write a test for it and if I can’t figure out how to write a test for it I have no business programming it in the first place so that’s how I got into it it was and it was definitely ground-up that is I started with small grain problems and and gradually worked my way to larger grain problems now so we did your go ahead I was thinking about this and reflecting back on on our early days together and if I recall correctly when we began working at C fiat-chrysler and using extreme programming as it’s now called we did the testing very strongly from the beginning but it wasn’t test first at the beginning I’ve got this memory of you standing up there saying the story’s not done until the codes written and the tests are written and the tests are passing so we have the notion that you deliver the end of an increment of functionality with code and tests together but the testers didn’t actually follow till later I can’t remember when that came because it seems to gradually it it just gradually appeared but I feel that at the very beginning it was the focus was on the tests on the test without the test first as it were so I think one of the fundamental questions is as a programmer do you deserve to feel confident and I certainly grew up in a culture that didn’t encourage that you know you’d write some stuff and you just hope it worked and somebody else would tell you whether they thought it worked and and you’d go home on Friday and just think oh god I just hope I didn’t break something and so I think going a step back from TDD or automated testing it was just – do you deserve to feel confident can you sleep at night knowing that your code works and I think the answer should be yes for programmer so if we agree on that then we can talk about how to achieve that TDD is one way to achieve that but there’s there’s lots of other ways I think that’s a it’s a great point because one of the things

that really got me into programming was some was Ruby right so I’ve been programming before Ruby but here could not come this this language that has a very specific mission statement programmer happiness right so I think those two things are related so the the program our happiness is certainly related to to feeling confident in in your code and confident in making changes to it and confident that’s going to work but it’s only one part of that picture and the three early days that Martin describes were oh you’re not done until you also have tests for a piece of functionality I’m completely on board with that right where I’m not on board is that that whole feeling of feeling great about your development style feeling programmer happiness as as Matz would always put it just wasn’t I wasn’t getting that from TDD right so maybe that’s that some of the patterns we talked about were where different people have different brain styles right why do some people enjoy Python and why do some people enjoy Ruby in and and even harder for me to comprehend how do some people enjoy Java but we sort of have different ways of a looking at things and then feeling good about them and what I was finding just from the red green refactor flow was that the whole notion of writing the test first and seeing that fail and doing that for every piece of code before I move forward was it’s not just a natural because a natural you can overcome just by by familiarity but even through familiarity that it just was not a pleasurable flow and I think that that’s where a lot of some of the explosive mm debate comes from is to some people writing your thesis first and then filling in the implementation of that works really well that’s exactly how their brain operates for other people me included it doesn’t work like that like it’s a much more tactile experience that I actually have to sit in and write it out that’s how I think through it I don’t think through it by proposing the hypothesis up first and first and then then trying to fill it in I think through it by writing it out and can as you say well I if I can’t write a test for it first I have no business writing it that’s I totally understand how that are you can arrive of that it’s just not at all how I feel about it like I can’t I have a hard time writing a test for it if I don’t have to see it first right so you have sort of these some people like it from going through the test and other people like that going to the test I think certainly important to realize we’re both getting to the same place getting to feel confident about the code and confident in making the changes to the code and that’s the part I’m not I’m not a fan of tying these two things together I’m not a fan of tying oh you deserve to be confident too TDD which I know you’re not doing right like TV is one path of getting to that place I’m seeing though a lot of people are mistaken those two things that you can’t have confidence you can’t deliver incremental functionality with tests unless you go through this mandated well paved road of TDD and thank you yeah so so last night was a hackathon at Facebook and and I have project on an internal tool that I was working on and half of it I could use TDD for and half of it I couldn’t so I had a big log that I wanted to process and the the entries in the log were not clearly specified anywhere but I wanted to put them into a data structure this kind of funky funky new data structure the the data structure I could build with TDD just fine because I could see I could I could break it down okay here though here the here’s the main case here’s the corner cases you know here’s how I go from one to many I just nice sequence of tests and uh and I could get into a real flow just make one test work and the next one and the next one the next one so that feels a certain way to me when I had to feed live data into it it feels a very different way because I don’t have that clear specification I just have to run a few million log entries through through this first and see what blows up so I put in I put in logging I put in a exception handling so any log entry that’s kind of funky that isn’t handled by the code gets saved someplace and then I can then I can go back and look at those entries that I can’t handle yet and I can use them but but I can’t I can’t write a test for because I don’t

even know what the inputs look like and that feels a different way to me so with the the parts of the code that I could use TDD for I’m in a real flow I write a test to make it work write a test make it work I look it up and it’s 12:30 so I really like that feeling the the ones where I die just have to run on live data and see what happens I feel a lot more anxiety but you know it’s a that’s it’s called work for a reason so I’m going to use the principles of double checking of regression testing of short feedback loops and try to get the shortest possible feedback loop out of this the unpredictable input that I’m getting so I had to mix the the two styles in in one night and it doesn’t bother me I think you know if I can play classical and jazz and then I’m in a better position to make music with more people but when I can reduce a problem to this sequence of tests does kind of derive the solution from a sequence of tests it has a special kind of feeling for me that works really well if I think back to how I learned a discrete map I always used examples you know there’d be some general you know prove this this statement and I think well what’s an example of that and then I’d come up with a few examples and I’d convince myself either that the statement was true or that was that it wasn’t and then I could write the proof but that’s my personal flow I work from specific to general this kind of inductive style and I understand not everybody’s like that what I think is interesting though is is even if I’ve been in that flow to I have been in a flow work where TDD was working out well for me and it was often as you described word who sort of worked was a very clear input and there was a very clear desired output and there were not a lot of context there was not a lot of environment to operate in it was very pure and for that like I that’s that would probably be the one case where I do actually like that flow like what I find is a a lot of my work is not like that right so it’s sort of fundamentally your Navy not like that and then the problem comes in what am I willing to sacrifice to get there right I can you can reduce any type of work certainly in the web space that you’re doing to that flow if you just that’s where the whole mocking thing comes in right if you just find a way and work hard enough you can remove all the sort of dependencies all the environment and reduce it to just one piece of input one piece of output is the flow important enough that you want to do that as a general case how much are you willing to sacrifice to be NetFlow more the time for more of your work for me the answer has been I’m not willing to sacrifice that much when the flow is natural when the work is sort of one was of algorithmic nature derived from and it separated from all this context all right great let’s switch into that mode but forcing all sorts of other kinds of programming work into this mode just such that we can feel the TDD loop that’s where I really see that that we’re going off track and just coming up with some in my opinion nasty hacks and we start making really bad trade offs for for other aspects of the design of the code and for the productivity of the code so I’m curious for both of you guys like how much are you guys willing to sacrifice to get to that TDD flow are you willing to do use heavy amounts of mocking do you find that it generally you can get there in say MDC style programming or were the boundaries for you guys Martin’s waiting for me to talk so I guess I’m talking so as I see I do that I do this all the time that’s it’s it’s a question of trade-offs and and my students have seen me do that a thousand times so for me TDD is a question of trade-offs and I think David you point out an important part of the trade-offs which is to make designs intermediate results testable comes with a cost it comes with a benefit to and you got to figure out whether the costs outweigh the benefits that’s sometimes that’s a question of understanding how to design things well so if if I’m building a compiler and I think well the only kind of tests I can have or end-to-end tests and then I start building more and more cases of what the input looks like and I realize oh if I have this intermediate

representation of a parse tree then now I can test parsing I now have these two orthogonal dimensions I have parsing and then given a parse tree doesn’t compute the right results and then I start then I can do more testing of a finer grain but it was a design insight that that created that moment and compilers have have settled on that as being a reasonable trade-off okay it’s worth having this intermediate representation so that we can have testability and a bunch of other stuff that comes along with it so I’m not willing to say okay so there’s some fixed boundary and and I don’t want to twist the design just to make things testable because I always figure out I’m just missing some design idea and if I had a better idea for the design then I could have it both ways I could have something that was a more aesthetic more flexible design and it was more testable at the same time but if I don’t have that today David I think your point is is exactly right then what do you do do you do mock absolutely everything my personal practices I’m not almost nothing if I can’t figure out how to test efficiently with the real stuff I find another way of creating a feedback loop for myself I have to have the feedback loop and the feedback loop has to be repeatable but the ice like I just don’t go very far down the the mock path I look at a code where you have mocks returning mocks returning mocks and my experience is if I have if I use TDD I can refactor stuff and then I heard these stories people say well I use TDD and now I can’t refactor anything and I feel like I couldn’t understand that and then I started looking at their tests well if you have mocks returning mocks returning mocks your test is completely coupled to the implementation on the interface but the exact implementation of some object you know three streets away of course you can’t change anything without breaking the test so that for me is too high a price to pay that’s not a trade-off I’m willing to make just to get piecemeal development and this is I think at the heart of much of this is confusion over terminology and what these different things are when I read David’s initial blog post because I didn’t see the his talk until last night and one of the things that came through very clearly was his criticism of David your criticism of TDD and of sort of a design damage that comes flows through it had in itself very much tied in a notion of with this strong desire for isolation and mocking and that ver it’s very important point out that there is nothing sort of within the idea of how you do either TDD or unit testing that says you have to have that kind of isolation some people are very much in favor of it others aren’t and I remember in the early days of extreme programming unit testing people would criticize us the saying hey you’re not isolating your units properly that’s not unit testing and then we have this whole conversation with 24 different definitions of unit testing or whatever it was and said well a style of unit testing we don’t bother about the isolation and you know it’s working well for us thank you very much so that’s one thing that whether the TDD and the unit testing should be tied in with isolation and I look at it as there’s different schools of thought and then I’m with Kent I hardly ever use mocks and but I know good people who do so I don’t want to shoot everybody he uses mocks maybe give it ten more years and then then what Wolff Drummond out or something but we’ll see then is another thing which is a distinction between having what I call self testing code and TDD to me it’s really important to have self testing code the ability to be able to run a single command have your whole system self test itself in an acceptable amount of time um that is really powerful because then if you can do that you can refactor with confidence you’ve got a good chance of keeping you codebase healthy and that means you’re able to be fast to live the new features etc etc etc that is a really powerful thing and if I want to get them on a high horse and say you must do something I might be inclined to get on that particular high horse but for me at

least TDD is one way to approach fat TDD is a very particular technique and if done well one of the benefits is it gives you self testing code gives you other benefits as well with self testing code is one and and for me perhaps the primary benefit and the to get conflated now fundamentally if somebody gets self testing code by another route such as when we started and we were delivering the the code in the tests together at the end of each iteration and you’ve got that self testing confidence then I’m actually pretty happy I’m not going to kind of feel upset that somebody got there by another route and I share with with David in Kent there are problems where TDD doesn’t work terribly well and sadly most of the programming I do these days is not conducive to TDD and I miss it because actually I really like the TDD flow that really works for me just as it works the Kent but fundamentally we have two separate TDD from self testing code and realize I have different benefits that often get conflated together and we have to separate the idea of fully isolated unit testing from not isolated unit testing and realize that people have different preferences around those and that’s where I see that was what much of my reaction was against Wright that we had this prevailing definition of TDD as not from from you guys who are from Kent but from the people currently speaking in the programming environment that I found was a very mark heavy be was very moralistic about the specific technique to get to the place we all want to get to we all want to get to feel confident about magnification to the code having a self tested system that can run as a command and all these things but you you couldn’t be part of that boat unless you also signed up for the TVD road to get there and that’s just that’s where I found it it really frustrating that that we could have talks about whether we were being professionals or not and that went through well are you writing your test first or rewriting them second and related to that that the whole TBD notion was propped off as oh well that self testing code you get oh that’s just a byproduct that’s a side benefit that’s the leftovers from driving your design through the tests right that a lot of people really hinged on to this idea that TPT was not about self testing code that was just a nice nice to have but the key benefit was the notion that you could only improve your design by making more tests about those two things equated that easier to unit test system was a better design system that’s the manes of the heart of the fallacy that I want to take a real big swing at because I started seeing a whole lot of code where this was not true at all the code was not better it was not better designed just because it was more testable it was full of these three streets of mocks as you get to talk about because we were trying to apply the unit test paradigm to two areas of the code that really didn’t fit it very well and we were discarding all sorts of really useful techniques really useful patterns like active record because it didn’t fit this notion of unit tests that had to run in three hundred milliseconds couldn’t talk to the database and thus wasn’t isolated right so we have to throw out all this productive work just so that we could sort of worship at the altar of a test-driven design and that every aspect of our system could be unit tested in isolation that’s where I think we needed a reboot and I think we still need to read we haven’t rebooted enough like these things are still intermingled and while I think we’re all very much at the same page of trying to tease these things apart most people are throwing it all into one big pot they’re calling that pot T V D and T V D means sell tested code it means red green refactor it means MOX because you have to unit test everything it means they speak all of in some ways because I think that that’s what comes out of it the dish in the end it’s just not tasty the code is not great there are other more important sacred principles than just is detestable as in is the system clear is it understandable and all these other aspects of it so that’s where my frustration really is so I get to play Timecop now we said we’d do 30 minutes 30 minutes has come up and it seems that will do and we’ll do another one of these hangouts at some time in the future what we haven’t decided yet but I think one way we could immediately begin is to explore this question in what ways can a desire to TDD damage and architecture and is it damaged right some people say this hexagonal rail stuff is much better than that crap DHH is imposing on us is that

the case how do we evaluate it those sound like good questions to tackle in the next episode alright thanks a lot thank you checking in will set up a new date and and shoot out another link that’s great bye bye bye