Y U NO TDD

During this year's GeeCON the crew organised an Open Space evening. (An Open Space is a self-organising meeting where the agenda is created by the people attending.) I participated and ran a session on the question why we are not doing Test Driven Development. (Y U No Do TDD?) I am running TDD trainings from time to time and wanted to get more insight where people are stuck with TDD.

Context
As I said, an Open Space is self organising, and only people interested in TDD attended my session. This is a typical problem of communities of practice - only people interested in the topic attend - which results in us living in a bubble. For example long time TDD practitioner Thomas Sundberg and Shirish Padalkar, lead consultant at ThoughtWorks, participated in the discussion. Depending on the background of each individual participant, my original question was understood as:

• Why are you not doing TDD on production work at all?
• Why are you not doing TDD most of the time?
• Why are you not doing TDD all the time?

I collected the reasons not to do TDD during the session which I want to share here. Text inside quotation marks, e.g. "hi" quotes exactly what people said. While the previous three questions are slightly different, the reasons seem to be similar. I grouped the answers. I did not want to contradict or debunk these answers and have to hold back not to do so ;-)

Prototyping
I am "experimenting with something", the "expected outcome is unclear" and "it's only a prototype". Obviously these are valid reasons as Spikes are outside of TDD. These answers usually coming up quickly makes me wonder if they are kind of excuses sometimes. Experimenting with new libraries and APIs is covered further down, so what are we experimenting with? I worked with many developers who would agree that the expected outcome of their current ticket was unclear - because they did not take the time to analyse the story and understand the solution they were supposed to build? Additionally most of our prototypes go to production after all, don't they ;-)

Time Pressure
Another reason - given by some of my clients too - is their need to go fast: "I need to go very fast", there is "no time for that" and I "believe to be faster without it". While they might be wrong in the long term I understand the effects of pressure. One person made it more explicit, while he has no strong deadline, he said "I have a huge backlog, I am stressed". Indeed when I am extremely stressed, I find it hard to maintain a structured approach, especially if a lot of task switching is involved. Besides the needed skill to apply TDD under high load, much discipline is required to endure pressure. In such situations Strong Opinions and Dogma might help.

Missing the Bigger Picture
I am just "writing a script for myself". Maybe there is no need for automated tests when writing a one time script for myself. TDD has a testing aspect - and it has many other aspects like designing software, fast feedback and working incrementally. TDD is not only about testing. Some people miss that or have only partial understanding of the benefits or do not care for these benefits at the moment. The opposite reason is "because I know how the class will look like". Yes TDD is about software design as I said before, and I would like my class to work, too. Some people only want the fast feedback, e.g. using REPL based development and "looking at UI is faster".

Missing Priority on Testing
When starting with TDD, the testing aspect is most visible. After all we have to write a reasonable test first. For teams and organisations with low or missing priority on testing, people are "looking down on testing" and I got answers like "testing is a culture thing", "testing is not a first class activity" and "I am not asked to create a test by my project manager". Indeed it is hard to keep following TDD if it is looked down upon and if there is no time for quality work.

Avoiding Context Switches
There is a certain amount of context switching involved in TDD. Similar to Edward de Bono's Six Thinking Hats, we have different states which we have to be mindful of and which call for different actions. George Dinwiddie created a TDD Hat to show that. Maybe this switching is "not natural for some people". "I don't want to interrupt creative design with verification" and "I prefer staying in building hat and not change to testing hat". Similar one participant said that it is "easy to write code, harder to write tests, so I do it afterwards". I understand and there is certainly an urge to jump into the code and get hacking. I rarely feel that urge and I enjoy pair programming using the Ping Pong style because it enforces the separation of states without any (inner) discussion.

Missing TDD Skills
This is obviously the largest area and there is nothing wrong with not knowing how to apply Test Driven Development: Honest people just say "I can't do it". Many are aware of this problem and seem to be disappointed with existing material and/or look for more material to study TDD: "It is not taught at universities", "there are no good books" and "I am missing real examples". I know from my own experience that TDD is not easy to learn and some people are "scared for life after a bad experience" with it. Now the best way to learn TDD is to have someone show you while pairing with you. Even if there is no pair programming in your workplace, you can still experience it during a Coding Dojo or Coderetreat. Short of that, I recommend Kent's Beck Test Driven Development by Example, which is a short and excellent introduction.

New Language or Library
When discussing TDD and unit testing with a client, he said I "don't know the target technology" and "React is a new technology for us". I had to laugh. To me this sounded like "I got a car and know how to drive forward, but am not able to drive backwards." On the other hand I live at the dead end of a road and I see drivers working really hard to avoid driving backwards. Are they not able to do it? So maybe stopping halfway in the game (of skill acquisition) is natural after all. When working with some new language or working with an unknown API, I specially rely on tests to support me, these are Learning Tests.

It's too hard to test
I agree some things are harder to test than others. "Android is hard to test", "Vaadin is hard to test" and "some libraries are hard to test". (I have not worked with Android or Vaadin, I quote people.) We might need to know more about design to decouple things. This is definitely true for legacy code, as "existing code is usually hard to test". Some people see the root cause, like in "I don't know how to manage boundaries". In such situations we need (to know) more tooling. We definitely "need more tooling to test the UI" as UI is traditionally considered hard to test from a TDD perspective. Still, Steve Freeman and Nat Pryce, authors of Growing Object-Oriented Software Guided by Tests, always start their TDD (outer) loop with an UI test. GOOS is a great book and I recommend reading it if you want to go deeper into TDD.

It's too simple to test
If there are things which are too hard to test, there must also be things which are too simple to test, right? It is "useless to test, it is so simple" and it "makes no sense to test it". Maybe a better description is that it is "unclear what is important to test". From a TDD perspective no such things exist and I guess these reasons arise from the test after process, when looking at each public method and thinking how to test it. Further excessive test isolation, see Solitary vs. Sociable Unit Tests, will cause that.

Barriers to TDD adoption
Here is Matt Wynne's summary of Barriers to TDD adoption from a session during Lean Agile Scotland 2016. I recommend checking out the Twitter thread as Matt added detail discussions on temptation of fast reward, permission and safety to learn, "the egotist" and other reasons not covered by me.


What about test-induced design damage?
Maybe the only real reason not to do TDD is to keep the design integrity of your system. This idea was started back in 2014 by David Heinemeier Hansson, also known as DHH, and led to the whole Is TDD Dead? debate. DHH said that when using TDD code sometimes suffers tremendous design damage to achieve two testing goals: Faster tests and easy-to-mock unit tested controllers and that the design integrity of the system is far more important than being able to test it any particular layer. It is ironic that this never comes up during any group discussion or team interview. Probably because it is an expert level reason. If you followed the debate, DHH knew TDD, he used it for some time and liked it. And then, only then, did he know when not to apply it.

Work Harder

With this article I want to prompt you to work hard(er). What does working hard mean? When I looked for different translations of German Strengt Euch an, I found several ones, which are all suitable: Work hard. Keep it tight. Push yourselves. Put some heart into it. Put some muscle in it. Put your backs to it. Make your best effort. Just play it like you fucking mean it. Obviously hard work needs your strength. It also needs your heart - motivation and dedication. It might test your limits, both physical and psychical ones.

My favourite example of hard work was writing books in the Dark Ages. Monks were copying books by hand, adding drawings and decorations as they went. An extreme example is the Codex Gigas, which was handwritten by a single, anonymous monk. Because the scribe was a monk he may only have been able to work for about three hours a day, and this means that the manuscript including decoration probably took at least 20 years to finish, and could even have taken 30. Now that is a whole lifetime for collecting and handwriting a single book. Imagine the dedication.

Maybe less extreme is the life of master craftspeople. Several years ago I wrote about a master craftsman hand-crafting rakes. He could retire any time, still chose to improve his methods and work hard all day.

Benefits of hard work
Most of you will agree, that hard work pays off. Here is a little TED talk by Richard St. John, who explains why it is so: Hard work is the real secret to success. I vividly remember a professor of Mathematical Analysis during my studies: At the beginning of a lecture he wrote a theorem on the blackboard, followed by an easy proof. The proof looked good and we students understood what was going on. Then he said "Was man leicht kriegt ist nichts wert" ("What you get easily is worth nothing"), pointed out a mistake in the proof and erased it. He used the remaining of the hour to sketch a valid proof which was complicated and much harder to follow.

Hard work has more value
Theodore Roosevelt even said that "Nothing in the world is worth having or worth doing unless it means effort, pain, difficulty..." Maybe he was exaggerating, but a little pain never hurts. (Pun intended ;-) So is the motto in weightlifting training: No Pain, No Gain. Psychologist Katarina Veselko mentioned the topic in one of her talks. During the following conversation she explained: It not so much that we do not value things that are easy to get, it is more about the fact that when we need to invest a lot of energy, time, money, ... into achieving something, the result will be more valuable for us because we have invested a lot into it. It is some kind of Cognitive dissonance. She also believes that most things worth having are not easy to get; most things that are valuable to us are not achieved in a way that is always fun, fast and easy, but also includes obstacles and challenges.

Instant Gratification
The term Instant Gratification is often used to label the satisfactions gained by more impulsive behaviours: choosing now over tomorrow. And today it is harder to delay gratification than it used to be. We have been trained by technology and social media to expect results fast, without much effort. For example, there are more than 2 million hits on Google how to earn 5k extra, many of then offering "work from home, 30 minutes daily" up to "basically doing nothing". The same is true for losing weight without any diet and so on. If you want to read more on Instant Gratification and its problems, I recommend Courtney Ackerman's Definition with Examples.

Conclusion
It seems that working hard has become unpopular. Why work towards a long term goal when many things are handed over on a plate, or downloaded at the click of a button, or ours in twenty-four hours for just £9.99 extra? I do not think that life works like that. Why do I come up with this in my blog? I am writing this because I think we Craftspeople need to work harder. I am not talking about working more hours. In fact working overtime is against the idea of Craftsmanship because we need our free time to reflect, exchange and learn.

I am talking about pushing ourselves more: Following up on that refactoring you did not finish last week. Refactoring mercilessly, always pushing to keep the whole code base clean and consistent, even during architectural changes. I am talking about not accepting lesser standards because of low level, different or even obsolete technologies or environments. I am talking about overcoming pointless bureaucracy or managers who have no idea about code quality.


I know it is hard. Years of arguments with colleagues, fighting superiors, struggling to grow and working broken processes have taken their toll. Sometimes I feel old. But I have to defy them. Let us continue pushing, trying to work harder and make an effort!

Thoughts on Mastery

During summer time I was on vacation in Tyrol which is an excellent place for hiking. A local magazine told a story about Josef Frauenschuh and his business of hand-crafted rakes. Josef is a true master craftsman: He is of old age, many years past official retirement. He creates large wooden rakes of highest quality using different kinds of wood for each part of the rake. His rakes are balanced and lightweight. He uses the best tools he can get, all of them are self-made and after 45 years he is still improving his tools and process. Obviously he does not accept any compromise. Once, when he needed more space to create the shafts, he simply made a hole in the wall of his workshop.

I liked the story a lot. It made me think about our craft and the concept of mastery.

I believe that one major aspect of mastery is age. It is not necessary by itself but rather is a side effect of the time needed to master a subject. Masters are aged because they have been practising their craft for 30 years or more and still try to improve. Following this definition it is obvious that there can not be many masters of software development, because our industry is still young. Only a few like Uncle Bob are working in the industry long enough, whereas most of us have less than ten years of experience. The so-called senior developers with five years of experience are just young journeyman, if at all.

Josef is working hard. Although he could retire any time, he is working five days a week because the demand for his rakes is high. He is a master and his rakes are masterpieces, but does he get rich? I do not think so. Following the story I assume he has to put considerable effort into each rake, and sometimes the wood splinters and he has to start over. Being curious I compared the prices and his rakes sell for around 70 Euro, whereas eBay has some for ten to 20 Euro, although no wooden ones. So yes, the masterpiece is up to five times more expensive than a regular one.

Still it seems that Josef is not earning five times more per hour than other workers. Could it be that master craftsmen are relatively underpaid because they put in extra work to create magnificent things? Maybe his rakes are expensive to compensate for the small number he is able to produce. That does not compare well to our industry. We (developers) are greedy. We expect a good salary matching our experience and competence. The more experience we have, the more money we want, which seems only fair. The never ending demand for IT professionals spoils us and salaries rise during the first years of junior developers' careers. At least they rise up to a certain point, which might be 15 years of experience, when we become "too senior" for most employers. But this is another story.

I do not know Josef Frauenschuh but I believe him to be a true master craftsman. Instead of bragging about his achievements, he is modest and admits that sometimes he even has to listen to outsiders to improve his rakes. He does not need titles of seniority, nor is he proclaiming himself to be a master. I guess he considers himself still a simple carpenter. He is a great example. Some of his modesty would suite us software people well, and I will start with myself.