Asteroid L

Here's an excellent article about why you should be doing Test-Driven Development.

No, really, it's excellent; go there and read it, then come back here.

A little harsh, isn't it? But very true. It's excellent.

However, something in it made me a little uncomfortable while reading, and it wasn't too hard to figure out what.

There's a lot of people out there under the misconception that unit tests and TDD are a QA method, and that if they do it right their software will have no defects (or “bugs”). That's a dangerous misconception. It's bad for your software, because it won't work; and it's bad for TDD, because when it blows up in your face, there's a pretty good chance you'll go out there telling other people that TDD doesn't work. It does work; and it probably did work for you. It just didn't do what you were mistakenly expecting it to do.

Now, if you will, go back to the article and search for any instance where Uncle Bob tells you TDD will make your software defect-free. He never claims that. The closest he says is “your software will work better”, which is true; TDD reduces bugs a lot, but most TDD champions (at least the ones who know what they're talking about) consider that a nice side-effect at best. (So if he doesn't make the wrong claim, why am I uncomfortable with the article? Because I can easily see proponents of the “TDD as QA” misconception misusing Uncle Bob's article as proof that they're right.)

TDD is not a QA tool. TDD is a development process, I'll even say a programming process. Its main benefits are, in order of (IMO) importance and relevance:

1. Clearer and cleaner design. I'm talking about technical, architectural design, not visual. By forcing yourself to write down what you expect the software to do in a formal language (code), you come out with a clearer idea of what you're going to do; and by designing your internal APIs so that they can be easily called by unit tests, you end up with more modular and maintainable structures.
2. Cleaner code. I've seen people whose unit tests are confusing but production code is crystal-clear. That's obviously not ideal, but it's much better than confusing production code. By focusing most of the effort in writing the test (therefore understanding what you're doing) and then writing the simplest code that makes the test pass, you make it harder to write convoluted code. (Harder, not impossible.)
3. More confidence. Once you've written the test and you're confident that the test expresses the problem, you'll understand exactly what the solution is, and later after the code is written and deployed, you'll trust your old code a lot more.
4. More reuse. To be honest, this isn't even about writing the test first, but in fact there's a step that often comes before writing the test: looking at the appropriate test file, reading the other tests, and checking if what you want is already there. (Because, you know, you need to find the right file in the tree and the right place in the file to add your test.) If there's something that does almost exactly what you want, and that you had never seen before, you'll write your new test and modify the existing functionality. If there's something that does exactly what you want, you save time and don't increase the code complexity.
5. Faster. This is almost always difficult to claim, but it really does stand to reason. Think about the other benefits above; they alone make your coding a lot faster already, enough to offset the time you spend reading and writing tests. You'll end up writing less code, because you know exactly what you need and you won't write fluff. You'll end up rewriting your code less as you iterate, because writing the test made the solution clear to you. Writing code is much like the scientific method; you come up with a working hypothesis, check if it works, adapt as necessary. It might feel like we spend most of our time (in the non-TDD world) writing code, but in reality we spend most of our time figuring out stuff, followed by checking or rewriting code. Clearer code reduces time spent on the former, and writing your verification first as code reduces the latter.

As a nice side-effect, TDD also reduces defects. It does that by (a) making the design and structure cleaner and clearer; (b) making the code cleaner, therefore easier to work with later; (c) encouraging the programmer to think about the problem being solved and write “the right code”. See a pattern? And yes, (d) preventing regressions on the unit level by keeping the unit tests around to run later. But let's be honest: how many regressions are at the unit level? If your answer wasn't “very few”, there might be something else wrong with your process.

Now here's a few reasons why TDD will not take you to the magical no-bug land:

• Each unit test was written by the same person who wrote the corresponding code. Therefore, any misunderstanding of the problem, incorrect assumptions, or weakness in skill (come on, we all have those, that's why we work in teams and continuously learn from each other) will be reflected in the test as well as the code.
• It won't catch bugs on the feature/functional level; combine them with acceptance/functional/customer tests.
• It won't catch “subjective” bugs, also known as poor design. Even if the acceptance tests exist and say the software should do X and do it in such and such way, it takes a critical human to look at the running software and realise it's stupid to do X in practise. That sometimes takes the form of a technical or business attribute, but quite often it's visual or even aural. How often have you heard an UI/UX designer tell you something like, “I know we consistently paint this kind of widget red everywhere else, but now that I'm looking at it in this instance, it looks stupid”? Or “This is actually not related to the stuff around it, it's related to that stuff on the other side of the screen, so it should be over there”? How do you write an automated test for that? Or for the sound coming out crisp? Some classes of bugs need to be found by a human first, and then you can use that information to write an automated test.
• Negative tests. Strange corner cases (“what if the customer is in a zip code with extra sales taxes and extra shipping cost and uses a coupon?”) also fall in the category of, someone needs to think of it first before an automated test can be written. Some developers are really good at this, most aren't. Testers are trained to think this way and will come up with this kind of thing. More importantly, trying to come up with all the negative tests in the testing phase of TDD is really tedious, and can easily make the process so slow as to justify the arguments of the detractors. In the semi-official TDD graph (found in many places, but for example at Ward's Wiki) you'll see that testing is not only done first, but it's done between every two steps; the finding of corner cases should happen between coding and integration, and between integration and deployment.
• In a perfect world, you'll write your software from scratch, using TDD from day one, and should you decide to use any third-party libraries, you'll chose only ones that are fully tested (or if you're an open source or Free Software kind of person, only Free/open source libraries with full-coverage test suites). In real life, you'll be working with legacy code that doesn't have full unit or acceptance test coverage, so manual/exploratory testing will be essential to prevent regressions in pre-TDD features. Hopefully, as you find those regressions, you'll write new tests; but it will take you years to get enough coverage to be fully confident, if ever.

Conclusion: TDD is great for developers and you should use it everywhere. But it's not a QA strategy.

This is in response to yet another attempt at artificially limiting distribution of information online to protect expired business models, the AP's NewsRight.

I originally wrote it as a rather large rant on Google+, but I guess it's too long for that medium, and probably worth blogging.

Aggregators provide a hugely important service both to me and to you. In this day, information is global.

It used to be the case that I'd be more likely to get my information from a local outlet; a paper published in the town or city where I live, or maybe a local TV station. These would often republish stories written somewhere else, and there was a very well thought-out system for them to pay for this.

Now I have access to information from the whole world. But that means, there's way too much of it out there. Attention and “eyeballs” have become a more scarce and precious resource than content. Why would I read your article, rather than someone else's, or even spend my time playing games or writing fiction? I have precious little time, and it's mathematically impossible to read everything written every day that could be interesting for me.

Then there's management/economy theory. The “new wave” of theory today is “consumer delight”. It used to be the case that most business defined their goals as “providing what their customer needs”. Then at some point in the 20th century the thinking changed to “making money”. Then in the 70s it changed to “creating shareholder value”. Some very smart people today are saying those goals are destructive, to the economy in general, to the customer, and to your own ability to compete. The idea is that the ultimate goal of a business is to not only provide what the consumer needs, but to do it with as much excellence as you can afford; the money you make is a means, a part of the process, necessary to sustain the business and the people, and not the ultimate goal.

From that angle, your ultimate goal is to write the best story, and your ultimate metrics of success are second that it gets read as widely as possible, and first and foremost, that the people who read it get the most value out of it.

Therefore the concern at the center of your business is how stories get produced; that is where good practises need to be preserved and new things need to be tried and optimisations made. The concern of how to get compensated is necessary but secondary, and that means it should be an option at any time to rethink the business model, turn it upside down even, if that's the best for the primary goal.

Back to aggregators then: how am I supposed to know about your publication? If once every two or three months (and that's being generous) you publish an article that's the absolute best about a topic I'm interested in, am I supposed to visit your website every day just because that chance exists? That would mean visiting dozens of websites every day to get my news. I'm more likely to go with a smaller number of sites that have inferior articles but a better average.

Aggregators are there to save both of us: if I can find a good aggregator that picks those good articles from you, that's great, because it's probably the only way that article will make its way to me; you get read, and I get better information.

Now, that is currently a problem, because your model for compensation depends on people visiting yoru site. Can you see my point of view, that in light of all this, the thing that needs to be fixed is your compensation model? That the compensation model is the one weak link here, the one thing that is clearly wrong?

It's like the debate about how much profit is lost because people download music and movies. The reality is almost none, because those people are in 4 groups: (a) being most of them, wouldn't have bought the content anyway; (b) already bought it and want it in a different format; (c) download, taste, and then go ahead and buy; and (d) the very few that would have bought it if they couldn't download. So in the majority, it's not a case of buying or downloading, but rather downloading or ignoring.

In the case of news it's not a choice of aggregators or going to the source, it's aggregators or not hearing about the article at all. So from the point of view of the aggregators, you should be paying them for getting your article to the right eyeballs out there. (Which of course is also preposterous, because before you can pay the aggregators for that service, you need to make money somehow, and it's in their best interest to help you figure out how, and help you implement whatever solution turns out to be practical.)