David revealed that he’d actually read a third book, Test Driven by Lasse Koskela. He said it was not a bad book but he wasn’t convinced by the part which talked about driving development through acceptance tests. The section on test driving the database was fine for simple applications, but in David’s experience it is not always simple to swap the database used in live with an in-memory database such as HSQL. Often the application will use stored procedures or other specific features of the target database. Inigo pointed out that if you have a requirement to support both Oracle and SQL Server, say, then using a third database would actually not be such a big step.

Inigo referred to a study by Microsoft which showed that TDD took 15-35% longer than traditional development but was 40-90% better in terms of bug counts.
He suggested that it is the fact that the tests are written at the same time as the code that is important, not the fact that tests come first. David suggested that you lose some advantages if you don’t follow the rules. Writing the tests first mean you always know where you are and that every feature should have a test. Tom said that be coded test-first when bug fixing, but found that doing true TDD was a bit tedious if followed to the letter. I mentioned Ping Pong programming, where a pair of developers play a game by adding a failing test and getting their partner to make it pass and add their own failing test. The idea is to try to write as little code as possible. There was some surprise that this was not mentioned in the books we had read.

We talked about the Kent Beck book. Inigo found it to be a little lightweight and quite philosophical while Miquel thought it was easy to read. Inigo thought it had extra sections that were not strictly needed in a book about TDD, for example the chapters on refactoring and patterns. However, he liked the design of the code that is explored in the book, which is a currency conversion application. Inigo had expected some interesting details of JUnit 4, but once he saw the copyright date he realised that this was not possible.

I found the Dave Astels book felt quite outdated. It is quite a few years old now, so many of the technologies such as JUnit and mocking have moved on quite significantly. It could also do with some typesetting love as it is not the most beautiful book to look at. I also felt that it was rather long – I couldn’t bring myself to read through the extended example which makes up most of the book. Julian (as a newcomer to the subject) did feel that it was good that it was so practical, unlike Kent Beck’s more theoretical approach. So perhaps it is the better of the two for someone new to this area.

We moved on to talking about different testing techniques. I mentioned Jumble, a mutation testing framework which I have described on my work blog. David talked about how he used Fuzzing in his previous job to find bugs in an XML parser. His experience was that it was best to have a smart fuzzer that understood XML as this was more likely to find bugs than just adding random noise to a file.

Inigo mentioned Hamcrest matchers and the use of assertThat in JUnit. It was generally agreed to be good for readability of code and error messages, but not so good with respect to IDE auto completion. David pointed out that JUnit 4 has this problem over JUnit 3 since the assertion methods need to be statically imported.

Finally Miquel asked about how to test JavaScript. Tom and David described Selenium/WebDriver as the best tool in this area.

Miquel started off by saying that he found the book quite hard to read a possibly and little repetitive. I agreed that it seemed to be longer than it needed to be. Miquel did like the mind maps that appeared at the start of each chapter though.

We talked about the idea of having the tester be a bridge between the customer and the developers. I liked this alternative approach to the traditional idea of having testers simply being the last line in the chain before software goes into production.

For me, the most important part of the book was the idea of the testing quadrants. They act as a checklist which make it easy to consider what kinds of testing you are or are not doing and if you are doing enough. There are 4 quadrants, along 2 axes. Tests are either business or technology-facing and either support the team or critique the product. There is a diagram in this blog posting which describes it a little more.

Other things which I found interesting were “Wizard of Oz” testing (p138), Ripple Effects (p143) and Exploratory Testing (p197).

Wizard of Oz usability testing is a prototyping technique where a paper mock up is animated by testers simply using pieces of paper. So it is like traditional prototyping but with the addition of a limited amount of interactivity rather than just a static picture.

The idea of “Ripple Effects” is to try to avoid unexpected knock-on effects of changes made to a system. The idea is to explicitly run through a checklist of existing areas of functionality and consider any impacts. This very simple thing is done to avoid the shortsightedness that can sometimes arise when an agile project focusses on each story as it comes up without considering the larger picture.

Finally, I liked the comparison between Exploratory Testing and the Agile Manifesto as explained by Michael Bolton. Since I didn’t know too much about Exploratory Testing before reading this, it was intriguing to see the parallels. There is an article by James Bach at Stickyminds that explains the concepts quite well.

But now I have a book sitting on my desk that I’d like to read, but I need a push to get started. So maybe we could wake the group from its sleep to cover this? It is Agile Testing by Lisa Crispin and Janet Gregory. Ideally I’d like to discuss this in the first week of August. Anyone else interested? UPDATE: Let’s pencil in Thursday 30 July!

[UPDATE: It looks like Agile Estimating and Planning is the winner in this contest, although Miquel made the election more complicated by not listing his preferences in order.]

Miquel began by saying that much of it was common sense. Tom agreed, but said that it was one of those books people say should be given to novice programmers. But then those beginners would probably miss out on the subtleties, so the actually time when it would be useful would be rather limited. Inigo thought much of it was too obvious, except for the handful of things he disagreed with, such as the variable state pattern and his views on final variables. On the other hand, he thought that there could have been more discussion about immutable objects and functional-style programming. It was generally agreed that they weren’t really ‘patterns’ as such – Inigo thought they were mostly ‘idioms’.

On the plus side, Inigo like the idea of the book and wanted to like it more. I agreed and liked the back-to-basics approach of saying “hold on why do we even split logic into methods?”. I also liked the slightly different way of thinking about the reader of your code. Kent Beck asks what the user will read ‘between the lines’ because of the implementation approach you have taken. Inigo liked the way he admitted his ignorance about such topics as concurrency and referred the reader to other books.

It was generally agreed that the book’s Java focus might be a surprise to someone picking the book up cold. I found the performance measurement section to be out-of-place. Inigo thought it would have been much more useful to have explicitly given details of hardware, operating system, JVM version etc. Miquel pointed out that speed is one thing, but maybe the memory footprint could be important for large collections too, but that is not mentioned.

I wasn’t sure that the use of the word ‘Symmetry’ was correct (see p15). To me, something can have parts of differing sizes but still be symmetric. I think the word ‘Uniformity’ would be better and in fact that very word is used by the author on p101 in relation to JUnit. Tom said that this concept was one of the few things he had taken from the book and he’d hoped for more. It seemed to him that some of the patterns were just dictionary definitions of the term, such as the ‘patterns’ around exceptions.

I think I was the only person to read the part about frameworks. I thought this was quite interesting though, as it discussed the differences between this sort of code and application code. In a nutshell, sometimes you might make a framework uglier on the inside to keep it clean on the outside and to limit difficulties when you make changes in the future. Kent Beck seems well qualified to talk on this, having been involved in the move from JUnit 3 to 4.