How does testing fit in?

Unfortunately it was just Miquel and me at this meeting to discuss testing and how testers fit into an agile project. The discussions were based around Agile Testing by Lisa Crispin and Janet Gregory. I’d managed to read about half of it, while Miquel had managed a little less.

Miquel started off by saying that he found the book quite hard to read a possibly and little repetitive. I agreed that it seemed to be longer than it needed to be. Miquel did like the mind maps that appeared at the start of each chapter though.

ponte vasco da gama by rolhasWe talked about the idea of having the tester be a bridge between the customer and the developers. I liked this alternative approach to the traditional idea of having testers simply being the last line in the chain before software goes into production.

For me, the most important part of the book was the idea of the testing quadrants. They act as a checklist which make it easy to consider what kinds of testing you are or are not doing and if you are doing enough. There are 4 quadrants, along 2 axes. Tests are either business or technology-facing and either support the team or critique the product. There is a diagram in this blog posting which describes it a little more.

Other things which I found interesting were “Wizard of Oz” testing (p138), Ripple Effects (p143) and Exploratory Testing (p197).

Wizard of Oz usability testing is a prototyping technique where a paper mock up is animated by testers simply using pieces of paper. So it is like traditional prototyping but with the addition of a limited amount of interactivity rather than just a static picture.

The idea of “Ripple Effects” is to try to avoid unexpected knock-on effects of changes made to a system. The idea is to explicitly run through a checklist of existing areas of functionality and consider any impacts. This very simple thing is done to avoid the shortsightedness that can sometimes arise when an agile project focusses on each story as it comes up without considering the larger picture.

Finally, I liked the comparison between Exploratory Testing and the Agile Manifesto as explained by Michael Bolton. Since I didn’t know too much about Exploratory Testing before reading this, it was intriguing to see the parallels. There is an article by James Bach at Stickyminds that explains the concepts quite well.

Organizational Change

It was a bit of quiet meeting – just me and Tom turned out for this one. This discussions were around the book Fearless Change by Mary Lynn Manns and Linda Rising.

Tom began by saying that he was a little impatient reading the book. He wasn’t sure if this was because the book wasn’t getting to the point quickly enough, or if he was just short of time to read it. I pointed out that there is an Appendix that sums up the patterns and shows where they fit together. I think it would have been even better if there had been a map to show the connections between the patterns and the order in which they are applied.

I mentioned some of the patterns I found to be most interesting. In particular, Ask for Help, Group Identity, Piggyback and Just Do It. Once you read them, they seem obvious, but perhaps when you actually need to apply them it might not be so clear.

Ask for Help is just that and is a call to go against the grain and actually seek out others to help your cause rather than assuming that you need to do it all personally.

Group Identity is about forming a team or a movement around a name. I mentioned IBM’s Black Team, which I believe is mentioned in Peopleware. I’m not sure what Tom made of this, especially the idea that they would start wearing black to form an identity. But it seems like they were effective.

Piggyback is where you use pre-existing meetings and other structures to help move your idea forward. There was a story about starting a fringe conference just after another one to improve the chances of it getting attendees.

Finally, Just Do It is a call to get on and make changes. This seems to fit in with the maxim that it is “Easier to ask forgiveness than permission”.

Tom wondered if the book needed to be quite so big. I thought that a few patterns seemed to be very similar, such as Corridor Politics and Whisper in the General’s Ear. Both of these are about influencing key decision-makers in an informal situation. I might have preferred it if the initial twelve chapters which introduce the patterns in their settings had been supplemented by brief descriptions of the patterns. I’m not sure that having an alphabetic list is that helpful.

It sounds from the above that I didn’t like the book. But I did. For more, take a look at the interview with Linda Rising at InfoQ.

Finally, Tom had a flash of inspiration that this book fits into a tradition of management patterns books started by Machaevelli and The Art of War. I think that was the most profound comment of the evening.

Meeting 20 – Agile Retrospectives

We had a meeting last night at the Lamb and Flag to discuss Agile Retrospectives by Esther Derby and Diana Larsen. I was joined by Miquel, Steven, Mick, David and newcomer Graham.

Half of us had not read the book, so I explained the structure of it: The first three chapters explain what retrospectives are, how to structure and run them. The next five chapters are essentially a set of patterns. Each chapter contains a series of activities that can be carried out at each of the stages of the retrospective. The book ends with a chapter on larger scale release and project retrospectives and a closing one on supporting the changes that come out of the process.

Miquel commented that the book seems to expect you to be able to get through quite a few activities in quite a short time. Graham agreed and pointed out that many of the activities seemed too long to fit into a one or two hour iteration retrospective. I thought that this was because the emphasis is on the longer activities in the middle of the session, while the beginning and end are quite quick.

Overall Miquel thought it was a well-structured book. Graham liked the way that it gave examples so that you didn’t spend too long just thinking about theory and principles. He also thought that most of the text could apply to situations outside of agile software development. Everyone seemed to agree on this. From my perspective, the main thing to come out of the book was the five stages that a retrospective is comprised of: Setting the Stage, Gathering Data, Generating Insights, Deciding what to do and finally Closing the Retrospective. When I saw Diana Larsen talking about this at JAOO she mentioned something which isn’t in the book. That is that you need to ensure that people do the “Gather Data” stage. Technical people tend to jump to the “Generate Insights” stage and business people tend to jump to “Decide what to do”.

I thought that something that was missing was some sort of diagram to show the connection between activities. Appendix C shows which ones you can use at which stage, but it doesn’t show which ones should follow other ones. It would be helpful to had something as simple as a list, giving a one line description and which other activities it is associated with.

Mick said that when he first heard about Retrospectives he was confused about how you could solve all of your problems in half an hour. It was only later that he realised that you don’t solve all of your problems, you work out what they are and how you can start moving in the right direction. The next retrospective allows you to see how you are doing and possibly change course.

Miquel pointed out that it is suggested that the retrospective facilitator role should rotate through a development group. Having read the book though, he thought that it seemed to be quite a skilled job, so maybe this would be difficult.

Graham asked about the minimum size of team where these techniques would be useful. I suggested that even on a team of only two people it would be useful having time out from the day-to-day work to think about things. David said that going to the pub with the team could also serve the same purpose. Graham was uncertain if this would just be forum for moaning in the absence of the formal process. I agreed and said that it is fine saying at the pub “We need more tests”, but it is much better to say “We will achieve 90% test coverage in these packages within the next month”. David suggested that Big Visible Charts showing progress would be a good way to keep these targets in people’s minds.

We then drifted off into more general topics. I liked David’s story about his boss showing up around noon declaring “We need to make sure people come in earlier!”.

Interesting note: if you look carefully at the cover of the book you’ll see that the compass has the letters N,S,W and O on it. So I think it is a German compass. Strange!

Meeting 18 – Peopleware

A quiet meeting this time, with just me, Miquel and newcomer Brett showing up to discuss Peopleware by De Marco and Lister (Looks like you didn’t choose the right one to come along to – usually we have more people and so the discussions flow a bit more).

Miquel started off the discussion by saying that he liked the book. He found it full of common sense ideas that companies generally ignore (or even do the opposite!). I liked the idea that most problems on software projects are sociological rather than technical.

Brett said that he had seen teams really coming together usually when there was a tight deadline to meet. But this didn’t usually last long. We then talked about overtime, which the authors are very much against. As a short term tactic to get a release ready it is fine, but once you start doing it for more than a few days then the downsides outweigh any advantages.

Miquel asked if the extra chapters in the 2nd edition had added much. There was a general consensus that the original book had covered most of the territory, so the extra chapters didn’t add a great deal.

Brett then said that although he had been an agile/XP enthusiast for some time, he hadn’t been able to apply these ideas because his work had always been quite isolating. Also, no-one he worked with was interested in trying any of these ideas out. I described how I use ideas such as the daily stand up meeting to help communication within the team.

Miquel pointed out that a literal reading of the book might suggest that stand up meetings are a waste of time and that everyone should be shut up in their own little office. This goes against a lot of agile thinking. But we agreed that if you look at what De Marco and Lister say closely they are against pointless status meetings and huge open plan spaces. Proponents of agile would tend to agree that stand up meetings should be short (that’s the reason you stand up) and that the best development environment is the war room. So there isn’t really much disagreement there.

I have read the book before, so there were less ‘a-ha!’ moments for me than the first time around. One thing I didn’t remember was the idea of a quota for errors, whereby a manager asks the members of his team what dead ends they have been down recently. The answer should not be ‘none’, because you would like people to experiment, which means sometimes failing.

Miquel liked the part at the beginning about the ‘High-Tech Illusion’ which makes managers think that they should spend their time thinking about technical issues rather than people issues. Not only are techie types better at solving those kind of problems, there is also this illusion that we ‘work in computers’. In fact, unless you are developing chips or writing operating systems you are just applying technology to solve real problems.

We moved on to talk about the idea of moving your development team away from the corporate building to actually get your project done. This sounded pretty radical and I was worried that it could hamper communication with other teams and with internal customers. I wonder if people really do this? I preferred the idea of getting your people to stay on extra days after a conference hotel to get some uninterrupted work time.

Finally we discussed the Black Team, which was an elite team of testers in IBM. The book holds them up as the best sort of jelled team, with their own in-jokes and even their own dress code (Black obviously). I was slightly concerned that their approach would alienate the people whose code they were testing. Lean teaches us that we should optimize the whole. If the feared Black Team don’t work well with their peers, will the whole development organisation work well?