Monday, 4 February 2008

Book Review: Testing Computer Software by Kaner, Falk, Nguyen

imageI thought I'd read this again for review purposes. I didn't expect it to surprise me, but it did, massively.
One of the most realistic testing books available, starting almost immediately in the preface discussion "Its not done by the book". The book sets out its target audience as simply "the person doing the testing"
[amazon.com][amazon.co.uk]


"...find and flag problems in a product, in the service of improving its quality. your reports of unreliability in the human-computer system are appropriate and important...You are one of few who will examine the full product in detail before it is shipped."
This 'definition' works better for me than Ron Patton's definition, but Ron's book reads more gently and easily. 'Testing Computer Software' contains a lot of very direct opinions from the authors which you will see presented as authoritative is'ms (this is X) which may distance the reader if the reader currently adopts a very different mindset - which I think happened to me on first reading. So if it happens to you, don't switch off, don't skim. Analyse your response. Read this book in a better way than I did.
Ron's book only targets beginners whereas Testing Computer Software works for both beginners and more experienced testers - if the 'more experienced' mind doesn't rebel too quickly.
I don't think I read this book properly the last time I read it. Certainly I wasn't doing explicit exploratory testing at the time and I think I dismissed the text as a little too ad-hoc. But just a few pages in I can now see that the book outlines some lessons that I then had to learn through experience e.g. "Always write down what you do and what happens when you run exploratory tests." Sigh, if only I had read the book properly first time round. 
Chapter 1 starts with an overview of 'exploratory' testing and a possible strategy that an experienced tester might adopt. A 'show' don't 'tell' approach to explaining software testing.
1st cycle of testing
  1. Start with an obvious and simple test
  2. Make some notes about what else needs testing
  3. Check the valid cases and see what happens
  4. Do some testing "on the fly"
  5. Summarize what you know about the program and its problems
2nd cycle of testing
  1. Review responses to problem reports and see what needs doing and what doesn't
  2. Review comments on problems that won't be fixed. They may suggest further tests.
  3. Use your notes from last time, add your new notes to them, and start testing
I found some notes that I made when I read the book first time through lodged inside the cover. At the time I first read the book I took umbrage at the notion that "the best tester is the one who gets the most bugs fixed." I now read that as "the best tester finds the bugs that matter most". But I still find myself reticent about using the phrase "the best tester is" as that suggests an 'always' to me and I really can't say that that statement would 'always' apply.
Chapter 2 sets out the various ground rule axioms so the reader doesn't have to learn them the hard way e.g. "you can't test a program completely" "you can't test it works correctly" etc.
Chapter 3 seems like a general reference section on test types but even here we find good old fashioned, hard won experience, box outs which challenge your thinking.
Chapter 5 - reporting and analysing bugs works well on repeated reading and everyone involved in testing would benefit from re-reading it occasionally.
Problem tracking (chapter 6) pulls no punches in its description of the 'real' world that I have encountered and you may well encounter on some projects;
  • "Don't expect any programmer or project manager to report any bugs"
  • "Plan to spend days arguing whether reports point to true bugs or just to design errors."
  • ...
Fortunately the chapter contains a lot of advice as well:
  • Each problem report results from an exercise in judgement where you reached the conclusion that a "change is worth considering"
  • Hints on dealing with 'similar' or 'duplicate' reports (and how to tell them apart)
The Test Design chapter (7) speeds through a whole set of useful 'stuff' and again has plenty of experience behind it to learn from.
Most people will not test printers so chapter 8 presents the opportunity for the reader to deconstruct it and learn some generalised 'lessons' otherwise the obvious temptation results in skipping it and learning nothing.
Skipping across to Chapter 12 I see that I learned the very important lesson that the test plan can act as a tool as well as a product from this book, and that for me was worth the initial time with the book as it clarified a lot of thoughts in my head and helped me approach the particular project I worked on at the time in a different way; incrementally building up my thoughts on the testing, making my concerns and knowledge gaps visible.
I did not find this an easy book to read. Even on a second reading.  I frequently felt mentally bludgeoned by authoritative sentence phrasing. Which for a book that embraces exploratory testing and contextual thinking I find a strange dichotomy.
But don't let this stop you reading the book. This book deserves its best-selling status, and still deserves to sell in vast quantities. The writers have crammed so much practical advice in here that I heartily recommend it.
I can see that my thinking has changed since I last read the book. Which sadly suggests to me that the book wasn't a 'persuasive' argument for this 'experienced' tester at the time when I really needed it to help me. So please, gentle reader, if you consider yourself an 'experienced' tester try and read it with a clear mind.
If you consider yourself a beginner then you will probably get a lot out of the book immediately - Chapter One alone pays for the price of admission.
[amazon.com][amazon.co.uk]

No comments:

Post a Comment