Category: Testing

Sources of Learning

Over at Jeroen Mengerink’s blog (and there’s the 1st of the sources to learn from) he talks about learning (and the lack thereof) of people in testing. Given the topic list of this blog “On Testing, Patterns and Learning”, this topic is very interesting to me, too.

I recently attended a Scrum Training and one of the group exercises was to suggest ways to implement certain practices we were given. One of the topics was “focus on both, technical excellence and good design, fosters agility” (my translation, the training was held in German). I started filling the spot for possible immediate actions on the board: Conferences, user group meetings, books, blogs, podcasts… at which point a colleague stopped me, so others could add to it. However they didn’t.

So, if you’re looking for information about testing here are some recommendations I have, for today let’s cover the letter ‘B’ 😉

Jeroen also mentioned conferences and especially a presentation by Alan Richardson (@EvilTester on Twitter): Please go to Jeroen’s post and read it, then watch Alan’s presentation, both are really recommended stuff.

For now, I’d love to hear what book(s) and/or blog(s) you recommend that helped you learning about testing or improving your testing.

TDD and Manually Solving Tasks

Uncle Bob Martin recently wrote “Why is estimating so hard?”. Among other things, the article explains the difference between manually doing a task (in this case breaking a text into lines of a certain maximum length) and actually writing the program to do it.

The way (many) humans do this is by trial & error, as Uncle Bob says:

Why was it so hard to write down the procedure for doing something so basic and intuitive?

Answer: Because when we do it manually, we don’t follow a procedure. What we do instead it continuously evaluate the output and adjust it until it’s right.

In an earlier article (on of before 7. Oct. 2005) “The Three Laws of TDD” Uncle Bob described three rules (or laws) of TDD:

Over the years I have come to describe Test Driven Development in terms of three simple rules. They are:

  1. You are not allowed to write any production code unless it is to make a failing unit test pass.
  2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
  3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

The similarity between these three rules and the non-procedural way used for manual tasks surprises me: Is it possible that the TDD-way of writing software works so well, because it models the way we approach manually solving problems? Thinking of it, my first reaction is this: Sure, writing code (whether test-driven or not) is the manual work of solving some problem. So far, there’s not much news here.

But then, there seems to be more to it… To me, it is fascinating to think about the ‘meta level’ of what we’re looking at: Code writing is the manual, sapient way of creating a procedure to solve some problem. It is neither automated nor do we have a process or procedure for this kind of work.

To me TDD is an aid, a technique to help me write code in a more methodical and disciplined way. It is not at all a step to make software development more mechanical or predictable.

This brings me back to the starting point of this post: Estimating. It’s hard to predict a complex system like software development, where humans do creative work on hard problems, trying to find the solutions to problems they have not solved before. This is hard (and to me fun) work.

Should it ever turn out that there’s an easy way to (reasonably) correctly estimate it, I will be very surprised.

Presentation at The Bonn Agile Meetup About Test Automation

Monday night I presented about “Test Automation” at the Bonn Agile Meetup, a group of now more than 100 people interested in all kinds of topics related to, well, ‘Agile’.

First of all, thank you all for coming! With a bit more than 30 people, I heard that this was the largest number of people joining the Meetup. Second of all a big thanks to the folks at Doo for providing a great location, support, beverages & food. I really enjoyed the hour speaking & coding as well as the great 3 hours of discussion & talking afterwards.

Without further ado, these are the (German only) slides: test-automatisierung_bonn-agile_april-2012.

As the main technical part, I presented my preferences about organising automated test (or checks, if you prefer that term) in Cucumber:

  1. I like to have the scenarios to be as high level and close to the ‘business language’ as possible.
    If a user with certain rights or a given role needs to login, I’d rather note that role/right, not an explicitly given username & password in the scenario file.
  2. I also prefer the step definitions to be as clean as possible. Sure, we likely need to refer to a concrete user name here, and be clear about what steps (on the application level) are done in which order. Assertions (of whatever kind you prefer) is also what I like to see in the step definitions: What is actually done and what the expectations are — that’s what I prefer here.
    Code I don’t like in step definitions: Statements that create a browser (headless or otherwise), HTML or XML parsing code, SQL statements of whatever kind…
    Whenever that appears in the step definitions, I lose track of what I’m about to check.
  3. As the details of dealing with the output of the system under test have to go somewhere, I usually create classes and/or modules wrapping all the details. This code actually implements the interface used to communicate with the tested system.

I didn’t come up with all this, but find it a good way for me to separate technical from business levels. Most of the ideas in fact come from @unclebobmartins Clean Coders videos (and his books about the same topic).

Doing Something And The Result of It

While I was recently working on some test automation task, I had the feeling that the automating of the tests seemed to be more important, than the actual automated tests (or checks) I created. It seemed to me that this is very similar to the saying that the planning in agile projects (and likely in other projects as well) is more important (or valuable) than the plan you get out of the planning. So I tweeted about it and within seconds Lisa Crispin agreed to this.

It seems to me there is a pattern—actually a question—at work: Is doing something more important than the result you get out of that activity? There’s a book by Mary and Tom Poppendieck ‘Leading Lean Software Development’, subtitled ‘Results Are Not the Point’, that hints in the same direction.

Is actually doing something really more valuable or important than the result of this activity? I’m not sure about this.

The result of having done something seems obvious: We have the result, something that didn’t exist before and has some value to someone (hopefully). But what about the value of the activity itself? Two things come to my mind immediately:

  • While actively working on a task, no matter whether it’s performing or rehearsing, we exercise and  get more used to doing it. We’re very likely getting better at doing this in the future.
  • It’s also likely that we learn something about the task itself, some technology we’re using or social aspects of work life.
  • A secondary result are scripts that help me to create a certain state in the system, a base camp, from where I can start explorations into new areas of the system.

What do you think about this? What makes the doing itself valuable to you, apart from the outcome?

Let’s go back to the creating of automated tests: While I was looking at newly implemented parts of a software system, as soon as an automated test executes and yields a reproducible result… it’s a regression test (or check as some would say). Now, regression tests are not particularly likely to disclose new defects, so what’s the value of automating anyway. While I can’t offer a generally valid answer, the automation effort itself helped me to uncover a number of bugs.

PS: An additional example if Ben Simo (@qualityfrog) tweeted this:

Read in 100+ yo teach book: teacher should script lesson, then throw away script at class start. Scripting useful for prep, not instruction.

Another great example of the pattern.

Navigation

%d bloggers like this: