Agile Testing Days 2012 — YAS (Yet Another Summary)

Feedback Learning Testing No Comments

Just like the years before I enjoyed the Agile Testing Days a lot. A fun theme throughout many of the talks were… unicorns and I think Gojko Adzic started it. This affected my brain so much, I said ‘Look, a unicorn!’ to my dog, when I took him for a walk for the first time after I came back from the conference. Actually, the animal crossing our path was a squirrel.

Here’s a short summary of my favourite parts. In order to stay in sync. with the printed & online conference program I’ll start counting the conference days at 0.

Day 0 – Tutorial “Software Testing Reloaded” with Matt Heusser & Pete Walen

Just as the full title “Software Testing Reloaded – So you wanna actually DO something? We’ve got just the workshop for you. Now with even less PowerPoint!” promised, there was only very little PowerPoint and a whole lot of testing & questioning.

I especially liked the way Pete & Matt presented the examples & exercises as well as the reasoning behind them.

Electronic Dice
Electronic Dice

There were a lot of games and actual testing. All was very hands on, well explained and debriefed at the end. To give just one example: We tested electronic dice, in order to give an estimation about how long testing would take and to come up with a recommendation whether these dice were ready to be used.

Questions like “Can we ship it now?”, “Is there a pattern/problem there?” and “What actually are the requirements?” were covered.

I also won a price, so my point of view might be biased. 😉

How to Reduce the Cost of Software Testing

Day 1

  • Get them in(volved) by Arie van Bennekum
    I found it super interesting to listen to one of the creators of the Agile Manifesto, especially since he pointed out that many of the principles and values have been around before the manifesto was written.
  • Myths About Agile Testing, De-Bunked by Janet Gregory & Lisa Crispin
    Lisa & Janet debunked myths as ‘Testing is dead’, ‘Testers must be able to code’ and ‘Agile = Faster’. Excellent story and fun presentation.
  • Consensus Talks – 7 10-minute-talks (including mine)
    The format of 10-minute talks, all back-to-back and no break included was the one I missed in the previous Agile Testing Days. This way a whole lot of ground is covered in a short time and it’s a great opportunity to explore new techniques in presenting without doing too much harm and/or presenting on conferences for the first time.
  • Self Coaching by Lasse Koskela
    Lasse explained how to coach yourself. After explaining how the human brain works he talked about how to step ‘out of the box’ (your personal point of view) in order to better understand what others actually say and to stay honest to yourself at the same time. Deep knowledge, very well explained indeed.
  • The MIATPP Award Night 2012
    Lisa Crispin won the MIATPP Award. Congratulations!

Day 2

  • Test Lab by James Lyndsay & Bart Knaack
    This year I went to the “Test Lab” and tested a small LEGO Mindstorm robot, that could move around on a coloured sheet of paper and react, depending which colour its camera would detect. The task: Find out what the specification of the robot is and find defects in its implementation.
    Very interesting: We had to come up with a hypothesis of how the robot was supposed to work as well as finding defects. I really enjoyed the way Bart & James gave feedback and asked the right questions.
  • “Reinventing software quality” – Gojko Adzic
    Breaking News: Unicorns Exist

    Gojko made the point that in agile testing we might (still) not focus on the right thing: To build the right software. Instead we concentrate on finding bugs and building the software in the right way. He illustrated this with one of his books:  He found a defect and then spent a while searching & listing more problems and ended up with a good number of them, definitely enough to make you worry about the quality of the book. However, the publisher explained that essentially all reviews were very positive! So: When people keep paying for your product or service, worrying about defects may not be that important.

Day 3

  • “Fast Feedback Teams” – Ola Ellnestam
    explained the importance and value of fast feedback. And he’s right: In many projects feedback could be gathered a lot earlier and be used to improve what features are built (as well as how they’re built). Other talks at least touched this topic as well. And while I wholeheartedly agree about this, I’m also a bit worried that we (as software creators) might forget about (or even ignore) slow changing aspects (for more about this see my previous post ‘Slow Feedback Cycles‘).
  • “Exceptions, Assumptions and Ambiguity: Finding the truth behind the Story” by David Evans
    UK Explained

    David explained how natural language is ambiguous, unclear and sometimes hard to understand. His examples included part of Jabberwocky (in several natural languages), music lyrics and last but not least programming. Also, I like his short introduction of the Cotswolds.

  • “It’s the Economy, Stupid! Learn the fundamentals about the one and only argument which will drag your management into agile practices” by Lucius Bobikiewicz
    Lucius talked about economic reasons of letting teams focus on 1 (one!) project/product, as opposed to working on multiple projects/products at the same time. It didn’t surprise me much, that letting teams finish one thing and then progress with the next is economically favourable. But  I was very surprised how much of a difference it made in the example he presented. These are the main advantages of the 1-project-only team Lucius presented:

    1. The time between the 1st project start and the first paid project (usually at the end of the project) is much smaller, meaning that significantly less money is needed to fund the upfront costs.
    2. Given the limited period of time in which a software can be sold, the product can likely be launched at the optimal time, whereas multi-product-teams may enter the market later and therefore always lag in selling.
    3. Since the overhead of context switching is minimised, the team can work on more projects/products per time unit. This advantage depends on the amount of time needed per context switch and the time the projects take .

    The economic point of view Lucius presented was a surprising and welcome detour form the other sessions which were much more focused about technology and/or ‘doing the right thing and doing it right’.

Every morning of the conference days 1-3 Lisa Crispin organised a Lean Coffee in one of the hotel bars. In small groups we discussed topics people were interested in. In order to cover some topics we limited the time for each topic to an initial 8 minutes and then added another 4 minutes depending on a quick vote. I find this is a very fun way to find and discuss topics. Thank you Lisa for organising them!

As a leaving thought: The chair persons were given (foldable) chairs as a present — ‘chairs for the chairs’. Super funny, if a tad bit unpractical to get home via train or airplane.

A big “Thank you!” to José, Uwe and Madelaine as well as the other organisers that made the Agile Testing Days such a great and enjoyable conference.

Slow Feedback Cycles

Feedback Learning 6 Comments

In his keynote ‘Fast Feedback Teams’ Ola Ellnestam (at the Agile Testing Days 2012) talked about the importance of fast feedback and how to shorten the cycle durations. To me, there seems to be a trend in focussing on getting feedback in shorter and shorter cycles.

While I completely agree that in many cases it takes too long to notice the effect of a change made earlier, I am also a little bit worried that we (as software developing teams and companies) might forget about  slow feedback cycles. Let me explain.

Feedback loops

Let’s look at aspects of feedback loops. There is an actor to change the system operation as well as a sensor that reads some information. In order to actually be a feedback loop, the sensor is driving the actor.

Note that most systems allow data to be read at any time and (essentially) any frequency. However systems usually also do not immediately respond to changed input. Here’s an example: You could read a thermometer in your room as often as you like. And you can also turn the control of the heater up or down pretty often and yet the room temperature will not change immediately.

Usually there is no point in acting a whole lot faster than the system can react. If it takes ½ an hour for the room temperature to adapt to a newly set value, reading the thermometer and changing the control every minute doesn’t make much sense.

Some waste and some risk

Now, with the focus on short time scales and a high frequency of the measurements (no matter wether they are qualitative or quantitative), I can imagine two undesirable outcomes:

  1. In a situation where the system reacts slowly to changing parameters, measuring often is waste. This is because the system couldn’t possibly yield another outcome (yet).
  2. If we only measure with a rather coarse-grained way (e.g. only the values red, yellow, green), we might miss a slow trend. (I guess this is the reason we often suddenly must buy winter tires, even though we clearly noticed that the temperatures decreased over the past weeks and we’re wearing our long sleeved shirts and anoraks too.)

That said, if the system reacts fast on a change and we can get the feedback quickly, it’s very likely a good idea to actually get that feedback. A good example for this case: Having fast unit tests in programming. 🙂

A Surprising Use of an Interface

Testing 2 Comments

This post relates to my short talk at the Agile Testing Days in Potsdam, Germany 2012.

Earlier this year I tweeted this (there’s another blog post about this as well):

Automating tests/checks, seems more valuable than having automated tests—like planning being more valuable than having the plan.

In my case the automating certainly led to other interesting results — and that’s the topic of this post.

Some context

Let’s assume we’re using Cucumber to automate tests for some application and follow the typical Cucumber setup: A folder features containing the feature files and inside of that another folder (step_definitions) containing, well, the step definitions. A support folder inside the feature directory keeps supporting files, such as environment configuration(s) etc.

Keeping it clean

The way I started to work is to start simple: At first I put everything inside the step definitions. Pretty soon though, I extract code chunks into methods of their own. And then move those method definitions into their own class (or module) – an in fact a file of their own.

That way, I end up with two more levels of abstraction below the feature descriptions and the step definitions: Classes (or modules) containing method definitions (see picture).

feature files, step defs, classes/modules, methods

Here I follow Uncle Bob Martin‘s advice of “Extract till you drop” (see his book ‘Clean Code‘, and episode 3 (Functions) of the video series at cleancoders.com): Extract methods until there’s nothing left to extract (and extract that code into the right place).

A first result: A high level interface to the system

One of the results of doing this, is a really high level and clean API to the system at hand. In addition to that I find that step definitions are much easier to understand, since they (mostly) contain a business level description of how the system is used and accompanying assertions to check whether the system behaves as it should.

Enter Pry

Pry is a command line tool for Ruby, like the interactive ruby shell irb that comes with Ruby. However, Pry is a lot more powerful and offers to display method definitions (even those defined in the C sources of internal Ruby classes) and a whole lot more. For a good introduction see the Pry Screencast by Josh Cheek on vimeo.

A trivial and contrived example

Let’s assume we’re trying to find stuff on the web (I’m on a Mac; there are hints about how to achieve this on Windows in the code comments):

require 'safariwatir' # Windows: require 'watir'

class GoogleSearch
 def initialize
 @browser = Watir::Safari.new # Win: Watir::Browser.new
 end

 def search_for search_term
 @browser.goto 'http://wellknown-search-engine.example.com/'
 @browser.text_field( :name, 'q' ).set search_term
 @browser.button( :id, 'gbqfb' ).click
 @browser.ol( :id, 'rso' ).links.map(&:text)
 end
end

Given that interface and Ppy, the creation of a console app borders on being trivial (yet, it took me a long while to get to this point):

require_relative 'google_search'
require 'pry'

GoogleSearch.new.pry

Think about this for a moment: Take Pry and your API and in 3 lines of code you get a console app for your system. The console can be used like this:

[10] stephan@nibur … #ruby google_search_console.rb
[1] pry(#<GoogleSearch>):1> search_for 'pry video josh'
=> ['Pry Screencast on Vimeo',
# ...

As mentioned above, you can even display the method definition (including lines of code, file name etc.)

[8] pry(<GoogleSearch>)> show-method search_for

From: /Users/stephan/…/google_search.rb @ line 8:
Number of lines: 6
Owner: GoogleSearch
Visibility: public

def search_for search_term
 @browser.goto 'http://google.de/'
 @browser.text_field( :name, 'q' ).set search_term
 @browser.button( :id, 'gbqfb' ).click
 @browser.ol( :id, 'rso' ).links.map(&:text)
end
[9] pry(#<GoogleSearch>)>…

Leaving thoughts

It seems that doing something can be more valuable that having the result. Notwithstanding, we still need to produce something of value for our business.

I came to realise that clean code matters more than ‘only’ providing a code base that’s easy to understand and change: Without the high level API, I might not even have noticed the possibility to create a ‘business level console’.

Having a console app like this allows to prepare starting points for exploratory testing (that might be ‘automation assisted exploring). Or short: I think a clean and high-level interface plus Pry is a cool tool for exploration.

© 2012-2024 Stephan Kämper — Mastodon

MENU

Navigation