Author: Stephan

About Stephan

I'm a self-employed software tester. I test and write software, listen to music and go hiking — sometimes simultaneously. Prior to this was a physicist and oceanographer, among other things.

Slow Feedback Cycles

In his keynote ‘Fast Feedback Teams’ Ola Ellnestam (at the Agile Testing Days 2012) talked about the importance of fast feedback and how to shorten the cycle durations. To me, there seems to be a trend in focussing on getting feedback in shorter and shorter cycles.

While I completely agree that in many cases it takes too long to notice the effect of a change made earlier, I am also a little bit worried that we (as software developing teams and companies) might forget about  slow feedback cycles. Let me explain.

Feedback loops

Let’s look at aspects of feedback loops. There is an actor to change the system operation as well as a sensor that reads some information. In order to actually be a feedback loop, the sensor is driving the actor.

Note that most systems allow data to be read at any time and (essentially) any frequency. However systems usually also do not immediately respond to changed input. Here’s an example: You could read a thermometer in your room as often as you like. And you can also turn the control of the heater up or down pretty often and yet the room temperature will not change immediately.

Usually there is no point in acting a whole lot faster than the system can react. If it takes ½ an hour for the room temperature to adapt to a newly set value, reading the thermometer and changing the control every minute doesn’t make much sense.

Some waste and some risk

Now, with the focus on short time scales and a high frequency of the measurements (no matter wether they are qualitative or quantitative), I can imagine two undesirable outcomes:

  1. In a situation where the system reacts slowly to changing parameters, measuring often is waste. This is because the system couldn’t possibly yield another outcome (yet).
  2. If we only measure with a rather coarse-grained way (e.g. only the values red, yellow, green), we might miss a slow trend. (I guess this is the reason we often suddenly must buy winter tires, even though we clearly noticed that the temperatures decreased over the past weeks and we’re wearing our long sleeved shirts and anoraks too.)

That said, if the system reacts fast on a change and we can get the feedback quickly, it’s very likely a good idea to actually get that feedback. A good example for this case: Having fast unit tests in programming. 🙂

A Surprising Use of an Interface

This post relates to my short talk at the Agile Testing Days in Potsdam, Germany 2012.

Earlier this year I tweeted this (there’s another blog post about this as well):

Automating tests/checks, seems more valuable than having automated tests—like planning being more valuable than having the plan.

In my case the automating certainly led to other interesting results — and that’s the topic of this post.

Some context

Let’s assume we’re using Cucumber to automate tests for some application and follow the typical Cucumber setup: A folder features containing the feature files and inside of that another folder (step_definitions) containing, well, the step definitions. A support folder inside the feature directory keeps supporting files, such as environment configuration(s) etc.

Keeping it clean

The way I started to work is to start simple: At first I put everything inside the step definitions. Pretty soon though, I extract code chunks into methods of their own. And then move those method definitions into their own class (or module) – an in fact a file of their own.

That way, I end up with two more levels of abstraction below the feature descriptions and the step definitions: Classes (or modules) containing method definitions (see picture).

feature files, step defs, classes/modules, methods

Here I follow Uncle Bob Martin‘s advice of “Extract till you drop” (see his book ‘Clean Code‘, and episode 3 (Functions) of the video series at cleancoders.com): Extract methods until there’s nothing left to extract (and extract that code into the right place).

A first result: A high level interface to the system

One of the results of doing this, is a really high level and clean API to the system at hand. In addition to that I find that step definitions are much easier to understand, since they (mostly) contain a business level description of how the system is used and accompanying assertions to check whether the system behaves as it should.

Enter Pry

Pry is a command line tool for Ruby, like the interactive ruby shell irb that comes with Ruby. However, Pry is a lot more powerful and offers to display method definitions (even those defined in the C sources of internal Ruby classes) and a whole lot more. For a good introduction see the Pry Screencast by Josh Cheek on vimeo.

A trivial and contrived example

Let’s assume we’re trying to find stuff on the web (I’m on a Mac; there are hints about how to achieve this on Windows in the code comments):

require 'safariwatir' # Windows: require 'watir'

class GoogleSearch
 def initialize
 @browser = Watir::Safari.new # Win: Watir::Browser.new
 end

 def search_for search_term
 @browser.goto 'http://wellknown-search-engine.example.com/'
 @browser.text_field( :name, 'q' ).set search_term
 @browser.button( :id, 'gbqfb' ).click
 @browser.ol( :id, 'rso' ).links.map(&:text)
 end
end

Given that interface and Ppy, the creation of a console app borders on being trivial (yet, it took me a long while to get to this point):

require_relative 'google_search'
require 'pry'

GoogleSearch.new.pry

Think about this for a moment: Take Pry and your API and in 3 lines of code you get a console app for your system. The console can be used like this:

[10] stephan@nibur … #ruby google_search_console.rb
[1] pry(#<GoogleSearch>):1> search_for 'pry video josh'
=> ['Pry Screencast on Vimeo',
# ...

As mentioned above, you can even display the method definition (including lines of code, file name etc.)

[8] pry(<GoogleSearch>)> show-method search_for

From: /Users/stephan/…/google_search.rb @ line 8:
Number of lines: 6
Owner: GoogleSearch
Visibility: public

def search_for search_term
 @browser.goto 'http://google.de/'
 @browser.text_field( :name, 'q' ).set search_term
 @browser.button( :id, 'gbqfb' ).click
 @browser.ol( :id, 'rso' ).links.map(&:text)
end
[9] pry(#<GoogleSearch>)>…

Leaving thoughts

It seems that doing something can be more valuable that having the result. Notwithstanding, we still need to produce something of value for our business.

I came to realise that clean code matters more than ‘only’ providing a code base that’s easy to understand and change: Without the high level API, I might not even have noticed the possibility to create a ‘business level console’.

Having a console app like this allows to prepare starting points for exploratory testing (that might be ‘automation assisted exploring). Or short: I think a clean and high-level interface plus Pry is a cool tool for exploration.

Attending a Code Retreat

The “Softwerkskammer” regional group of Düsseldorf, Germany organised its first Code Retreat this weekend (1. Sept. 2012), which I attended. The facilitator, Patrick Cornelißen (of orchit), has posted a summary of the Code Retreat, so I won’t repeat him, except for stating that the task was “Conway’s Game of Life” and we wrote code in 6 time slots (45 minutes each) each one followed by a 15 minute retrospective. Here are my observations and notes:

About the programming and pairing

First of all I found it super exciting to get pairing with programmers. Although not every one else was a programmer, I was the only tester attending and I found pairing up with programmers very exciting (Would my coding skills be sufficient?), productive and fascinating.

One pairing partner made me write the tests in much smaller steps, than I would have ever done. After struggling with the approach of implementing the desired functionality in the test (without defining any method or class), I finally realised just how small steps are possible. Those were really tiny steps. It occurred to me rather late in the slot, that what we created was a rather functional way to solve the problem. In a later slot I asked Sergey Shishkin to pair program using Clojure.

One (kind of) surprising constraint we were offered, was to avoid if-then-else and case constructs by using polymorphic dispatch. Apparently this approach was a challenge for nearly every one, see Partick’s tweet about it:

Verwunderte Gesichter bei der Vorstellung des “No-If” Constraints 🙂 #coderetreat

General remarks, tips and an observation

  1. When pair programming using a notebook, especially a relatively small notebook, I recommend using an external keyboard. If you’re limited to the internal keyboard either the ‘driver’ can’t easily type on the keyboard or the ‘co-pilot’ can’t easily watch what’s happening on the screen.
  2. If you want to learn more about your favourite programming language, editor and/or IDE, attend a Code Retreat.
  3. Pairing for the first time and/or pairing with people you didn’t meet before will give you new ways of thinking and programming.
  4. I was the only tester and the only one who used fork and knife at lunch time when eating pizza. 🙂 However, I used my hands like everyone else in my second serving. 😉

I very much enjoyed the day. Thank you to everyone involved!

A Week of “Rapid Testing Intensive”

I signed up for the Rapid Testing Intensive event starting next week and am looking forward to a week of learning new things about software testing.

I’ll attend online and if anyone would like to talk about it or team up with me, leave me a note in the comments below or contact me via @S_2K on Twitter. In case you like to follow the tweets about the event, the Twitter hash tag is #RTI1.

Edit: Since that hash tag is already in use, make that #RTI2012.

An Anti Pattern when using Continuous Integration & Automated Tests

The Setting

Once in a while I meet teams who are using a Continuous Integration (CI) system, for their builds and executing the unit tests (yes, there are still teams using a CI for building the software, but not executing automated tests [or checks]). So this is the setting:

  • There is a CI system.
  • There may be unit tests being automatically executed either after check ins into the version control system or at least nightly.
  • In some cases there are failing tests (or checks).
  • Broken builds are displayed on the start page of the CI system in the (intra) net, but not ‘radiated’ so everyone can and will see them.
In this post I focus especially on the cases where the last two points are true.

The Observation

I think this is problematic at least, likely also dangerous, because when there are failing tests and/or broken builds  for some time the team will very likely develop features base on broken software. Getting back to a clean and working system gets harder and harder as time goes on.

The Aftermath

A change in the behavior of the automated check can go unnoticed: Since no one watches the first test fail, it’s unlikely that someone will realize that other checks start failing, too. Furthermore new features might depend on the brokenness somewhere else.

Now What?

Do not live with ‘broken windows‘ & apply a ‘Stop the Line‘ (or check out what Google says about Stop the Line) policy: As soon (read: right after it’s discovered) as a problem becomes known, solve it and then continue with other work. Like everything else, this is supposed to be done by the whole team. When everyone helps, things get solved quickly.

A leaving thought

In my opinion, introducing a Continuous Integration System and then ignoring the results is certainly a major anti pattern, that can do more harm than good for the development process. The effort of setting it up and keep running and then ignoring the information it provides is a form of waste. Is it possible that in such a case you’re better off not having a CI system? Honestly, I’m not sure.

Note however, that I am in strong favour of…

  • having a CI system,
  • actually using it to build the systems and execute automated checks against it,
  • radiating the results to the team and
  • acting upon those results.

Navigation

%d bloggers like this: