A Week of “Rapid Testing Intensive”

I signed up for the Rapid Testing Intensive event starting next week and am looking forward to a week of learning new things about software testing.

I’ll attend online and if anyone would like to talk about it or team up with me, leave me a note in the comments below or contact me via @S_2K on Twitter. In case you like to follow the tweets about the event, the Twitter hash tag is #RTI1.

Edit: Since that hash tag is already in use, make that #RTI2012.

An Anti Pattern when using Continuous Integration & Automated Tests

The Setting

Once in a while I meet teams who are using a Continuous Integration (CI) system, for their builds and executing the unit tests (yes, there are still teams using a CI for building the software, but not executing automated tests [or checks]). So this is the setting:

  • There is a CI system.
  • There may be unit tests being automatically executed either after check ins into the version control system or at least nightly.
  • In some cases there are failing tests (or checks).
  • Broken builds are displayed on the start page of the CI system in the (intra) net, but not ‘radiated’ so everyone can and will see them.
In this post I focus especially on the cases where the last two points are true.

The Observation

I think this is problematic at least, likely also dangerous, because when there are failing tests and/or broken builds  for some time the team will very likely develop features base on broken software. Getting back to a clean and working system gets harder and harder as time goes on.

The Aftermath

A change in the behavior of the automated check can go unnoticed: Since no one watches the first test fail, it’s unlikely that someone will realize that other checks start failing, too. Furthermore new features might depend on the brokenness somewhere else.

Now What?

Do not live with ‘broken windows‘ & apply a ‘Stop the Line‘ (or check out what Google says about Stop the Line) policy: As soon (read: right after it’s discovered) as a problem becomes known, solve it and then continue with other work. Like everything else, this is supposed to be done by the whole team. When everyone helps, things get solved quickly.

A leaving thought

In my opinion, introducing a Continuous Integration System and then ignoring the results is certainly a major anti pattern, that can do more harm than good for the development process. The effort of setting it up and keep running and then ignoring the information it provides is a form of waste. Is it possible that in such a case you’re better off not having a CI system? Honestly, I’m not sure.

Note however, that I am in strong favour of…

  • having a CI system,
  • actually using it to build the systems and execute automated checks against it,
  • radiating the results to the team and
  • acting upon those results.
%d bloggers like this: