I signed up for the Rapid Testing Intensive event starting next week and am looking forward to a week of learning new things about software testing.
I’ll attend online and if anyone would like to talk about it or team up with me, leave me a note in the comments below or contact me via @S_2K on Twitter. In case you like to follow the tweets about the event, the Twitter hash tag is
Edit: Since that hash tag is already in use, make that #RTI2012.
Posted by Stephan on Tue, July 17, 2012
Once in a while I meet teams who are using a Continuous Integration (CI) system, for their builds and executing the unit tests (yes, there are still teams using a CI for building the software, but not executing automated tests [or checks]). So this is the setting:
- There is a CI system.
- There may be unit tests being automatically executed either after check ins into the version control system or at least nightly.
- In some cases there are failing tests (or checks).
- Broken builds are displayed on the start page of the CI system in the (intra) net, but not ‘radiated’ so everyone can and will see them.
In this post I focus especially on the cases where the last two points are true.
I think this is problematic at least, likely also dangerous, because when there are failing tests and/or broken builds for some time the team will very likely develop features base on broken software. Getting back to a clean and working system gets harder and harder as time goes on.
A change in the behavior of the automated check can go unnoticed: Since no one watches the first test fail, it’s unlikely that someone will realize that other checks start failing, too. Furthermore new features might depend on the brokenness somewhere else.
Do not live with ‘broken windows‘ & apply a ‘Stop the Line‘ (or check out what Google says about Stop the Line) policy: As soon (read: right after it’s discovered) as a problem becomes known, solve it and then continue with other work. Like everything else, this is supposed to be done by the whole team. When everyone helps, things get solved quickly.
A leaving thought
In my opinion, introducing a Continuous Integration System and then ignoring the results is certainly a major anti pattern, that can do more harm than good for the development process. The effort of setting it up and keep running and then ignoring the information it provides is a form of waste. Is it possible that in such a case you’re better off not having a CI system? Honestly, I’m not sure.
Note however, that I am in strong favour of…
- having a CI system,
- actually using it to build the systems and execute automated checks against it,
- radiating the results to the team and
- acting upon those results.
Posted by Stephan on Sat, July 14, 2012