Write Meaningful Error Messages in Automated Tests

Here’s a tip: Write your automated tests with the failure in mind. Especially, consider a future maintainer who may need a useful error message. 

This can help when the test fails in the future (and it probably will). A descriptive message helps understand the technical issue you’re looking at and will ideally guide you to finding a solution.

Let’s look at some examples that leave something to be desired. These messages may be true, but don’t help to understand the underlying problem:

  • Expected true, but got false
  • The result message is malformed
  • fail

Yes, I have seen these or very similar messages, that are rather useless. 

Imagine how much more helpful, the following messages are:

  • Expected condition XY to be true in context AB of object O
  • The message ‘<output the actual message>’, is malformed and cannot be processed further
  • Got <actual result> instead of <expected_result> when processing XY

These improved messages can guide you, help you remember the context and figure out the underlying issue when the test fails.

I find that this improvement shortens the time spent with failure analysis. It makes my days more productive because I get a message that tells me about the context where thing went wrong. 

Do you have similar ideas about how to improve (automated) tests?
I’d love to hear about them.

Why I think We Can Do better Than asking ‘Why’ 

The other day I read this (German toot) on Mastodon (This toot apparently has ‘timed out’ by now. My answer is still available on Mastodon, though Stephan: “@Minnasophie@troet.cafe @wandklex@mastodon.art Da…” – Software development is a social activity):

Die Frage ‘Warum’ blockiert oftmals das Annehmen einer Situation.


My translation to English: ‘The question ‘Why’ often blocks accepting a situation.’

I immediately reacted to this.

I thought back to the day a doctor diagnosed me with cancer. He saw a question I was about to ask and before I started to speak, he told me to not  ask ‘Why’. He advised me to  instead concentrate on how I wanted to deal with the situation.

That really changed my thinking. To help me with this, he asked whether I had any plans for the near to mid-term future, something that I wanted to achieve. This changed my thinking from intense worrying extremely to something I I could focus on.

I still worried about the treatment! But I also thought about any plans that I had. And indeed, there was one big plan that I did have: To give a keynote at Agile Testing Days 2018! I WANTED to do it. Badly.

I later realised, another good news that was hidden in the question about my plans for the future: Apparently the doctor thought I had a future, at least the chance to have it.

As it turned out, I was not able to give the planned keynote, since the therapy massively compromised my immune system and that meant I had to avoid larger crowds of people. During the traditional flu season, I was strongly advised to not meet more than 3-4 people at a time. And to avoid even that whenever possible.

Eventually, in 2019, I could give the keynote, and it was received well. Very well, indeed.

Now, having read that quote from above, I started to think about asking ‘Why’ – or even asking ‘Why’ five (!) times to dig into the cause of whatever one is investigating. There may be better questions or approaches. Is asking ‘How do we want to deal with this?’ a better way?

I can imagine that especially when trying to understand a situation that went bad, avoiding ‘Why’ may be a good idea. At least in German ‘Warum’ (of the similar terms ‘Weshalb’ or ‘Wieso’) already carries with it a tiny bit of shaming or (suspecting) guilt. This is  something that one very likely wants to avoid, if the plan is to actually get to the root causes of a situation.

Given the learnings from my illness, I think the following question is an improvement on asking ‘Why?’:

How do we want to deal with this situation?

In my experience, asking ‘Why?’ often also involves ‘you’ as in ‘Why did you do X?’. This adds to the potential of shaming or assuming guilt: Now, there is also some separation between the one asking and the person being asked. 

I had good experiences using ‘we’, when figuring out problems. Using ‘we’ signals that, in fact, we have a problem. 

Using ‘We’ puts us on the same side of a task: we are facing the problem together. Therefore neither of us ‘is’ the problem (or maybe we all are), but… the problem is the problem. 

Having made clear that this problem is our problem, we can collaborate to find a good solution. Even better, we might find three (or more) solutions and then select the best one for the given context, at that moment in time, and implement it.

This is related to Jerry Weinberg’s observation about understanding problems:

If you can’t think of at least three things that might be wrong with your understanding of the problem, you don’t understand the problem.”

Gerald M. Weinberg. Are Your Lights On?. Part 3 – What Is The Problem, Really?. https://leanpub.com/areyourlightson

Agile Testing Days 2022

It’s that season again: I attended the 2022 edition of the Agile Testing Days. There are already blog posts by Lisi Hocke ‘Agile Testing Days 2022 – The Unicorn Land We Build Together‘ and Stéphanie DesbyWhat I’ve learned at Agile Testing Days 2022‘. They already cover a lot of the remarkable sessions – and really, I think there’s no replacement to attend this conference in person. I’ll focus on the few things I find particularly noteworthy.

I arrived the day before the conference started and went to the gym & pool area first. After a 5½ hour drive through changing weather conditions (snow, fog, hail, sunshine, rain, more snow, sleet, and finally some more sunshine), this is just the right thing for me to shake off the stress.

Tutorial Day ‘Problem Solving with Agile Thinking and Practices‘ with Ben Linders

I very much enjoyed this well organised tutorial. Ben presented several way to find problems and then solve them. This included some role playing and games. What I appreciate: We got all the material needed to use the games in our projects as well as his ebook ‘Impediments – Problem? What Problem?’. That’s going to be so useful when applying the exercises in my projects – or playing the game with my teams (when I will meet teams in person).

The day was made even better, since I received the confirmation that I am booked for my next project. This is especially nice, since I have worked with the team already and like it a lot!

Day 1

I found Lily Higham’s talk ‘Testing the BBC World Service‘ exciting, since she explained how her team has to cover an incredibly large number of systems, languages and devices. One important insight: ‘Test with real devices’. Also remarkable: She noted how the BBC has to deal with being blocked in some countries and how, in some cases, professional smugglers help spreading the news anyway. No details about this were shared for a good reason. We were advised against trying the smuggling ourselves.

Dermot Canniffe presented ‘BDD And The Sleepy Developer‘, and made the point about the effects a sleep disorder can have during work incredibly well.

During the OpenSpace on this day we shared some information about Mastodon – and what to consider when joining it. My point of view: Two things are important:

  1. The server you chose
  2. The account name

While I didn’t use them in the Open Space, here are slides I prepared just in case:

For easy reference, here are the links used in the slides:

As is a tradition by now the first day ended with a themed costume party. This year the ‘dress code’ was ‘Fairytales’. This is also the Award Night to celebrate MIATPP (Most Influential Agile Testing Professional Person) of the year, this time won by Janet Gregory. Congratulations!

Day 2

On this day, I missed quite a few sessions I wanted to attend, since I held a workshop “Fast Feedback Using Ruby”. With about 10 people attending it was just the right size to help with the many things that can (and will) go wrong in a workshop where coding is a major part. Being (maybe over-) prepared helped a lot: While I had some material prepared to be downloaded, I knew (from experience in an earlier year) that conference WiFi may – or may not – work well. I now bring my own WiFi, so I have a back-up in case the conference one doesn’t work so well. I also had the downloadable material on a tiny small local web server – and a USB stick. Yes, I was over prepared, very likely.

In case you considered this workshop, but attended another session: There is an ebook(let) available on LeanPub, which covers the workshop. And there still are a few coupons let to grab it for free: https://leanpub.com/fastfeedbackusingruby/c/ATD-2022. Additionally there is a GitHub repository at https://github.com/s2k/fastfeedbackusingruby_workshop. The slides from the workshop are available as Fast Feedback Using Ruby slides.

The day ended with the first ‘AgileTD book fair‘ and the ‘Digesting Poets Society‘. I had the pleasure to present ‘Software People … Work From Home — Insights & Experiences From Planet Earth‘, a free ebook to which contributors from (so far) 28 countries and 33 authors provided text about life during the pandemic. Only while I was talking to some of the new readers, I realised that this wasn’t a writing task for me (I still have to add my contribution), but a project management job. Organising so many people from so many countries was – and still is – some work, but oh what a pleasure, too.

Day 3

What I remember from this day most of all, is what Stéphanie Desby shared with me about her break from – and return to working in tech. I find it super interesting what leads people to leave tech, at least for a while, and then come back. As I had to leave tech for while myself, that’s probably no too surprising.

An Afterthought

I remember that in some (pre-pandemic) years, the conference covered the whole week, either with 2 tutorial days, or 4 conference days. While I thought the conference should move back to this longer format, now I’m not so convinced anymore. 4 full days with a LOT of input, talking and, yes, having fun, is demanding.

That said: I am already looking forward to the Agile Testing Days 2023! 🦄🌈

Another Way to Write Ruby Code

Disclaimer: I am not suggesting to change the Ruby style guide.

At a workshop I was giving a few years ago, someone not used to writing Ruby code found an interesting way, that you can write Ruby code.

First let’s assume a class Thingy, that doesn’t do anything useful. It’s just needed to demonstrate the way to write Ruby.

Assume this is in file ‘thingy.rb’:

# frozen_string_literal: true

# Thingy is only used to demonstrate
# a way of writing Ruby code
class Thingy
  def initialize(*args)
    @args = args

  def this(other)
    @args << other

  def that(*other)
    @args << other

  def content

Now, let’s use this class in another script, that’s showing the alternative way to write Ruby code (in file use_thingy.rb):

# frozen_string_literal: true

(require_relative 'thingy')

part = (Thingy.new 'String', :symbol, Math::PI)

(p ((part.this 'Wahoodie?!').that :huh, Math::E).content)

Notice that rather LISP-like way to parenthesise, in line 7 in particular. I still am surprised that this is possible in Ruby and actually behaves the way I’d expect.

Running that script yields the following output:

> ruby use_thingy.rb
["String", :symbol, 3.141592653589793, "Wahoodie?!", [:huh, 2.718281828459045]]

It’s also entertaining that Rubocop does not complain about this code:

> ls
thingy.rb     use_thingy.rb
> rubocop .
Inspecting 2 files

2 files inspected, no offenses detected

This is one of the reasons I like programming in Ruby so much: One can discover new ways (even if probably not very useful ones, sometimes) even after years of using it.

In case you’d like to experiment with this code: It’s on GitHub: https://github.com/s2k/alternative_way_to_parenthesise_in_ruby

Getting Started with Ruby and rbenv on a Raspberry Pi

In preparation of a workshop at Agile Testing Days 2022, I’m setting up a Raspberry Pi as a backup system for participants, to be prepared if things go wrong. Especially one of the first steps “Installing Ruby – If Necessary” has the potential to fail or take too long.

With the keyboard configured (see the previous post “Setting Up a Raspberry PI with a German Mac Keyboard“), the next step is installing a recent Ruby version. I’ll use rbenv , a widespread tool to manage Ruby versions on a machine.

Installing rbenv

The rbenv page suggests to install the tool using a basic git checkout.

~ $ git clone https://github.com/rbenv/rbenv.git ~/.rbenv
Cloning into '/home/guest/.rbenv'...
remote: Enumerating objects: 3138, done.
remote: Counting objects: 100% (288/288), done.
remote: Compressing objects: 100% (147/147), done.
remote: Total 3138 (delta 165), reused 231 (delta 131), pack-reused 2850
Receiving objects: 100% (3138/3138), 626.69 KiB | 3.12 MiB/s, done.
Resolving deltas: 100% (1955/1955), done.

Following the next step in the docs, .bashrc is updated to initialise rbenv:

~ $ echo 'eval "$(~/.rbenv/bin/rbenv init - bash)"' >> ~/.bashrc

Restarting the terminal app actually loads the updated .bashrc, and then rbenv is installed and configured.

Another step is to also install the ruby-build plugin, which rbenv uses to compile and install new Ruby versions. I’ll use git to clone this plugin and upgrade it (as documented in https://github.com/rbenv/ruby-build#readme):

~ $ git clone https://github.com/rbenv/ruby-build.git "$(rbenv root)"/plugins/ruby-build

Now, rbenv can be used to install Rubyies:

~ $ time rbenv install 3.1.2
To follow progress, use 'tail -f /tmp/ruby-build.20221110172124.19039.log' or pass --verbose
No system openssl version was found, ensure openssl headers are installed (https://github.com/rbenv/ruby-build/wiki#suggested-build-environment)
Downloading openssl-3.0.7.tar.gz...

As a last step set this new Ruby version to be used globally:

rbenv global 3.1.2

That’s it. Ruby 3.1.2 is now available for the user an the Raspberry Pi.


%d bloggers like this: