Author: Stephan

About Stephan

I'm a self-employed software tester. I test and write software, listen to music and go hiking — sometimes simultaneously. Prior to this was a physicist and oceanographer, among other things.

Expressing Expectations

Long long ago, while preparing experiments for my diploma thesis in physics, my tutor taught me to express my expectation of the outcome of experiments before actually running them. I was to not only to express them in my head, but speak them out loudly, may be even make a note.

This helped a lot, when figuring out where my thinking didn’t match the experimental evidence. There was no denying when my expectation differed from the empirical result. Typically, there were two sources for the differences:

  1. My mental model wasn’t good enough to match the result of an experiment.
  2. The experimental setup wasn’t designed well enough to show the effect I was trying to measure.

I find that this is still working well in software development (both the coding part as well as testing). A related article about this is Peter Naurs seminal paper ‘Programming as Theory Building’.

When doing TDD (test driven development), explicitly expressing the outcome before running a test may not always lead to a surprise. In the very beginning, when a test tries to create an object, without having a class definition, it will cause an error message that is easy to predict.

However, when work has progressed a bit, I regularly run into situation where I expect the next new test to fail in a certain way, but when running the test, the actual error message is a surprise to me. These are situations where learning can happen, by figuring out why the actual behaviour does occur, instead of what I predicted.

Near the end, the surprise is caused differently: I’ll write a new test and expect it to fail. – It doesn’t. Again, this is a good reason to explore why exactly I thought the test would fail.

Particularly in a pair (or ensemble) programming setup, the inability to come up with a new failing test is a sign, that the implementation is good enough … for now.

Recently, I did a programming exercise as part of a technical interview, and expressing my thoughts and expectations while working on the code helped me to find a solution. Additionally, the interviewer didn’t only see what I was typing, but could follow my way of thinking. This relates to Naur’s paper mentioned above: The testing and tested code I wrote isn’t everything there is to my mental model of the given problem or its solution. The way of thinking is important too. This is why I find vocally expressing my expectations & thinking while actually doing the work.

What are your preferred way to actually do the work you’re doing? I’d love to hear about it.

Agile Testing Days 2021 – Part 3

If you haven’t you might be interested to read the previous part of this series as well.

This day impressed me most by the three key notes and discussing the effects of corona and working from home with Anne Colder.

Thursday

The first keynote of the day was Ard Kramer’s “How to nudge your way through agile testing”. Ard presented six ways to nudge people to make a decision in a certain way – probably a way that we want them to go. In a very consistent way he made the distinction between doing something (the nudging) and the ethics of doing it.

It’s so important to aware of these techniques, because then we can more consciously decide whether or not to follow the nudging.

These are the six kinds of nudging he explained:

  • Default options
  • Commitment through consistency
  • Anchoring
  • Decoy effect
  • Zeigarnik effect
  • Activate unconscious behaviour

The name Zeigarnik effect was new to me, although I read about the way it works somewhere on the net. Basically it states that one can remember an activity that has been interrupted (not not completed) more easily at a later point in time.

The Tester’s Learning Toolkit” was the second keynote, presented by Vera Baum. Supported by incredibly great graphics Vera explained the various levels of experience people may have.

  1. Novice
  2. Apprentice
  3. Crafter
  4. Expert

She explained how we can develop from level to level, and why it may not be the best approach to let experts teach novices.

The last keynote of the day – and the conference – was Vernon Richards‘ “What does the ‘Coach’ in ‘Quality Coach’ mean?”. He introduced 6 styles of leadership and how they can be applied in the context of software quality. I loved his way of giving examples his experiences in applying them.

With this highly interesting and super entertaining keynote ended the official program of the Agile Testing Days 2021.

In the evening, I discussed the effects of having to work from home with Anne Colder, leading to another contribution for the ebook “Software People … Work From Home“. Stay tuned. 🙂

This ended a truly brilliant experience of the Agile Testing Days. It was so great to finally meet real testers in real live, discuss software related topics during breakfast, lunch and dinner… as well as in between.

Thank you! Thank you to everyone I spoke to, especially the organisers who ran an incredible conference!

Agile Testing Days 2021 – Part 2

If you haven’t read it yet, you may like to start with the first part of my Agile Testing Days 2021 summary.

Wednesday

I started Wednesday with a Lean Coffee. At the Agile Testing Days this is traditionally facilitated by Janet Gregory and Lisa Crispin.

We discussed a good number of topics, such as how to get folks to try new ways, ways to practice testing and the role of testing/testers. I like this way of quickly covering a broad field of topics, to give people input to work with. I recommend tying this format, if you haven’t done so yet.

Jutta Eckstein‘s keynote “Agile Comes with a Responsibility for Sustainability” covered the three main parts of sustainability:

  1. People, the social equity bottom line
  2. Planet, the environmental bottom line
  3. Profit, the economic bottom line

See for example the Wikipedia article about the ‘Triple Botton Line’ for more details. I find that these aspects are all incredibly important and also believe that we, the community of software people, have a shared responsibility to help achieve improvements in all three areas.

The second keynote on Wednesday was João Proença‘s “Limitless within our boundaries”. He explained nicely how limiting options can improve creative work. His example were his Band that got completely stuck, when they experienced the possibilities of a music studio for the first time. It’s the same in most other contexts, including software development. I very relaxed presentation style and body language. I found it super nice that he reminded me, that I recommended submitting a session to the Agile Testing Days – and I am really, really glad I did.

The session “Resistance is futile” by Anne Colder & Jantien van der Meer, was about how to facilitate changing behaviour. The session was Star Trek themed and I am luck I survived in my red shirt. 🙂 Details about the model of “The Rider, the elephant and the path” are, for example, available at https://www.creativehuddle.co.uk/post/the-elephant-and-the-rider. I like the way they explained the model as well as the exercises to illustrate how it can be used.

For me Bruce Hughes‘ keynote “How to be an Ally to Non-binary Folk in Tech” was next. It was brilliant, hilarious, sad and helpful. The standing ovations she received were absolutely well deserved. The blog post appache attack helicopter wrote at https://undevelopedbruce.com/2020/07/06/non-binary-in-tech/ explains what was covered in xer keynote. I recommend reading it. To her ‘appache attack helicopter’ is an acceptable pronouns BTW. See: https://undevelopedbruce.com/about-me/ (read that as well, I may help further understanding the complexities of existence). Lisi Hocke tweeted a great summary of this keynote:

The last session I attended as Søren Wassard‘s “Digesting Poets Society”. A very nice, relaxing and emotional bonus session about another way to get creative: Poems. We mostly heard English poems, but also one from a Welsh author, recited in Welsh by a Welsh (and no other than Bruce).

Part 3 of the series is available now, too.

Agile Testing Days 2021 – Part 1

After a two year break, I could finally attend the Agile Testing Days in person. I was a bit worried about having to deal with the corona virus floating around and infection counts increasing significantly in Germany. However, the organisers handled this exceptionally well: Only vaccinated or recovered people were allowed to attend, every attendee received two test sets for self-testing and the certificates to prove vaccination/recovery were checked twice: During check-in at the hotel and the conference. During the conference it became obvious that many (likely most?) people tested every day. For me this meant I felt a lot better!

Additionally, they offered ‘colour coded’ masks: A blue masks indicated “I’m OK with getting near people, even hugging” and black ones which meant “I prefer some distance, please”. I took a black one and was pleased that people respected this. Thank you everyone!

What follows is a summary of some sessions and events.

Monday

Originally I had planned to give my workshop “Get a Lot Out of This Conference” (a teaser video is available on Vimeo), however it turned out that most first time attendees weren’t at the conference on Monday already, some would even only arrive on Wednesday. Since this workshop makes most sense when done before the conference, we decided to cancel it. And that’s OK since my ‘plan B’ was to attend the session to build a Tiny House. – Sadly this session also had to be cancelled, for reasons outside the control of any one at the conference. Thankfully the organisers allowed us non-builders to join on of the other sessions of this day and I could participate in “A Path to Holistic Testing” presented by Lisa Crispin and Janet Gregory. Alex Schladebeck also joined my table. It was super interesting to see how being a CEO can change focus, when it comes to sign-up forms for courses (this was the example we used in the exercises). Hint: The GDPA plays an important role.

I learned a lot in this super entertaining tutorial. This will be very useful in helping my team.

Tuesday

A session I looked forward to a lot was Maaike Brinkhof‘s “The Struggle with Learning How to Code”. I enjoyed the story telling about her experiences in learning to code immensely. She also offered great tips for how to prevent the barriers she ran into.

The second talk I attended was Alex Schladebeck who presented “Unit Testing and TDD from the tester perspective”. It was a great presentation and I am already looking forward to getting the slides which contain links to resources and further information.

Some insights of Alex' talk: TDD can be seen as exploratory programming.

The ‘traditional’ highlight of this day is the costume party including dinner and the award ceremony for the MIATPP (Most Influential Agile Testing Professional Person) award. This year I had the honour to be asked to announce the winner. As every year, first the dinner is served and then the award. I had a hard time keeping a straight face when the (future) winner asked if they may join my table. Introducing the winner and keeping the tension up to the very last moment was great fun, especially since apparently no-one had any clue who might be the MIATPP this year.

Again congratulations, Raj Subrameyer, MIATPP Award Winner of 2021!

Continue reading the second part at https://seasidetesting.com/2021/11/22/agile-testing-days-2021-part-2/

Contributing to Open Source – Getting Started

A sign in LegoLand saying "Adventure Land"

Recently, I started to contribute more to open source projects I use and like. When I started to contribute at all, I did so, with the most obvious thing for me: Bug reports. 🙂 This can be particularly useful, since the bug reports that I write for my projects, stay behind closed doors. – Like it or not, companies producing commercial software very often don’t fancy public bug reports. That’s a long story for another time…

Here are some tips:

  • Start small. Probably smaller than that.
    You can start on documentation. ➙ It can be unclear or contain typos.
    My smallest contribution so far, was fixing a typo in the Cucumber documentation.
    The steps to deliver it were large, compared to the fix itself:
    • Fork a repository
    • Create a branch on the fork
    • Fix the typo
    • Create a pull request
    • Get it accepted
      This seems like a lot, just to fix a typo.
      However, I think the learning experience was worth it.
  • Learn about the two most used models used in open source projects, the “Fork and Pull Model” and the “Shared Repository Model”.
    Both are explained in the GitHub docs about “collaborative development models“.
  • Check out Marko Denic on Twitter. He wrote “Make Your First Open Source Contribution” which explains to to get started on GitHub.
    You can contribute to his site https://tech-blogs.dev by adding you own blog, if you have one. The project is hosted on GitHub.
  • Start in a friendly & helpful community
    I find this really important: When you have a safe place to ask all the question you may have, and can expect a friendly answer, that will help actually asking those questions. It certainly helped me, asking about which development model is used, how big or small a pull request should be, etc.
  • Create your own project and make it open source
    Again: You can start really small. My smallest GitHub repository is probably https://github.com/s2k/seasidetestings-iterm-colours. It’s just only file that can be used as a colour preset in iTerm2 and some documentation how to load the file in iTerm.
  • If you find that something can be improved or isn’t working as expected, provide a bug report. These are important contributions to any software project: If the people writing the software don’t know that something’s wrong with it, they can’t fix it.
    For example I wrote https://github.com/fakefs/fakefs/issues/224, when I noticed that this Rubygem didn’t worked the way I expected.
    Since I knew how to write the RSpec to check this, I did – and it probably helped to get it fixed.

If you’re looking for much more information about how contributing to open source this book may be for you: “Forge Your Future with Open Source” by VM (Vicky) Brasseur.

Post Scriptum, 2022: Richard Schneeman published “How To Open Source“, another book about how to start contributing to OSS. I recommend this one as well.

Why Good Error Messages Can Save Time and Effort

TL;DR

Make error messages precise enough to help users, so that they can resolve the problem, or at least enable them to provide meaningful input when talking to support.

The Context

For a project, I was working on an application that stored it’s content as text files. Git was used as a storage backend, so the application could track who made what changes when. – So far so good. Testing this application was done on a specific test environment somewhere on the net.

While testing was possible, it was inconvenient to have to also log in the the logging server so track what exactly was happening, and updating the application itself was not as easy either. Therefore, one day I started to set up the whole thing on my local machine. The setup and configuration was surprisingly easy. We had good documentation for where to put which files. This was the easy part. I could start the application & tell it Git repository contained the application content (those text files mentioned above).

The Problem

The problem occurred, when I tried to actually get the application data:

XYZ cannot access the repository!”

The application’s error message as displayed to the user

That’s not particularly helpful. On order to support failure analysis, an error message shown to a user should help identifying the problem and solving it, or to have a meaningful exchange with support.

I checked the log files and quickly found some related information:


<timestamp> INFO : <class_info_and_linenumber> - SSH folder: <absolute_path_to_ssh_folder>
<timestamp> DEBUG: <class_info_and_linenumber> - SSH identity file: <absolute_path_to_ssh_key_file>
<timestamp> ERROR: <class_info_and_linenumber> - Exception <library_name>Exception: Invalid private key: <key_identifier> requesting Git repo
<full_class_name>.<ExceptionType>: Cannot list remotes for git url <ssh_url_to_git_repository>

I found this surprising, since I was using the exact same identity files to access the very same Git repository from my IDE & other Git clients on my machine. Had something corrupted my key? If so, what was it?

I started by increasing the logging of the SSH library I’m using, to make sure I was using the exact same key pair in all situations. A good while later, after googling for the error message, talking to developers, and reading some documentation, I found out this: The library being used wasn’t coping well with the encryption algorithm I used when generating my public & private key pair.

While I used ‘ed25519‘, a (relatively) recent algorithm (at the time of writing this), the library (in the version that was used in the application) expected the key to be generated using ‘RSA‘. Some details are also in the post ‘“Invalid privatekey” when using JSch‘ on stackoverflow. The problem is the misleading error message: The key wasn’t invalid at all, since I could access the repository using it with other programs that (apparently) used other libraries. The key type was unknown to the library. That’s similar, but different.

A Better Error Message

Had the library error message been something like Key <key_identifier> not recognised; see documentation for supported key types, identifying and solving the problem would have been much faster and less frustrating. Words matter in error messages, too.

Error Messages Should Help You Resolve the Error

I prefer error messages to be detailed enough to help me identify the real cause of the issue and ideally give a hint at what I can do.

Take widespread error messages for pass word fields for example. They may read like this:

Your password must have at least
12 characters overall
1 uppercase & 1 lowercase character
1 number
1 of these characters: # $ % & – ? = / . , ;

A contrived error message, that’s entirely likely

Yes, there’s a lot to be said about the value and limitations of requirements like these, but at least the message helps a user to set a password that complies with the mentioned rules.

A good error message can help users to identify a problem and probably resolve it as well. They don’t have to contact your support team. Your support team doesn’t have to go though the process of identifying the same problem(s) over and over. This saves time (and probably money) on both sides, yours and the user’s.

Conveniently start a JavaScript shell (jsc) on macOS

For one of my projects I wanted an easy way to try JavaScript on a command line (similar to pry or irb in Ruby). Here’s how I found out where the program is located and how to set up my Mac to conveniently start it.

1. Find out where jsc is located on the machine:

$ find / -name jsc -type f 2>/dev/null
/System/iOSSupport/…/JavaScriptCore.framework/Versions/A/Helpers/jsc
/System/…/JavaScriptCore.framework/Versions/A/Helpers/jsc
/System/Volumes/Data/…/JavaScriptCore.framework/Versions/A/Helpers/jsc
/System/Volumes/Data/…/JavaScriptCore.framework/Versions/A/Helpers/jsc    

Searching from the root folder may be a bit excessive, though. You may consider a more limited search.

2. Look into the ‘system frameworks’ for versions

The path /System/Library/Frameworks/… was where I went to look what else is inside:

$ ll /System/Library/Frameworks/JavaScriptCore.framework/Versions
total 0
drwxr-xr-x  4 root  wheel  128 Jan  1  2020 .
drwxr-xr-x  4 root  wheel  128 Jan  1  2020 ..
drwxr-xr-x  5 root  wheel  160 Jan  1  2020 A
lrwxr-xr-x  1 root  wheel    1 Jan  1  2020 Current -> A

Aha, there’s a link named Current that (currently) points to A. I used this link in the next step. This way I can still use the same link, even if (when!) an OS update causes the file that’s linked to changes.

3. Link to the current version

I set a link somewhere in within $PATH. I have a bin folder in my home directory, so it put the link there:

ln -s /System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/Helpers/jsc ~/bin/jsc

3. Set a variable ‘console’ for output

To output things, use this to define a variable console in a running JavaScript shell.

var console = {log : debug};

While jsc provides a print function, I find it convenient to stick to the more idiomatic console.log.

4. Use ‘jsc’

$ jsc
>>> var console = {log : debug};
undefined
>>> console.log(function(){})
--> function (){}
undefined
>>> 1 + 1
2 

%d