Category: Feedback

Reviewing Submissions for the Agile Testing Days

Conferences Feedback Uncategorized 2 Comments

Other reviewers have also blogged about this topic:

The other day a discussion about the review process of the Agile Testing Days developed:

Since I contributed to this thread and was a reviewer for this years programme, here’s my take. It’s my personal view and other reviewers may well have other aspects they focus on.

  1. On the conference page there are blog posts covering how to write a good proposal. I suggest to read them. This blog contains some tips as well:
  2. The conference offers a list of ‘hot topics’ which changes each year. If a proposal fits to this list it’s a plus, since this is a step towards a consistent conference programme.
  3. I prefer proposals that catch my interest, without telling too much about the topic. – If a proposal already expels everything well, time may be better spent in another session.
  4. A well written abstract text, that is easy to understand (for me) is a plus, too. This includes avoiding typos and grammatical mistakes. We all make them, and even the best spell checkers can’t catch all issues. But still, some proposals are really hard to understand due to language problems. Don’t let that get in your way of getting accepted. My tip: Get feedback by a native English speaker before (!) submitting. Many well known testers and speakers offer help and it is worthwhile accepting this help.
  5. Understand what the fields in the proposal form are meant for. Fill them to provided the information that is asked for,
    Avoid repeating the same text in different parts for the form. Change the wording at least a bit. In some cases the title, sub headline, main statement and key learning(s) contained exactly the same or very similar text. To me as the reviewer this is a little bit boring, and doesn’t help me understand what the session is about.
  6. Sometimes, repeating is worthwhile: It helps to understand what is important. Use this tool carefully.

Some questions may guide to writing a good proposal:

  • Will this help the reviewer to give me a high rating?
  • Am I giving enough information to inform a potential attendees decision to come to my session?
  • Am I giving too much information?
  • Is this a good fit for the conference this year?

A leaving personal note: It took me years to get accepted at the Agile Testing Days, even for what was then called a ‘consensus talk’. In the very early years the proposals weren’t very well written, in some years I failed to match the overall conference theme. And then it clicked, I asked for help, gave workshops, a tutorial and, in 2019 even a keynote. For me it was worth the effort.

Good luck and may your proposals be accepted!

Expressing Expectations

Experiment Feedback Learning Pattern Testing No Comments

Long long ago, while preparing experiments for my diploma thesis in physics, my tutor taught me to express my expectation of the outcome of experiments before actually running them. I was to not only to express them in my head, but speak them out loudly, may be even make a note.

This helped a lot, when figuring out where my thinking didn’t match the experimental evidence. There was no denying when my expectation differed from the empirical result. Typically, there were two sources for the differences:

  1. My mental model wasn’t good enough to match the result of an experiment.
  2. The experimental setup wasn’t designed well enough to show the effect I was trying to measure.

I find that this is still working well in software development (both the coding part as well as testing). A related article about this is Peter Naurs seminal paper ‘Programming as Theory Building’.

When doing TDD (test driven development), explicitly expressing the outcome before running a test may not always lead to a surprise. In the very beginning, when a test tries to create an object, without having a class definition, it will cause an error message that is easy to predict.

However, when work has progressed a bit, I regularly run into situation where I expect the next new test to fail in a certain way, but when running the test, the actual error message is a surprise to me. These are situations where learning can happen, by figuring out why the actual behaviour does occur, instead of what I predicted.

Near the end, the surprise is caused differently: I’ll write a new test and expect it to fail. – It doesn’t. Again, this is a good reason to explore why exactly I thought the test would fail.

Particularly in a pair (or ensemble) programming setup, the inability to come up with a new failing test is a sign, that the implementation is good enough … for now.

Recently, I did a programming exercise as part of a technical interview, and expressing my thoughts and expectations while working on the code helped me to find a solution. Additionally, the interviewer didn’t only see what I was typing, but could follow my way of thinking. This relates to Naur’s paper mentioned above: The testing and tested code I wrote isn’t everything there is to my mental model of the given problem or its solution. The way of thinking is important too. This is why I find vocally expressing my expectations & thinking while actually doing the work.

What are your preferred way to actually do the work you’re doing? I’d love to hear about it.

Why Good Error Messages Can Save Time and Effort

Feedback Testing 1 Comment

TL;DR

Make error messages precise enough to help users, so that they can resolve the problem, or at least enable them to provide meaningful input when talking to support.

The Context

For a project, I was working on an application that stored it’s content as text files. Git was used as a storage backend, so the application could track who made what changes when. – So far so good. Testing this application was done on a specific test environment somewhere on the net.

While testing was possible, it was inconvenient to have to also log in the the logging server so track what exactly was happening, and updating the application itself was not as easy either. Therefore, one day I started to set up the whole thing on my local machine. The setup and configuration was surprisingly easy. We had good documentation for where to put which files. This was the easy part. I could start the application & tell it Git repository contained the application content (those text files mentioned above).

The Problem

The problem occurred, when I tried to actually get the application data:

XYZ cannot access the repository!”

The application’s error message as displayed to the user

That’s not particularly helpful. On order to support failure analysis, an error message shown to a user should help identifying the problem and solving it, or to have a meaningful exchange with support.

I checked the log files and quickly found some related information:


<timestamp> INFO : <class_info_and_linenumber> - SSH folder: <absolute_path_to_ssh_folder>
<timestamp> DEBUG: <class_info_and_linenumber> - SSH identity file: <absolute_path_to_ssh_key_file>
<timestamp> ERROR: <class_info_and_linenumber> - Exception <library_name>Exception: Invalid private key: <key_identifier> requesting Git repo
<full_class_name>.<ExceptionType>: Cannot list remotes for git url <ssh_url_to_git_repository>

I found this surprising, since I was using the exact same identity files to access the very same Git repository from my IDE & other Git clients on my machine. Had something corrupted my key? If so, what was it?

I started by increasing the logging of the SSH library I’m using, to make sure I was using the exact same key pair in all situations. A good while later, after googling for the error message, talking to developers, and reading some documentation, I found out this: The library being used wasn’t coping well with the encryption algorithm I used when generating my public & private key pair.

While I used ‘ed25519‘, a (relatively) recent algorithm (at the time of writing this), the library (in the version that was used in the application) expected the key to be generated using ‘RSA‘. Some details are also in the post ‘“Invalid privatekey” when using JSch‘ on stackoverflow. The problem is the misleading error message: The key wasn’t invalid at all, since I could access the repository using it with other programs that (apparently) used other libraries. The key type was unknown to the library. That’s similar, but different.

A Better Error Message

Had the library error message been something like Key <key_identifier> not recognised; see documentation for supported key types, identifying and solving the problem would have been much faster and less frustrating. Words matter in error messages, too.

Error Messages Should Help You Resolve the Error

I prefer error messages to be detailed enough to help me identify the real cause of the issue and ideally give a hint at what I can do.

Take widespread error messages for pass word fields for example. They may read like this:

Your password must have at least
12 characters overall
1 uppercase & 1 lowercase character
1 number
1 of these characters: # $ % & – ? = / . , ;

A contrived error message, that’s entirely likely

Yes, there’s a lot to be said about the value and limitations of requirements like these, but at least the message helps a user to set a password that complies with the mentioned rules.

A good error message can help users to identify a problem and probably resolve it as well. They don’t have to contact your support team. Your support team doesn’t have to go though the process of identifying the same problem(s) over and over. This saves time (and probably money) on both sides, yours and the user’s.

Writing a Ruby Related book(let) on LeanPub

book Feedback Ruby 2 Comments

I wrote “Fast Feedback Using Ruby” using LeanPub. The original version was published back in 2015. Since I’m updating it to cover the most recent (at the time of this writing) Ruby 2.7 and RubyGem versions. I figured, it’s a good idea to leave a trace of my workflow and setup. 🙂

My workflow look like this:

  1. Edit Text (or image) file
  2. Save file to local disk
  3. Commit to local Git repository
  4. Push to GitHub
  5. Generate preview PDF on LeanPub using a (post push) web hook
  6. Have Leanpub automatically push files to Dropbox
  7. Have Dropbox synchronise PDF file to local folder
  8. Restart at 1
Writing workflow using LeanPub, GitHub and Dropbox

Whenever I push changes to GitHub, a new preview of the book is generated and sent to my computer. This means I get frequent and fast feedback about how the newly written text looks.

This nicely fits my working preferences and it also matches the topic of this particular book.

Announcement: Workshop & ebook: “Fast Feedback Using Ruby”

book Feedback Learning Ruby Testing Tools for Testing No Comments

At the “London Tester Gathering Workshops 2015” (also see #LTGWorkshops on Twitter) I offer a workshop “Fast Feedback Loops & Fun with Ruby”. For this, I wanted to give attendees a handout, to make applying the stuff covered easier and it was planned to be a list of brief recipes.
Suffice to say that the handout grew (and it’s entirely possible that this is ‘feature creep’ in action). In fact, it grew to the point that I decided to turn it into an ebook. The book is not done yet, there’s some copy editing to do.

Fast Feedback Using Ruby

In any case: There will be an ebook, and it will be available on LeanPub at https://leanpub.com/fastfeedbackusingruby/. If you’re interested, please leave a note on the book’s pages.

© 2012-2022 Stephan Kämper

MENU

Navigation

%d bloggers like this: