Category: Testing

A Year of ‘Explore’

Learning Testing 2 Comments

At the end of last year, I decided I would not make any new year’s resolutions in favour of selecting a ‘word of the year’, see the blog post from early 2013. I picked the word ‘explore‘ for 2013 and since the year is (nearly) over, it’s time to look back and see how it worked out.

The Black Box Software Testing Courses

At the time of this writing there are four BBST (Black Box Software Testing) courses the Association for Software Testing offers:

  1. Foundations
  2. Test Design
  3. Bug Advocacy
  4. Instructors

I took the first three of them in the first half of 2013, and the last one in October. And while I do recommend taking these courses, I have to say that I needed a good amount of time to work through all the material, labs & exercises. Especially the ‘Test Design’ course offered a phenomenal amount of material.

I totally recommend these courses to everyone working in software testing and software development in general.

Conferences

For a while now, I try to go to two conferences each year, a programmer conference as well as a tester conference. I used to recommend this to my fellow ‘programming tester colleagues’, but now I’ve also started to recommend it to the ‘testing programmers’ as well. While I focus on software testing, I find it useful to know a bit about programming, too.

I hoped to be able to go to the EuRuKo (European Ruby Conference) in Athens, Greece in summer. In the end it didn’t work out as planned and I couldn’t go. However, I gave my ticket away and received two post cards in return. Thank you, you know who you are.

In September I attended the BARUCO, the ‘Barcelona Ruby Conference’ in Spain, and in October I went to the Agile Testing Days in Potsdam, Germany. At both conferences I gave short presentations about testing and the two values of software. Furthermore, at the Agile Testing Days I had the pleasure to assist Lisa Crispin and Janet Gregory at the  beginning of their keynote presentation:

A remark: The BBST Instructors course and the Agile Testing Days overlapped a bit. If you can, I suggest to avoid a commitment like that. Although it did work for me (in the end), this is a way of exploring, I’ll avoid in the future.

Projects

Early this year, I wanted to try some rather short software testing projects and joined uTest (now renamed to be Applause), where I worked on several apps for mobile devices as well as OS X. Given my background with longer running projects, having just a few days for testing was a refreshing experience. I also joined a project using Calabash to automate testing (well, checking actually) on Android devices.

In late June I joined a new longer running project using a whole bunch of technologies I’ve heard about before, but which were new to me – Another way to explore! So now I work in a project using MongoDB, Varnish, Puppet and Vagrant. All of them are really interesting technologies, and the team doesn’t stop there: Every now and then we take a day to, well, explore new ways that may improve our work.

Summary

It’s been a very exciting and busy year and I am convinced that picking a ‘word of the year’ instead of making new year’s resolutions made a big difference. Instead of a plan, I felt I had some guidance that helped in deciding what to do (and what not to do in some cases). I will pick a word of the year for 2014 as well and if you also pick one, or if you already had one for 2013, why not write a short comment whether (or not) it helped you and in which way?

Testing And The Two Values of Software

Feedback Testing 8 Comments

The Two Values of Software

Sure enough: If your team’s software fails to provide value now (or in the near future), something’s gone badly wrong. You don’t like your software project to end up similar to this, right?

Smoking Remainders
Smoking remainders of a ‘Biikebrennen’ — don’t let your project end like this.

As Alistair Cockburn explained in “Agile Software Development” software development can be seen as a game played in rounds — and most teams prefer to stay in the game for many rounds.

To me it’s interesting (and a bit worrying) that most of the testing techniques and approaches are focussed on whether the current version of our software…

  1. is in fact the right software and
  2. if it works correctly.

In other words, testing focusses on validation and verification, it concentrates on a relatively short period of time: the next release.

Now, Uncle Bob Martin presented the idea of the primary and secondary value of software in episode 9 (‘The Single Responsibility Principle’) of his CleanCoders video series. He presents the secondary value first (which most people believe to be more important): ‘The secondary value of software is its behavior.’ He also explains the primary, more important value as: ‘The ability of software to tolerate and facilitate such ongoing change is the primary value of software. The primary value of software is that it’s soft.’

In Alistair Cockburn’s description the order is reversed (see Agile Software Development, 2nd edition p. 37 ff), but in my opinion the order isn’t always that important: I am worried that testing doesn’t consider one of the two values of software.

To my experience, essentially all projects focus on current or near-future behaviour of software, but rarely actively work to keep the software ready to deal with future requirements. I understand that YAGNI (You ain’t gonna need it) may play a role here. But mind you: YAGNI mostly applies to actually implemented features that are not needed. It may also be valid in order to prevent going overboard with too much abstraction and flexibility. In the end, you always need to find a balance between following and ignoring the SOLID principles (a good part of the Clean Coders videos covers this in detail).

Concerning the 1st value of software (according to Uncle Bob Martin), I’m convinced that testing needs to answer questions like this:

  • Is “the ability of software to tolerate and facilitate such ongoing change” a job purely for programmers?
  • Can software testers contribute to the primary value? Should they?
    • If yes, is this part of the job or should we keep out of this business?
    • How can we as testers contribute?
  • Can other disciplines help?

What do you think? Is this topic important for testing? Should we treat it in the way Michael Bolton advises testers with respect to “Quality Assurance Business” (spoiler alert/hint: He advises us to stay out of it)?

I personally think testers should get involved: A tester is somebody who knows that things can be different, as Jerry Weinberg says.

Also, I definitely want my clients, as well as their clients, to be happy with the software I help to develop, not only today, but also in the future, even when most original developers of a software system have left the team or company.

When your project (or product life cycle) comes to an end it should stay in your memory not like the smoking remainders of a fire, but with the bright colours of a great sunset.

Bright Sunset
Bright Sunset

Anthropomorphism And Dehumanisation In Testing — How Does It Affect Us?

Anthropomorphism Testing No Comments

Where it began

I find it interesting to keep track of where and when I run into new information relevant to software testing. This time it was an article in a German magazine for dog owners (issue 6 2012, to be precise). The topic was anthropomorphising dogs. And I think digging deeper into anthropomorphism can help us understand software development and testing better.

What is anthropomorphism?

This is the definition of anthropomorphism given on Wikipedia:

Anthropomorphism or personification is any attribution of human characteristics (or characteristics assumed to belong only to humans) to other animals, non-living things, phenomena, material states, objects or abstract concepts, such as organisations, governments, spirits or deities. (http://en.wikipedia.org/wiki/Anthropomorphism)

Why is it worth thinking about?

The German magazine article mentioned Adam Waytz as the author of a study and that way I found “Social Cognition Unbound: Insights Into Anthropomorphism and Dehumanization” by Waytz et al. (citation see end of this article).

The authors of the paper (details see below) say that “computers … can seem to have minds of their own” and mention that people “…curse at unresponsive computers…” (at least I have). If similar situations happened to you, maybe while testing software, then anthropomorphism affects your work.

I’m convinced that this topic requires more than a single post. This post is an introduction to the topic.

A brief summary of the paper

The authors discuss reasons for anthropomorphism as well as dehumanization and present three “primary factors——elicited agent knowledge, sociality motivation, and effectance motivation” which determine whether someone is likely to anthropomorphise or not.

Let’s look at the three factors:

Elicited Agent Knowledge
Anthropocentric knowledge is accessible and/or applicable.
Social Motivation
This is the need to be accepted, part of society and to connect to others.
Effectance Motivation
The motivation derived from understanding, explaining and (being able to) change or influence the environment.

Some ways this affects software testing

I will not go into much detail in this introductory post, but will instead list some of the behaviours that can be explained through this model:

  1. Effectance motivation and scripted testing
    I will not add to the discussion of the pros and cons of scripted testing, but instead introduce a possible explanation why testing and testers are not well respected in environments where scripted testing is used.
    When a rather strict execution of scripted test cases is demanded, the testers can’t influence their environment much. It’s about following scripts, manually operating the system under test and checking whether given conditions are met or not. In some cases changing scripts is not allowed, just following them. To others this looks like manually executing source code, something a computer is much more suited for. In other words: Testers work like computers, they’re perceived less human-like than they are.
    That way effectance motivation is inhibited, leading to dehumanising of the testers (at least to a certain degree).
  2. Test versus development teams—and what they perceive of each other
    I’ve worked in an environment where testers and developers were separate from each other. The most important communication was though the artefacts we were exchanging: Installed software on the test system, defect reports in a bug tracking system and may be the source code.
    We couldn’t see each other as the humans we are, it was (mostly) artefacts, the ‘other side’ didn’t even look remotely human. Thus neither side felt much respect for each other.
  3. System complexity
    The tester assumes the system is more complex than it is or has an intention that’s not there.
    Software systems, at least today, do not have a human intent. If a system under test is subject to anthropomorphism the tester assumes the system is more complex (or complicated) than it actually is and therefore puts more effort into testing. This can lead to new ways of testing or new test ideas and therefore improve the testing.

Leaving thoughts

This post introduced the idea that anthropomorphism affects software testing and described some of the effects. As mentioned above, I think there’s much more to discover for testing, so stay tuned for more posts about the topic!

Do you have examples of anthropomorphism in software testing? Please share them.

Cited paper

Waytz, Adam, N. Epley and J. T. Cacioppo. 2010. Social cognition unbound: Psychological insights into anthropomorphism and dehumanization. Current Directions in Psychological Science. 19: 58-62.

At the Agile Testing Days 2010: Alternatives to Certifications / Part 1

Learning Testing No Comments

This post appeared fist at http://zenandtheartofautomatedtesting.eu/ on 10. Oct. 2010.
Since I’ll take down that site, the article is relocated.

At the Agile Testing Days: Alternatives to Certifications

A session of the Open Space Day (facilitated by Brett L. Schuchert) at the Agile Testing Days dealt with alternatives to certifications. Elisabeth Hendrickson tweeted a link to the summary of this session already.

Apart from answering the question “What is the value of certifications?” and “Do we need something other than certificated at all?” (neither of which I’ll discuss here), there’s one important question: What are ways to demonstrate your testing knowledge other than certificates?

That’s what this article is about.

I’ll elaborate on some of the alternatives we discussed. Some of them here were brought up by me, others were not. So, if you disagree on a topic you’ve brought up, please let me know1.

Novice – Apprentice – Journeyman – Master

I like the idea of learning from practitioners who have a better understanding and deeper experience in the field I like to learn about. This is a central idea of Software Craftmanship. The idea here is that at each stage you’d learn (mostly) from others one stage ahead of you. This enhances almost everyone in learning by teaching, certainly a very powerful way to learn and internalise material you’ve just learned yourself.

Find someone who’s willing to help you to learn about the topic you chose and who’s experienced.

Note, this is not as much about finding someone to actually teach you2, but help you teach yourself.

Participation in open source projects (as testers)

There are lots of open source projects out there. Whether it’s automated regression checks (or tests, in case you don’t follow Michael Bolton’s terminology) exhaustive exploratory testing or other ways of testing (or checking): You’re probably using some open source software already, and improving this software by providing your testing feedback is a Good Thing.

You help to develop software you’re using, making your own life better3 and you demonstrate your testing practice.

Twitter

By following other testers on Twitter (and may be the #testing hashtag, too) and joining the discussion is another way. Even by ‘just’ following others and reading the linked to blog articles alone I learned a lot. However, joining the discussion at your pace4 will give result in a significantly richer learning experience.

1 Of course I wouldn’t mind hearing about your agreement as well. And in case you do agree; I’d like to hear from you, too.

2 Experienced practitioners happen to be usually rather busy. However they’re also happy to help you to help yourself.

3 Also known as “Eating your own dog food”

4 Some tweet at a remarkably high frequency, that’s not easy to follow at times. Don’t panic.

Agile Testing Days 2012 — YAS (Yet Another Summary)

Feedback Learning Testing No Comments

Just like the years before I enjoyed the Agile Testing Days a lot. A fun theme throughout many of the talks were… unicorns and I think Gojko Adzic started it. This affected my brain so much, I said ‘Look, a unicorn!’ to my dog, when I took him for a walk for the first time after I came back from the conference. Actually, the animal crossing our path was a squirrel.

Here’s a short summary of my favourite parts. In order to stay in sync. with the printed & online conference program I’ll start counting the conference days at 0.

Day 0 – Tutorial “Software Testing Reloaded” with Matt Heusser & Pete Walen

Just as the full title “Software Testing Reloaded – So you wanna actually DO something? We’ve got just the workshop for you. Now with even less PowerPoint!” promised, there was only very little PowerPoint and a whole lot of testing & questioning.

I especially liked the way Pete & Matt presented the examples & exercises as well as the reasoning behind them.

Electronic Dice
Electronic Dice

There were a lot of games and actual testing. All was very hands on, well explained and debriefed at the end. To give just one example: We tested electronic dice, in order to give an estimation about how long testing would take and to come up with a recommendation whether these dice were ready to be used.

Questions like “Can we ship it now?”, “Is there a pattern/problem there?” and “What actually are the requirements?” were covered.

I also won a price, so my point of view might be biased. 😉

How to Reduce the Cost of Software Testing

Day 1

  • Get them in(volved) by Arie van Bennekum
    I found it super interesting to listen to one of the creators of the Agile Manifesto, especially since he pointed out that many of the principles and values have been around before the manifesto was written.
  • Myths About Agile Testing, De-Bunked by Janet Gregory & Lisa Crispin
    Lisa & Janet debunked myths as ‘Testing is dead’, ‘Testers must be able to code’ and ‘Agile = Faster’. Excellent story and fun presentation.
  • Consensus Talks – 7 10-minute-talks (including mine)
    The format of 10-minute talks, all back-to-back and no break included was the one I missed in the previous Agile Testing Days. This way a whole lot of ground is covered in a short time and it’s a great opportunity to explore new techniques in presenting without doing too much harm and/or presenting on conferences for the first time.
  • Self Coaching by Lasse Koskela
    Lasse explained how to coach yourself. After explaining how the human brain works he talked about how to step ‘out of the box’ (your personal point of view) in order to better understand what others actually say and to stay honest to yourself at the same time. Deep knowledge, very well explained indeed.
  • The MIATPP Award Night 2012
    Lisa Crispin won the MIATPP Award. Congratulations!

Day 2

  • Test Lab by James Lyndsay & Bart Knaack
    This year I went to the “Test Lab” and tested a small LEGO Mindstorm robot, that could move around on a coloured sheet of paper and react, depending which colour its camera would detect. The task: Find out what the specification of the robot is and find defects in its implementation.
    Very interesting: We had to come up with a hypothesis of how the robot was supposed to work as well as finding defects. I really enjoyed the way Bart & James gave feedback and asked the right questions.
  • “Reinventing software quality” – Gojko Adzic
    Breaking News: Unicorns Exist

    Gojko made the point that in agile testing we might (still) not focus on the right thing: To build the right software. Instead we concentrate on finding bugs and building the software in the right way. He illustrated this with one of his books:  He found a defect and then spent a while searching & listing more problems and ended up with a good number of them, definitely enough to make you worry about the quality of the book. However, the publisher explained that essentially all reviews were very positive! So: When people keep paying for your product or service, worrying about defects may not be that important.

Day 3

  • “Fast Feedback Teams” – Ola Ellnestam
    explained the importance and value of fast feedback. And he’s right: In many projects feedback could be gathered a lot earlier and be used to improve what features are built (as well as how they’re built). Other talks at least touched this topic as well. And while I wholeheartedly agree about this, I’m also a bit worried that we (as software creators) might forget about (or even ignore) slow changing aspects (for more about this see my previous post ‘Slow Feedback Cycles‘).
  • “Exceptions, Assumptions and Ambiguity: Finding the truth behind the Story” by David Evans
    UK Explained

    David explained how natural language is ambiguous, unclear and sometimes hard to understand. His examples included part of Jabberwocky (in several natural languages), music lyrics and last but not least programming. Also, I like his short introduction of the Cotswolds.

  • “It’s the Economy, Stupid! Learn the fundamentals about the one and only argument which will drag your management into agile practices” by Lucius Bobikiewicz
    Lucius talked about economic reasons of letting teams focus on 1 (one!) project/product, as opposed to working on multiple projects/products at the same time. It didn’t surprise me much, that letting teams finish one thing and then progress with the next is economically favourable. But  I was very surprised how much of a difference it made in the example he presented. These are the main advantages of the 1-project-only team Lucius presented:

    1. The time between the 1st project start and the first paid project (usually at the end of the project) is much smaller, meaning that significantly less money is needed to fund the upfront costs.
    2. Given the limited period of time in which a software can be sold, the product can likely be launched at the optimal time, whereas multi-product-teams may enter the market later and therefore always lag in selling.
    3. Since the overhead of context switching is minimised, the team can work on more projects/products per time unit. This advantage depends on the amount of time needed per context switch and the time the projects take .

    The economic point of view Lucius presented was a surprising and welcome detour form the other sessions which were much more focused about technology and/or ‘doing the right thing and doing it right’.

Every morning of the conference days 1-3 Lisa Crispin organised a Lean Coffee in one of the hotel bars. In small groups we discussed topics people were interested in. In order to cover some topics we limited the time for each topic to an initial 8 minutes and then added another 4 minutes depending on a quick vote. I find this is a very fun way to find and discuss topics. Thank you Lisa for organising them!

As a leaving thought: The chair persons were given (foldable) chairs as a present — ‘chairs for the chairs’. Super funny, if a tad bit unpractical to get home via train or airplane.

A big “Thank you!” to José, Uwe and Madelaine as well as the other organisers that made the Agile Testing Days such a great and enjoyable conference.

A Surprising Use of an Interface

Testing 2 Comments

This post relates to my short talk at the Agile Testing Days in Potsdam, Germany 2012.

Earlier this year I tweeted this (there’s another blog post about this as well):

Automating tests/checks, seems more valuable than having automated tests—like planning being more valuable than having the plan.

In my case the automating certainly led to other interesting results — and that’s the topic of this post.

Some context

Let’s assume we’re using Cucumber to automate tests for some application and follow the typical Cucumber setup: A folder features containing the feature files and inside of that another folder (step_definitions) containing, well, the step definitions. A support folder inside the feature directory keeps supporting files, such as environment configuration(s) etc.

Keeping it clean

The way I started to work is to start simple: At first I put everything inside the step definitions. Pretty soon though, I extract code chunks into methods of their own. And then move those method definitions into their own class (or module) – an in fact a file of their own.

That way, I end up with two more levels of abstraction below the feature descriptions and the step definitions: Classes (or modules) containing method definitions (see picture).

feature files, step defs, classes/modules, methods

Here I follow Uncle Bob Martin‘s advice of “Extract till you drop” (see his book ‘Clean Code‘, and episode 3 (Functions) of the video series at cleancoders.com): Extract methods until there’s nothing left to extract (and extract that code into the right place).

A first result: A high level interface to the system

One of the results of doing this, is a really high level and clean API to the system at hand. In addition to that I find that step definitions are much easier to understand, since they (mostly) contain a business level description of how the system is used and accompanying assertions to check whether the system behaves as it should.

Enter Pry

Pry is a command line tool for Ruby, like the interactive ruby shell irb that comes with Ruby. However, Pry is a lot more powerful and offers to display method definitions (even those defined in the C sources of internal Ruby classes) and a whole lot more. For a good introduction see the Pry Screencast by Josh Cheek on vimeo.

A trivial and contrived example

Let’s assume we’re trying to find stuff on the web (I’m on a Mac; there are hints about how to achieve this on Windows in the code comments):

require 'safariwatir' # Windows: require 'watir'

class GoogleSearch
 def initialize
 @browser = Watir::Safari.new # Win: Watir::Browser.new
 end

 def search_for search_term
 @browser.goto 'http://wellknown-search-engine.example.com/'
 @browser.text_field( :name, 'q' ).set search_term
 @browser.button( :id, 'gbqfb' ).click
 @browser.ol( :id, 'rso' ).links.map(&:text)
 end
end

Given that interface and Ppy, the creation of a console app borders on being trivial (yet, it took me a long while to get to this point):

require_relative 'google_search'
require 'pry'

GoogleSearch.new.pry

Think about this for a moment: Take Pry and your API and in 3 lines of code you get a console app for your system. The console can be used like this:

[10] stephan@nibur … #ruby google_search_console.rb
[1] pry(#<GoogleSearch>):1> search_for 'pry video josh'
=> ['Pry Screencast on Vimeo',
# ...

As mentioned above, you can even display the method definition (including lines of code, file name etc.)

[8] pry(<GoogleSearch>)> show-method search_for

From: /Users/stephan/…/google_search.rb @ line 8:
Number of lines: 6
Owner: GoogleSearch
Visibility: public

def search_for search_term
 @browser.goto 'http://google.de/'
 @browser.text_field( :name, 'q' ).set search_term
 @browser.button( :id, 'gbqfb' ).click
 @browser.ol( :id, 'rso' ).links.map(&:text)
end
[9] pry(#<GoogleSearch>)>…

Leaving thoughts

It seems that doing something can be more valuable that having the result. Notwithstanding, we still need to produce something of value for our business.

I came to realise that clean code matters more than ‘only’ providing a code base that’s easy to understand and change: Without the high level API, I might not even have noticed the possibility to create a ‘business level console’.

Having a console app like this allows to prepare starting points for exploratory testing (that might be ‘automation assisted exploring). Or short: I think a clean and high-level interface plus Pry is a cool tool for exploration.

A Week of “Rapid Testing Intensive”

Learning Testing 2 Comments

I signed up for the Rapid Testing Intensive event starting next week and am looking forward to a week of learning new things about software testing.

I’ll attend online and if anyone would like to talk about it or team up with me, leave me a note in the comments below or contact me via @S_2K on Twitter. In case you like to follow the tweets about the event, the Twitter hash tag is #RTI1.

Edit: Since that hash tag is already in use, make that #RTI2012.

© 2012-2026 Stephan Kämper — Mastodon

MENU

Navigation

Privacy Preference Center

Consent Management

Necessary

Advertising

Analytics

Other