Word of The Year 2014

In the same way I did last year, I have picked a ‘word of the year’ (see my posts from late 2012 ‘Word of The Year 2013‘ and December 2013, ‘A Year of Explore‘).

For 2014, I go for ‘Create’:

Word of The Year 2014

Do you pick a word of the year for 2014? Which one?

Testing Rake Tasks

I’ve worked in several teams that used (and still use) Rake, a build tool written in Ruby, to perform various tasks—not just building software (or other artefacts). In this article I list some ways in which rake tasks can be tested, but I do not recommend all of them for all contexts.

Ways to test rake tasks

1. You don’t

Let’s face it, in some cases rake tasks are hardly tested at all, maybe just tried out a little bit. And in fact in some cases that’s just fine: When the tasks are particularly simple and/or they’re short lived anyway, a whole set of tests may be just too much. — But keep in mind that in this case using rake may also be going too far already.

For example, take to following code.

desc 'Write list of files in (sub) folders to "filelist.txt"'
task :dir_to_txt do
  File.open('filelist.txt', 'w') do | f |
    f.puts Dir['**/*']
  end
end

For most people this code is so simple, that they likely wouldn’t bother testing it (and yet… I’m not claiming it’s bug-free).

2. Non-Automated Testing

In many cases, rake tasks are ‘tested’ by manually running them and then examining the result. Depending on what you’re doing this can be fine. However, rake tasks process various input and they can be configured, too. For example, you may set an environment (such as ‘test’, ‘development, ‘production’) or a particular server to run them against. Sometimes you can cover the important combinations with manual testing. But then, maybe you’re missing a particular combination that causes a failure.

Also, don’t forget to document what, how and why you tested it the way you did. Someday someone will have to change the rake task, and then it will be important to know what behaviour is expected not to change.

2. Cucumber

You’re running the rake tasks in order to have a (more or less) permanent effect. You can test this using Cucumber, and the format could be like this:

Feature: Running task name_space:task_name
Scenario: Use default parameters
  Given the task is configured with the following setup
    | parameter_1 | 'some value' |
    | parameter_2 | a_number |
  When rake name_space:task_name is run
  Then some effect should be achieved

With Cucumber you can provide configuration data to run a single setup like the one above, but Cucumber also offers ‘Scenario Outlines’ which provide an even more parameterised way to run the checks. Combined with e.g. ‘all pairs testing‘, this can give a very reasonable test coverage (with respect to used values & combinations of these values).

3. MiniTest or RSpec + Refactored Rake Tasks

A rake task is just Ruby code. Therefore it can be unit tested just like any other Ruby code. (It can even be developed in a test-first way. Just saying.) If you’re also refactoring tasks into modules, classes and methods, it’s even easier to test and an additional benefit may be the fact that you’re creating an interface to the tasks that may be useful in a context other then running rake tasks.

4. ‘Redefine’ Rake Methods

Rake tasks are called using:

rake a_taskname

Rake searches for a file called ‘Rakefile‘ (or ‘rakefile’, with or without the ‘.rb’ suffix), and then processes it, in order to find a task with a matching name. Since the Rakefile itself doesn’t require ‘rake’ (using the command rake loads Rake’s context), you can (re-) define the methods rake provides (in a separate file), require that file, require your Rakefile and then use a test framework of your choice on those methods. Let’s look at a Rakefile with two simple tasks:

require 'fileutils'
namespace :doc do
  desc "Generate RDoc for the Rakefile"
  task :self do
    `rdoc ./Rakefile`
  end

  desc "Remove doc folder and its content"
  task :remove do
      FileUtils.rm_rf('doc')
    end
  end
end

While you could (re-) define rake’s methods ‘namespace’, ‘desc’, ‘task’ etc. and the call those methods to test them, this would only solve part of the problem: The tasks still change the outside world (in this case generate RDoc in a folder in one case and remove that fold in the other case). In at least some cases that’s undesirable, e.g. when there’s a task that’s supposed to switch your online shop into maintenance mode, just displaying a static page for all incoming requests. At least in those cases it’s preferable to check for the right behaviour instead of depending on checking a state change in the (real) world.

That’s possible in Ruby: You can redefine existing methods such as FileUtils.rm_rf and even ` in the example above. — Yes ` (the back tick) is a method in Ruby and you can redefine it. For more complicated (or complex) rake tasks, it’s a lot of work to find all the methods that you’d need to redefine. While it’s possible, I wouldn’t recommend it, since it seems to me that it’s also error prone and might well introduce more trouble (read: bugs) than anyone would like in their tests.

Some General Remarks

Since a rake task is code, I think it should be treated as carefully as any other code we write and it should be tested (and live in the same version control system used for other parts of the code base). Especially in case of tasks that control an online shop (e.g. switch to maintenance mode), deploy new software to production etc. it is in fact production code.

I personally prefer the rake tasks I’m working with to be well written, refactored and of course tested code. So I strive to extract single methods, modules and classes from rake tasks and test these in isolation.

Not matter what kind of automation you’re using to test your rake tasks, as soon as you’re starting to use a rake task to run these tests, e.g. as part of your regression tests on a Continuous Integration system like Jenkins, things may get tangled up: How would you test those rake tasks?

Other Articles on Testing Rake Tasks

Have A Good Time And A Great Year 2014

Have a peaceful and great time everybody, enjoy & relax.

I recently found this Santa Claus at the beach in Sankt-Peter Ording while taking my dog for a long walk.

Santa, Sand and Shell

Santa, Sand and Shell

Merry Christmas!

A Year of ‘Explore’

At the end of last year, I decided I would not make any new year’s resolutions in favour of selecting a ‘word of the year’, see the blog post from early 2013. I picked the word ‘explore for 2013 and since the year is (nearly) over, it’s time to look back and see how it worked out.

The Black Box Software Testing Courses

At the time of this writing there are four BBST (Black Box Software Testing) courses the Association for Software Testing offers:

  1. Foundations
  2. Test Design
  3. Bug Advocacy
  4. Instructors

I took the first three of them in the first half of 2013, and the last one in October. And while I do recommend taking these courses, I have to say that I needed a good amount of time to work through all the material, labs & exercises. Especially the ‘Test Design’ course offered a phenomenal amount of material.

I totally recommend these courses to everyone working in software testing and software development in general.

Conferences

For a while now, I try to go to two conferences each year, a programmer conference as well as a tester conference. I used to recommend this to my fellow ‘programming tester colleagues’, but now I’ve also started to recommend it to the ‘testing programmers’ as well. While I focus on software testing, I find it useful to know a bit about programming, too.

I hoped to be able to go to the EuRuKo (European Ruby Conference) in Athens, Greece in summer. In the end it didn’t work out as planned and I couldn’t go. However, I gave my ticket away and received two post cards in return. Thank you, you know who you are.

In September I attended the BARUCO, the ‘Barcelona Ruby Conference’ in Spain, and in October I went to the Agile Testing Days in Potsdam, Germany. At both conferences I gave short presentations about testing and the two values of software. Furthermore, at the Agile Testing Days I had the pleasure to assist Lisa Crispin and Janet Gregory at the  beginning of their keynote presentation:

A remark: The BBST Instructors course and the Agile Testing Days overlapped a bit. If you can, I suggest to avoid a commitment like that. Although it did work for me (in the end), this is a way of exploring, I’ll avoid in the future.

Projects

Early this year, I wanted to try some rather short software testing projects and joined uTest (now renamed to be Applause), where I worked on several apps for mobile devices as well as OS X. Given my background with longer running projects, having just a few days for testing was a refreshing experience. I also joined a project using Calabash to automate testing (well, checking actually) on Android devices.

In late June I joined a new longer running project using a whole bunch of technologies I’ve heard about before, but which were new to me – Another way to explore! So now I work in a project using MongoDB, VarnishPuppet and Vagrant. All of them are really interesting technologies, and the team doesn’t stop there: Every now and then we take a day to, well, explore new ways that may improve our work.

Summary

It’s been a very exciting and busy year and I am convinced that picking a ‘word of the year’ instead of making new year’s resolutions made a big difference. Instead of a plan, I felt I had some guidance that helped in deciding what to do (and what not to do in some cases). I will pick a word of the year for 2014 as well and if you also pick one, or if you already had one for 2013, why not write a short comment whether (or not) it helped you and in which way?

%d bloggers like this: