Category: Ruby

Processing a Number of Image Files

This is, again, is mostly a note to my future self. 🙂

Occasionally, I need to process a bunch of image files in a batch. Most often it’s about resizing them, so they fit into a given format of at most xy pixels, or precisely into, say a square format. Here’s how I do it using ImageMagick and a bit of Ruby code:

Processing a single file

ImageMagick comes with convert (it’s linked to …/bin/magick on my machine), as command line tool for processing image files in a whole lot of ways. In order to resize a single file so that it ends up as a square image in a given number of pixels one can use the following on the command line (I use zsh on macOS):

convert input_file.jpg -resize 1200x1200 -background White -gravity center -extent 1200x1200 output_file.jpg

This command

  • reads ‘input_file.jpg’,
  • creates a new image with 1200⨉1200 pixels,
  • a white background,
  • resizes the input image as needed,
  • puts it in the center of the new image
  • and saves it as ‘output_file.jpg’.

Batch processing files

For this step, I use a Ruby script (of course this can also be done in zsh, bash, Python etc.):

images = Dir['file_pattern*.jpg']
images.each do |fn|
  `convert #{fn} -resize 1200x1200 -background White -gravity center -extent 1200x1200  #{fn.gsub(/\./, '_res.')}`
end

Tip: Be sure to use different names for the input and output file names.

An Odd Behaviour When Creating A Rails App

I’m currently looking into Rails 7 and found an odd behaviour when creating a new Rails app. Before creating a Rails app a few other things need to be in place, most notably Ruby, and _a_ database. My set up is this:

Ruby
3.1.0
Rails
7.0.2
rbenv
1.2.0
SQLite3
3.37.0
PostgreSQL
14.2

The simplest way to create a new Rails app is this:

rails new app_name

You can also specify various options, including, for example --css=tailwind, to use Tailwind for CSS. Since that’s what is used in a book I’m reading (‘Agile Web Development with Rails 7’ by Sam Ruby & Dave Thomas), I ran this:

rails new app_name --css=tailwind

Now I got an exception logged in the output, while also the exit code of the command was 0. Interesting. Here’s the command and the output (lots of output omitted for brevity):

rails new a_new_app --css=tailwind
      create
  …
      create  Gemfile
         run  git init from "."
Initialized empty Git repository in /Users/stephan/dev/tmp/a_new_app/.git/
      create  app
…
      create  config/master.key
      append  .gitignore
      create  config/boot.rb
…
      remove  config/initializers/new_framework_defaults_7_0.rb
         run  bundle install
Fetching gem metadata from https://rubygems.org/...........
Resolving dependencies....
Using rake 13.0.6
Using minitest 5.15.0

…
Using rails 7.0.2
Bundle complete! 16 Gemfile dependencies, 75 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
         run  bundle binstubs bundler
       rails  importmap:install
…
Add Tailwindcss include tags and container element in application layout
      insert  app/views/layouts/application.html.erb
      insert  app/views/layouts/application.html.erb
      insert  app/views/layouts/application.html.erb
…
Add default Procfile.dev
      create  Procfile.dev
Ensure foreman is installed
         run  gem install foreman from "."
Successfully installed foreman-0.87.2
/Users/stephan/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/yard-0.9.27/lib/yard/rubygems/hook.rb:88:in `require': cannot load such file -- yard (LoadError)
  from ….rbenv/…/ruby/…/yard-0.9.27/lib/yard/rubygems/hook.rb:88:in `load_yard'
  from ….rbenv/…/ruby/…/yard-0.9.27/lib/yard/rubygems/hook.rb:163:in `setup'
  from ….rbenv/…/ruby/…/yard-0.9.27/lib/yard/rubygems/hook.rb:152:in `generate'
  from ….rbenv/…/ruby/…/yard-0.9.27/lib/yard/rubygems/hook.rb:63:in `block in generation_hook'
  from ….rbenv/…/ruby/…/yard-0.9.27/lib/yard/rubygems/hook.rb:52:in `each'
  from ….rbenv/…/ruby/…/yard-0.9.27/lib/yard/rubygems/hook.rb:52:in `generation_hook'
  from ….rbenv/…/ruby/…/request_set.rb:311:in `block in install_hooks'
  from ….rbenv/…/ruby/…/request_set.rb:310:in `each'
  from ….rbenv/…/ruby/…/request_set.rb:310:in `install_hooks'
  from ….rbenv/…/ruby/…/request_set.rb:209:in `install'
  from ….rbenv/…/ruby/…/commands/install_command.rb:210:in `install_gem'
  from ….rbenv/…/ruby/…/commands/install_command.rb:226:in `block in install_gems'
  from ….rbenv/…/ruby/…/commands/install_command.rb:219:in `each'
  from ….rbenv/…/ruby/…/commands/install_command.rb:219:in `install_gems'
  from ….rbenv/…/ruby/…/commands/install_command.rb:167:in `execute'
  from ….rbenv/…/ruby/…/command.rb:323:in `invoke_with_build_args'
  from ….rbenv/…/ruby/…/command_manager.rb:180:in `process_args'
  from ….rbenv/…/ruby/…/command_manager.rb:149:in `run'
  from ….rbenv/…/ruby/…/gem_runner.rb:53:in `run'
  from ….rbenv/versions/3.1.0/bin/gem:13:in `<main>'
Add bin/dev to start foreman
      create  bin/dev
Compile initial Tailwind build
         run  rails tailwindcss:build from "."
+ /Users/stephan/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/tailwindcss-rails-2.0.5-x86_64-darwin/exe/x86_64-darwin/tailwindcss -i /Users/stephan/dev/tmp/a_new_app/app/assets/stylesheets/application.tailwind.css -o /Users/stephan/dev/tmp/a_new_app/app/assets/builds/tailwind.css -c /Users/stephan/dev/tmp/a_new_app/config/tailwind.config.js

Done in 228ms.

This is … interesting, since yard is found, when listing matching gems:

> gem list yard

*** LOCAL GEMS ***

yard (0.9.27)

However, the command that is running at this time is (apparently) executed in the context of Bundler. Since yard isn’t listed in the Gemfile, Bundler won’t find or use it:

bundle info yard
Could not find gem 'yard'.

After uninstalling yard, creating a new Rails app works fine and doesn’t log an exception.

While this solves the issue of getting the exception, it was still unclear why this happened in the first place.

Now, what is yard?

YARD is a documentation generation tool for the Ruby programming language.

https://github.com/lsegal/yard

Aha, looking into my .gemrc (which is used to configure Rubygems), I found these two lines (Yes, I like to have the documentation generated 🙂):

install: --rdoc --ri --document=yri
update: --rdoc --ri --document=yri

After removing the --document=yri, creating a new Rails app worked fine even with yard being installed.

I still don’t completely understand, why this is an issue when specifying a CSS processor, but not otherwise. This may be topic for another post.
If you have an idea about why this happens, please let me know.

The Different Ways of Pry and Irb

I ran into an interesting little problem when doing sone Ruby (and Rails) work today: I wanted to try something in pry and, inside that Rails project I issued the following command:

$ pry -r sqlite3
…/gems/pry-0.14.1/lib/pry/pry_class.rb:103:in `require': cannot load such file -- sqlite3 (LoadError)
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:103:in `block in load_requires'
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:102:in `each'
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:102:in `load_requires'
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:143:in `final_session_setup'
  from …/gems/pry-0.14.1/lib/pry/cli.rb:82:in `parse_options'
  from …/gems/pry-0.14.1/bin/pry:12:in `<top (required)>'
  from …/bin/pry:23:in `load'
  from …/bin/pry:23:in `<main>'
  from …/bin/ruby_executable_hooks:22:in `eval'
  from …/bin/ruby_executable_hooks:22:in `<main>'

Interesting! I have both gems, pry and sqlite3 installed. A similar attempt using irb worked fine:

$ irb -r sqlite3
3.0.1 :001 > SQLite3::VERSION
 => "1.4.2"
3.0.1 :002 >

Maybe I confused the environment by setting some environment variable or other? With a newly opened command shell I got this:

$ pry -r sqlite3
[1] pry(main)> SQLite3::VERSION
=> "1.4.2"
[2] pry(main)>

Remarkable. In that new shell the irb command behaved the same as before.

The only difference I could find was that the original call to pry happened in a Rails project – which is using Bundler to organise the Rubygems used in the project and resolve dependencies. As a small experiment I set up a minimal project that uses bundler and a rather short Gemfile:

source "https://rubygems.org"

gem  'limit_detectors'

The gem ‘limit_detectors’ is one I wrote and I know that is doesn’t depend on other gems. Issuing the pry command again … worked? Why? Some more investigation was needed…

Finally, I realised that another gem (Guard, to be specific), uses pry. Therefore my Rails project’s development environment indirectly depends on pry. It looks like pry recognises that it’s running as part of a Bundler project, even when it is not explicitly called using bundle exec pry …, which in turn causes it (pry) to ‘only’ recognise the gems that are also installed using Bundler. – And sqlite3 isn’t in this case, since I’m using PostgreSQL throughout.

Since irb comes with the Ruby installation and is not part of the bundled gems, it ignores the bundler context when called like this:

irb -r sqlite3

However, explicitly calling it in the bundler context in a project that doesn’t depend on ‘sqlite3’ (directly or indirectly) will cause an error message:

$ bundle exec irb -r sqlite3
…/rubies/ruby-3.0.1/lib/ruby/3.0.0/irb/init.rb:376: warning: LoadError: cannot load such file -- sqlite3
3.0.1 :001 >

Nice! This is essentially the same problem, I faced at the very beginning, when pry couldn’t find the sqlite3 gem.

A Local Rails Server in a Local Network

I’m currently working on a Rails app, that I want to have available in my local network on the default port that Ruby on Rails uses: 3000.

To do that, I added a line to the development config (config/environments/development.rb):

config.hosts << 'hostname.local'

Update: Apparently adding the line config.hosts.clear works as well. Details about this are explained in the Rails Guides.

I started using Foreman to achieve this, and installed it using:

gem install foreman

Note that, according to a Wiki Page on Github, it’s not recommended to put foreman into the Gemfile.

Then I created a Procfile that Foreman uses to start the web server (which currently is the only process I need it to start):

web: bundle exec rails s -b 0.0.0.0 -p 3000

The -b 0.0.0.0 binds the process to that IP address and -p 3000 instructs it to use port 3000.

With that set up foreman start can be used to start a Rails server in development which is available in the local network.

This way I can check the layout and functionality on a mobile phone or one of my iPads.

Collecting Lists In a Ruby Hash and the ‘<<=' operator

The other day, I needed to quickly analyse a data set that came in form of a large CSV file. I wanted to collect a particular column of that table and collect all entries categorised by a key in another column.

A simplified version of the table could look like this:

Key Value1 Interesting_Value Other_Value
foo1723.5X
bar211.75Q
foo4212.6B
baz2717.8F
bar4947.2K

I strived for something like this:

result = { 
  foo: [23.5, 12.6],
  bar: [1.75, 47.2],
  baz: [17.8],
 }

Iterating over the rows is easy, and getting to the columns is no problem either: The CSV gem is well documented and supports this easily.

A nice way to accumulate data is Enumerable#each_with_object. Since I wanted the result to be grouped by a key value, I’d pass a Hash as the initial argument.

Step 1: each_with_object({})

However, since I’ve planned to append values for changing keys, the default value needed to be an Array, not the default of nil.

Step 2: each_with_object(Hash.new([])

This, however, returns the same empty Array, when a key isn’t found, but I wanted a new empty Array:

Step 3: each_with_object(Hash.new { [] })

This executs the block every time a default values is needed (i.e. the given key isn’t yet in the Hash).

The next step is to append the value found in a row to the (potentially new and empty) Array for the given key.

I thought it would work this way:

data_table.each_with_object( Hash.new { [] }) do |row, acc|
  acc[row['Key']] << row['Interesting_Value'] 
end

But, no, the result of this code is an empty Hash! It needs to be the <<= operator to work, as shown in the snippet of a pry session:

[2] pry(main)> data_table = CSV.read 'table.csv', headers: true
=> #<CSV::Table mode:col_or_row row_count:6>
[3] pry(main)> data_table.each_with_object( Hash.new { [] }) do |row, acc|
[3] pry(main)*   acc[row['Key']] <<= row['Interesting_Value']
[3] pry(main)* end
=> {"foo"=>["23.5", "12.6"], "bar"=>["1.75", "47.2"], "baz"=>["17.8"]}

It seems to me, that the Hash lookup with the given default value [] returns an Array, and the append operator << does in fact append the passed object to that Array, but then the result of that does not end up as a (new) value fo the given Hash key. In contrast, the <<= operator does assign the result of the append operation.

Navigation

%d bloggers like this: