Category: shell

Colourful Code with Pygments

Note To Self Ruby shell No Comments

This is another entry in the ‘Note to Self’ category. 🙂 I’m sure I will need this information at some later point in time again.

The other day I wanted to have some code syntax-highlighted and be able to select the colour theme and well as use the highlighted listing in a number of ways.

Since Pygments is a library made for this task and it also provides a command line tool: pygmentize. It took me some time to use the tool the right way and produce the result I was looking for.

To show how I ended up using it, I’ll use fd as an example, a Ruby utility I wrote that dumps file contents as hex codes and utf-8 characters.

The command below is run inside a directory that contains a sub-folder ‘bin’, and inside that a (Ruby) file fd. To achieve this you can do the following (preferably when in a folder where you keep your cloned Git repositories):

> git clone git@github.com:s2k/fd.git
Cloning into 'fd'...
remote: Enumerating objects: 532, done.
remote: Counting objects: 100% (57/57), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 532 (delta 48), reused 53 (delta 47), pack-reused 475
Receiving objects: 100% (532/532), 105.97 KiB | 526.00 KiB/s, done.
Resolving deltas: 100% (261/261), done.
> cd fd

Here’s the command to create an HTML file using the given theme:

> pygmentize -l ruby -O full,style=monokai,linenos=1 -o fd.html -f html bin/fd

Here’s what the Parameters mean:

-l ruby
Set Ruby as the language to ge highlighted.
-O full,style=monokai,linenos=1
full generates output that includes everything to display the colourised code.
style=monokai sets the theme to ‘Monokai’.
linenos=1 displays the line numbers in the output.
-o fd.html
Set the output file name.
-f html
Set the output format to HTML.

When generating HTML, the full seems to be particularly important, as otherwise the HTML won’t contain the CSS used to colour the code.
The resulting highlighted code looks like this:

The colourised code of the file 'fd', including line numbers in front of each line.
The pygmentized source code

In case you’d like to experiment with pygmentize, here’s some zsh code that prints a sorted list of the styles it knows about:

> pygmentize -L styles | grep "* \(\w\+\):" | sed "s/* \([a-z_]*\):/\1/" | sort

Processing a Number of Image Files

Learning Note To Self Ruby shell No Comments

This is, again, is mostly a note to my future self. 🙂

Occasionally, I need to process a bunch of image files in a batch. Most often it’s about resizing them, so they fit into a given format of at most x ⨉ y pixels, or precisely into, say a square format. Here’s how I do it using ImageMagick and a bit of Ruby code:

Processing a single file

ImageMagick comes with convert (it’s linked to …/bin/magick on my machine), as command line tool for processing image files in a whole lot of ways. In order to resize a single file so that it ends up as a square image in a given number of pixels one can use the following on the command line (I use zsh on macOS):

convert input_file.jpg -resize 1200x1200 -background White -gravity center -extent 1200x1200 output_file.jpg

This command

  • reads ‘input_file.jpg’,
  • creates a new image with 1200⨉1200 pixels,
  • a white background,
  • resizes the input image as needed,
  • puts it in the center of the new image
  • and saves it as ‘output_file.jpg’.

Batch processing files

For this step, I use a Ruby script (of course this can also be done in zsh, bash, Python etc.):

images = Dir['file_pattern*.jpg']
images.each do |fn|
  `convert #{fn} -resize 1200x1200 -background White -gravity center -extent 1200x1200  #{fn.gsub(/\./, '_res.')}`
end

Tip: Be sure to use different names for the input and output file names.

The Different Ways of Pry and Irb

Learning Note To Self Ruby shell No Comments

I ran into an interesting little problem when doing sone Ruby (and Rails) work today: I wanted to try something in pry and, inside that Rails project I issued the following command:

$ pry -r sqlite3
…/gems/pry-0.14.1/lib/pry/pry_class.rb:103:in `require': cannot load such file -- sqlite3 (LoadError)
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:103:in `block in load_requires'
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:102:in `each'
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:102:in `load_requires'
  from …/gems/pry-0.14.1/lib/pry/pry_class.rb:143:in `final_session_setup'
  from …/gems/pry-0.14.1/lib/pry/cli.rb:82:in `parse_options'
  from …/gems/pry-0.14.1/bin/pry:12:in `<top (required)>'
  from …/bin/pry:23:in `load'
  from …/bin/pry:23:in `<main>'
  from …/bin/ruby_executable_hooks:22:in `eval'
  from …/bin/ruby_executable_hooks:22:in `<main>'

Interesting! I have both gems, pry and sqlite3 installed. A similar attempt using irb worked fine:

$ irb -r sqlite3
3.0.1 :001 > SQLite3::VERSION
 => "1.4.2"
3.0.1 :002 >

Maybe I confused the environment by setting some environment variable or other? With a newly opened command shell I got this:

$ pry -r sqlite3
[1] pry(main)> SQLite3::VERSION
=> "1.4.2"
[2] pry(main)>

Remarkable. In that new shell the irb command behaved the same as before.

The only difference I could find was that the original call to pry happened in a Rails project – which is using Bundler to organise the Rubygems used in the project and resolve dependencies. As a small experiment I set up a minimal project that uses bundler and a rather short Gemfile:

source "https://rubygems.org"

gem  'limit_detectors'

The gem ‘limit_detectors’ is one I wrote and I know that is doesn’t depend on other gems. Issuing the pry command again … worked? Why? Some more investigation was needed…

Finally, I realised that another gem (Guard, to be specific), uses pry. Therefore my Rails project’s development environment indirectly depends on pry. It looks like pry recognises that it’s running as part of a Bundler project, even when it is not explicitly called using bundle exec pry …, which in turn causes it (pry) to ‘only’ recognise the gems that are also installed using Bundler. – And sqlite3 isn’t in this case, since I’m using PostgreSQL throughout.

Since irb comes with the Ruby installation and is not part of the bundled gems, it ignores the bundler context when called like this:

irb -r sqlite3

However, explicitly calling it in the bundler context in a project that doesn’t depend on ‘sqlite3’ (directly or indirectly) will cause an error message:

$ bundle exec irb -r sqlite3
…/rubies/ruby-3.0.1/lib/ruby/3.0.0/irb/init.rb:376: warning: LoadError: cannot load such file -- sqlite3
3.0.1 :001 >

Nice! This is essentially the same problem, I faced at the very beginning, when pry couldn’t find the sqlite3 gem.

Moving Left and Right in zsh (in macOS)

shell Uncategorized No Comments

I use the command line a lot on my Macs, in particular, I use iTerm2.

When moving left and right in a command the shortcuts I knew about so far were:

Key (combination)Moves Cursor…
Right Arrow ‘→’One character right
Left Arrow ‘←’One character left
CTRL + aBeginning of the line
CTRL+ eEnd of the line

After adding the following lines to ~/.zshrc, I now can also move left and right per word using ALT-← and ALT-→ respectively: 

bindkey "[D" backward-word # ALT-left-arrow  ⌥ + ←
bindkey "[C" forward-word  # ALT-right-arrow ⌥ + →

Especially when moving around in long command lines, this is really convenient & saves a lot of time.

Do you have keyboard shortcuts that you use regularly on the command line (or elsewhere)? I’d love to hear about them!

Passing Multiple Arguments To One Rake Task

Note To Self shell No Comments

Occasionally, I want Rake tasks to take multiple arguments. If the number of these arguments are know beforehand, Rake provides a nice way to express that in the Rakefile (see the Rake documentation for more details).

However, sometimes I like to pass a varying number of arguments the a task, which shall all be processed in the same way. Maybe its user IDs that need to be removed from a database, or a given list of piles to be processed.

Today I found a way to do this, by using Rake::TaskArguments#extras . The method extras returns a list of all the extra arguments passed to a Rake task, that are not named in the task definition. If no argument name(s) are declared in the task definition, then all arguments will be ‘extra arguments’.

An example Rakefile is this:

task :list do |t, args|
  puts "In task '#{t.name}"
  puts 'Passed arguments:'
  puts args.extras.map { |extra| "  * #{extra}" }
end

Calling this task from the command line and its output (in zsh) looks like this:

$ rake list\[one,2,3.141]
In task 'list
Passed arguments:
  * one
  * 2
  * 3.141

Note, the \ before the opening bracket ‘[‘. In zsh, this is needed to escape the ‘[‘ (pairs of ‘[]’ have a special meaning in zsh). Also note, that there are no spaces between the arguments, just the comma.

If you prefer to add a space after the comma, there’s a way to achieve this (again in zsh): Drop the ‘\’ and instead put the task name and the arguments in quotes:

$ rake "list[one, 2, 3.141]"
In task 'list
Passed arguments:
  * one
  * 2
  * 3.141

$ rake 'list[one, 2, 42]'
In task 'list
Passed arguments:
  * one
  * 2
  * 42

In this example it doesn’t affect the output whether single or double quotes are used. Quoting in shells is … complicated, see for example: http://zsh.sourceforge.net/Guide/zshguide05.html#l112.

Did you find this helpful? Or do you know a better way to pass multiple arguments to one Rake task? Please let me know.


© 2012-2023 Stephan Kämper — Mastodon

MENU

Navigation

%d bloggers like this: