How to call shell commands from Ruby?

Query:

How do I call shell commands from inside of a Ruby program? How do I then get output from these commands back into Ruby?

How to call shell commands from Ruby?

This explanation is based on a commented Ruby script from a friend of mine. If you want to improve the script, feel free to update it at the link.

First, note that when Ruby calls out to a shell, it typically calls /bin/shnot Bash. Some Bash syntax is not supported by /bin/sh on all systems.

Here are ways to execute a shell script:

cmd = "echo 'hi'" # Sample string that can be used

1.) Kernel#` , commonly called backticks – `cmd`This is like many other languages, including Bash, PHP, and Perl.Returns the result (i.e. standard output) of the shell command.

value = `echo 'hi'`
value = `#{cmd}`

2.) Built-in syntax, %x( cmd )

Following the x character is a delimiter, which can be any character. If the delimiter is one of the characters ([{, or <, the literal consists of the characters up to the matching closing delimiter, taking account of nested delimiter pairs. For all other delimiters, the literal comprises the characters up to the next occurrence of the delimiter character. String interpolation #{ ... } is allowed.

Returns the result (i.e. standard output) of the shell command, just like the backticks.

value = %x( echo 'hi' )
value = %x[ #{cmd} ]

3.) Kernel#system

Executes the given command in a subshell.

Returns true if the command was found and run successfully, false otherwise.

4.) Kernel#exec

Replaces the current process by running the given external command.

Returns none, the current process is replaced and never continues.

exec( "echo 'hi'" )
exec( cmd ) # Note: this will never be reached because of the line above

Here’s some extra advice: $?, which is the same as $CHILD_STATUS, accesses the status of the last system executed command if you use the backticks, system() or %x{}. You can then access the exitstatus and pid properties:

$?.exitstatus

Run shell commands in Ruby:

The way I like to do this is using the %x literal, which makes it easy (and readable!) to use quotes in a command, like so:

directorylist = %x[find . -name '*test.rb' | sort]

Which, in this case, will populate the file list with all test files under the current directory, which you can process as expected:

directorylist.each do |filename|
  filename.chomp!
  # work with file
end

How to run shell commands in Ruby?

Often times we want to interact with the operating system or run shell commands from within Ruby. Ruby provides a number of ways for us to perform this task.

Exec

Kernel#exec (or simply exec) replaces the current process by running the given command For example:


  $ irb
  >> exec 'echo "hello $HOSTNAME"'
  hello nate.local
  $

Notice how exec replaces the irb process is with the echo command which then exits. Because the Ruby effectively ends this method has only limited use. The major drawback is that you have no knowledge of the success or failure of the command from your Ruby script.

System

The system command operates similarly but the system command runs in a subshell instead of replacing the current process. system gives us a little more information than exec in that it returns true if the command ran successfully and false otherwise.


  $ irb             
  >> system 'echo "hello $HOSTNAME"'
  hello nate.local
  => true
  >> system 'false' 
  => false
  >> puts $?
  256
  => nil
  >> 

system sets the global variable $? to the exit status of the process. Notice that we have the exit status of the false command (which always exits with a non-zero code). Checking the exit code gives us the opportunity to raise an exception or retry our command.Note for Newbies: Unix commands typically exit with a status of 0 on success and non-zero otherwise.

System is great if all we want to know is “Was my command successful or not?” However, often times we want to capture the output of the command and then use that value in our program.

Backticks (`)

Backticks (also called “backquotes”) runs the command in a subshell and returns the standard output from that command.


  $ irb
  >> today = `date`
  => "Mon Mar 12 18:15:35 PDT 2007\n" 
  >> $?
  => #<Process::Status: pid=25827,exited(0)>
  >> $?.to_i
  => 0

This is probably the most commonly used and widely known method to run commands in a subshell. As you can see, this is very useful in that it returns the output of the command and then we can use it like any other string.

Notice that $? is not simply an integer of the return status but actually a Process::Status object. We have not only the exit status but also the process id. Process::Status#to_i gives us the exit status as an integer (and #to_s gives us the exit status as a string).

One consequence of using backticks is that we only get the standard output (stdout) of this command but we do not get the standard error (stderr). In this example we run a Perl script which outputs a string to stderr.


  $ irb
  >> warning = `perl -e "warn 'dust in the wind'"`
  dust in the wind at -e line 1.
  => "" 
  >> puts warning

  => nil

Notice that the variable warning doesn’t get set! When we warn in Perl this is output on stderr which is not captured by backticks.

IO#popen

IO#popen is another way to run a command in a subprocess. popen gives you a bit more control in that the subprocess standard input and standard output are both connected to the IO object.


  $ irb
  >> IO.popen("date") { |f| puts f.gets }
  Mon Mar 12 18:58:56 PDT 2007
  => nil

While IO#popen is nice, I typically use Open3#popen3 when I need this level of granularity.

Open3#popen3

The Ruby standard library includes the class Open3. It’s easy to use and returns stdin, stdout and stderr. In this example, lets use the interactive command dc. dc is reverse-polish calculator that reads from stdin. In this example we will push two numbers and an operator onto the stack. Then we use p to print out the result of the operator operating on the two numbers. Below we push on 5, 10 and + and get a response of 15\n to stdout.


  $ irb
  >> stdin, stdout, stderr = Open3.popen3('dc') 
  => [#<IO:0x6e5474>, #<IO:0x6e5438>, #<IO:0x6e53d4>]
  >> stdin.puts(5)
  => nil
  >> stdin.puts(10)
  => nil
  >> stdin.puts("+")
  => nil
  >> stdin.puts("p")
  => nil
  >> stdout.gets
  => "15\n" 

Notice that with this command we not only read the output of the command but we also write to the stdin of the command. This allows us a great deal of flexibility in that we can interact with the command if needed.

popen3 will also give us the stderr if we need it.


  # (irb continued...)
  >> stdin.puts("asdfasdfasdfasdf")
  => nil
  >> stderr.gets
  => "dc: stack empty\n" 

However, there is a shortcoming with popen3 in ruby 1.8.5 in that it doesn’t return the proper exit status in $?.


  $ irb
  >> require "open3" 
  => true
  >> stdin, stdout, stderr = Open3.popen3('false')
  => [#<IO:0x6f39c0>, #<IO:0x6f3984>, #<IO:0x6f3920>]
  >> $?
  => #<Process::Status: pid=26285,exited(0)>
  >> $?.to_i
  => 0

0? false is supposed to return a non-zero exit status! It is this shortcoming that brings us to Open4.

Open4#popen4

Open4#popen4 is a Ruby Gem put together by Ara Howard. It operates similarly to open3 except that we can get the exit status from the program. popen4 returns a process id for the subshell and we can get the exit status from that waiting on that process. (You will need to do a gem instal open4 to use this.)


  $ irb
  >> require "open4" 
  => true
  >> pid, stdin, stdout, stderr = Open4::popen4 "false" 
  => [26327, #<IO:0x6dff24>, #<IO:0x6dfee8>, #<IO:0x6dfe84>]
  >> $?
  => nil
  >> pid
  => 26327
  >> ignored, status = Process::waitpid2 pid
  => [26327, #<Process::Status: pid=26327,exited(1)>]
  >> status.to_i
  => 256

A nice feature is that you can call popen4 as a block and it will automatically wait for the return status.


  $ irb
  >> require "open4" 
  => true
  >> status = Open4::popen4("false") do |pid, stdin, stdout, stderr|
  ?>            puts "PID #{pid}" 
  >>          end
  PID 26598
  => #<Process::Status: pid=26598,exited(1)>
  >> puts status
  256
  => nil

Some things to think about when choosing between these mechanisms are:

  1. Do you just want stdout or do you need stderr as well? Or even separated out?
  2. How big is your output? Do you want to hold the entire result in memory?
  3. Do you want to read some of your output while the subprocess is still running?
  4. Do you need result codes?
  5. Do you need a Ruby object that represents the process and lets you kill it on demand?

You may need anything from simple backticks (“), system(), and IO.popen to full-blown Kernel.fork/Kernel.exec with IO.pipe and IO.select.

You may also want to throw timeouts into the mix if a sub-process takes too long to execute.

Unfortunately, it very much depends.

If you really need Bash, per the note in the “best” answer.

First, note that when Ruby calls out to a shell, it typically calls /bin/shnot Bash. Some Bash syntax is not supported by /bin/sh on all systems.

If you need to use Bash, insert bash -c "your Bash-only command" inside of your desired calling method:

quick_output = system("ls -la")
quick_bash = system("bash -c 'ls -la'")

To test:

system("echo $SHELL")
system('bash -c "echo $SHELL"')

Or if you are running an existing script file like

script_output = system("./my_script.sh")

Ruby should honor the shebang, but you could always use

system("bash ./my_script.sh")

to make sure, though there may be a slight overhead from /bin/sh running /bin/bash, you probably won’t notice.

My favorite is Open3

  require "open3"

  Open3.popen3('nroff -man') { |stdin, stdout, stderr| ... }

Hope you learned something from this post.

Follow Programming Articles for more!

About ᴾᴿᴼᵍʳᵃᵐᵐᵉʳ

Linux and Python enthusiast, in love with open source since 2014, Writer at programming-articles.com, India.

View all posts by ᴾᴿᴼᵍʳᵃᵐᵐᵉʳ →