Recently I finished up some shell functions that help me with some common git and testing workflows. They are up on Github, but I wanted to call them out since they might be helpful to others and just making something open source does not mean that it is discoverable. I think the philosophies are pretty solid even if you use different tools. You could use similar functions if you are using Bash or ZSH.
Overview
The general problem that I am trying to solve is that different tools like RSpec, git, or RuboCop produce output in a certain format, and I often want to do things with that output. For example, I might want to re-run the RSpec tests that just failed so I can verify fixes more easily.1 However, RSpec outputs in a certain format that is not easily consumable. For example:
...
Failed examples:
rspec ./code/fail_spec.rb:2 # something failed
rspec ./code/fail_spec.rb:8 # yet another failure
...
If I want to re-run these two tests, I could copy the two lines and paste into my terminal. This would have a couple of downsides though. One is that I would need to spin up one RSpec process for each test that failed. This is time-prohibitive if the project loads the Rails environment. It also prevents me from getting a reliable list of the tests that failed so I can repeat this process. Last, I’d like to ideally use a faster test runner like Zeus or Spring. So my real goal is to re-run the failing tests as quickly as possible.
One approach that I took for a few years was to copy the output, paste it into an editor (Vim), and then do some macros or other commands to munge it into the format that I want. However, this is time-consuming and potentially error-prone. It is also usually wasteful since I need to do it each time that I want to transform the output to a particular format, and often I don’t have the editor macros saved. It can be nice to have the list of tests to retry in an external editor to be able to check them off, but I prefer to not need the intermediate step.
Solution
The specific solution I made to solve this problem was to create a shell function that I called respec:
# copy the rspec failures, and this will rerun them as one command
function respec() {
pbpaste | \
cut -d ' ' -f 2 | \
sort | \
xargs rspec
}
First, I manually copy the tests that I want to run again. Using iTerm2, it is as simple as selecting the failure summary text since selections are automatically copied to the clipboard. pbpaste then echos the contents of the system clipboard. From there, we want a list of the tests that failed so we can run them again. The format of failing tests are:
rspec ./code/fail_spec.rb:2 # something failed
...
To be able to run any test that failed, we want:
./code/fail_spec.rb:2
One approach to solving this problem is to split the line by spaces and take the second item. We could do this with the cut command (saying the delimiter is space and we want the second field) or with awk (awk '{print $2}').2 From there, we just sort the files since they might have not been sorted depending on our test strategy, and we pipe the resultant tests to rspec using xargs.
Expanding on this
Something else we might want to do with the RSpec output is to edit the files that have failing tests. It could be that the tests need to be updated, or that it gives us faster access to the application files if we are using something like vim-projectionist. We don’t want to get all of the test failures, just the files that failed. So we could create a similar function that would edit the files that have failing tests:
# after copying the rspec failures, this will edit the files that had failures
function espec() {
pbpaste | \
cut -d ' ' -f 2 | \
cut -d ':' -f 1 | \
sort | \
uniq | \
xargs $EDITOR
Extracting common functions
There is some duplication in the current code. We’re always pulling the test failures from the clipboard, then getting the actual file and line numbers, and then sorting the tests. To DRY up the functions, we can create an intermediate function to take care of getting the test failures:
# returns the list of filenames from rspec output format
function rspec_paste() {
pbpaste | cut -d ' ' -f 2 | sort
}
Then we can call this function inside of our existing functions:
function respec() {
rspec_paste | xargs rspec
}
function espec() {
rspec_paste | cut -d ':' -f 1 | uniq | xargs $EDITOR
}
Much simpler and easier to read.
Running commands on modified files
Another common workflow that I have is to run any tests that have changed in source control. Back in the day I would have done a git status, copied the test files that I want to retest, and added them after an rspec command on the command-line. Again, this is suboptimal.
Since I’ve shown function extraction above, let’s skip to the chase:
# return all files that are changed but not deleted in git
# needs to handle
# R spec/1_spec.rb -> features/2_spec.rb
function modified_files() {
git status --porcelain | \
grep -v -e '^[ ]*D' | \
awk 'match($1, ""){print $2}' | \
sort | \
uniq
}
This function will print out any files that were modified but not deleted. This is relevant because if we deleted a test file, we don’t want to try to run it since RSpec will immediately fail if test files are not recognized. For more information on the output format of git-porcelain, check out the git docs.
On its own, modified_files is useless. But now we can make functions that work with it. From there, we can run RSpec on any spec files that were changed:
function gspec() {
modified_files | \
grep -e "_spec\.rb$" | \
xargs best_rspec
}
So basically we want to take any modified files, remove any that aren’t spec files, and then we run them through the best_rspec script. What’s that, you ask?
best_rspec
This is a small script that I put on my path to figure out what the fastest RSpec binary is and to use that to run tests. I’ve also aliased it to r, for faster typing of the command and to avoid needing to think about what environment I am running the tests in.
#!/bin/sh
if [ -f bin/rspec ]; then
# use spring rspec if it is around
./bin/rspec $@
elif [ -S .zeus.sock ]; then
echo "Running tests with Zeus"
zeus test $@
else
echo 'Running with naked `rspec`'
rspec $@
fi
If we have Spring configured for the project, use that since it will be no slower than using the normal RSpec binary. If we have a Zeus socket open, then use that. Otherwise, just use the default rspec command.
I updated respec and gspec and any other commands that run RSpec tests use this to give the same behavior for free. Considering which binary to use is something I previously would have spent a brain cycle or two on that now just happens automatically regardless of what project I am working on. I consider it a win.
Putting it all together
Let’s discuss a common flow that uses these different operations. First, I check out a new branch and want to add a feature. I write some tests and watch them fail, then make them pass. I can make sure that all of the test files that I’ve added or modified pass before committing them with gspec.3 When I think I’m done with my feature, I run the test suite and there are some test failures. After copying the test failure lines which are at the bottom of the RSpec output, I can run respec to quickly confirm that they aren’t intermittent tests. Then I can run espec to open the files up and fix the issues. I run respec each time I think I’ve fixed some of the tests, and the failure output will hopefully reduce each time so I am running fewer tests the next time. Finally, I run gspec again to make sure that the tests are indeed passing. Each time, we are using the Spring test runner to avoid needing to spend seconds reloading the Rails environment.
Food for thought
Consider what workflows you have after seeing a program’s output and how you might create small, composable, functions or programs to help automate them. In addition to RSpec or other testing tools, you might consider the output of code linters or static analyzers, code deployment tools, and so forth.
My configuration is up on Github, check it out for some additional helpers that I use to re-run cucumber tests and to work with RuboCop output for re-running that tool.
-
I realize that there are some plugins that will re-run the last failed tests, but I wanted a more general solution to this problem. ↩
-
If there are spaces in your filenames, I’m not sure that either approach will work well. Also, you should probably reconsider your naming conventions! :) ↩
-
I’d like to extend the
gspeccommand to also run tests for application files that changed. For example, if git believesapp/controllers/foo_controller.rbhas been modified, I’d like to runspec/controllers/foo_controller_spec.rbeven if it wasn’t changed. This should be fairly straightforward to do since Rails has a consistent directory format. This change would likely result in less manual effort on my part, since I often want to run the tests for any application files that changed. ↩
