What's faster? Zeus with Parallel
Spring, in the context of Rails 4.1,
RSpec 3, Capybara 2.4, and PhantomJs?
The bottom line is that both work almost equivalently as fast, and the
biggest difference for me concerned compatibility with the
parallel_tests gem. Zeus works fine with Parallel Tests,
although it makes little difference overall with or without Zeus. Spring
doesn't work with Parallel Tests, but you can work around this issue. So
stick with Zeus if it works for you.
And regardless of using Spring or Zeus, the shell scripts provided below
pgk are essential for quickly listing or killing
Zeus, Spring, Rails, or Phantomjs processes!
It's also worth noting that biggest advantage of using the Zeus or
Spring pre-loaders is to save the Rails startup time. On my machine,
this is about 3 to 5 seconds. That matters a lot if the test I'm
focusing upon only takes a second or two, such as when doing TDD.
However, when running a whole test suite taking minutes, 3-5 seconds can
get swallowed up by other things, such as rspec-retry, which retries
failing capybara tests.
I've written about my integration testing setup: Capybara, PhantomJs,
Poltergeist, and Rspec
For a while, I've been eager to upgrade to Rails 4.1 and RSpec 3.
Finally, in August, 2014, the gem ecosystem allowed this to happen!
I've got a related article on my tips for upgrading to Rails 4.1 and
Once I had upgraded nearly every gem in my client's large Rails project
to the latest gem versions, I was pleasantly surprised that I could once
again get Zeus, Guard, RSpec, Capybara, Poltergeist, Parallel Tests,
etc. to all play nicely together.
Always curious as to the value of the latest defaults in Rails, I
decided to try out Spring. Both Spring and Zeus preload Rails so that
you don't have to pay the same start up cost for evry test run. Here's a
RailsCast on the topic: #412 Fast Rails
The end results is that both Zeus and Spring give great results and are
very similar in many ways. The biggest difference for me is that only
Zeus (and not Spring) works with Parallel Tests. Interestingly, I got
very similar results when using Parallel Tests with our without Zeus. It
turns out that it is possible to run Parallel Tests with Spring
installed so long as you disable it by setting the environment variable
DISABLE_SPRING=TRUE parallel_rspec -n 6 spec.
The bottom line for me is that I don't have any good reason to move away
from Zeus to Spring, and the fact that Spring is part of stock Rails is
not a sufficient reason for me. That being said, on another project
which is smaller, I'm not motivated to switch from Spring to Zeus.
Note in below commands, one must insert
zeus in the command to be
using zeus. If using Spring, be sure that you're using the Spring
modifed binstub scripts in your bin directory by having your path
appropriately set or using
The times shown below are from both sample runs of a single directory of
non-integration specs and from the full test suite of 914 tests, many of
which are Capybara tests, on a 2012, Retina, SSD, 16 GB, MacBook Pro
while running Emacs, RubyMine, Chrome, etc. Times were gathered by
running commands prefixed with the
time command. Running
seems a bit slower than using spring. However, when running the
integration tests, my test execution time was always variable depending
on the number of Capybara timeouts and retries.
|command||zeus loader||spring loader||no loader|
|parallel_rspec -n 6 spec||2:28.7||n/a||2:28.0|
Zeus and Spring vs. plain RSpec
Here's some advantages and disadvantages of using either either Zeus or
Spring compared to plain RSpec.
- Both save time for running basic commands like rspec, rake, rails,
etc. The performance of both is very similar.
- Both can be extremely confusing when they fail to update
automatically. This tends to happen after updating gems or running
database migrations. You end up yak shaving when you don't see your
changes taking effect! I.e., put in some print statements, and then
you don't see them shown when they should. Arghhhh!
- Rspec-retry seems essential
in dealing with random Capybabara failures with either Zeus or
Spring. I often see less of these errors when I don't use
Zeus/Spring nor parallel_tests.
Zeus vs. Spring
- Zeus works with the
gem. This more than
halves my time for running my entire test suite. However, when
writing this article, I found that it made little difference, at
least when slowed down by sporadically failing capybara tests that
are retried. That being said, I'm certain that Parallel Tests with
Zeus is faster or at worse the same as without Zeus.
- You need to start up separate shell process, running
An advantage of this is that if there's a problem starting up, the
output in the Zeus console window is fairly clear.
- You run the command "zeus rake" rather than just "rake".
Consequently, I made some shell aliases (see below).
- Zeus only uses the environment from when Zeus was started and
ignores any environment variables when commands are run.
Spring vs. Zeus
- Spring is a default part of
Rails, so you know it's well supported, and bugs will be fixed fast.
- Slightly simpler to install and use than Zeus.
- Spring lacks support for parallel_tests. See this Github
issue: incompatible with spring
You can, however run parallel_tests so long as run the
command like this:
time DISABLE_SPRING=TRUE parallel_rspec -n 6
spec. I.e., you need to set
DISABLE_SPRING so that
parallel_rspec does not use Spring.
- Spring is a bit opaque in terms of errors given there's no console
window. See README for how to see
the Spring log.
Be sure to disable either Zeus or Spring when updating gems. Consider
restarting Zeus or Spring after a database migration. See the below
pgk for seeing and killing Zeus/Spring
Relevant Gems Working For Me
The right combination of gems seem pretty critical in getting all the
parts to play nicely together. As of August 15, 2014 the most recent
compatible versions of the following gems worked well together. This
means running "bundle update" without locking the gem versions.
Zeus Shell Configuration (ZSH)
echo "> $1"
eval time $1
DIFF=$(( $END - $START ))
echo "It took $DIFF seconds"
alias zr='zeus rake'
alias parallel_prepare='rake parallel:prepare ; "rake parallel:rake\[db:globals\]" '
echoRun "zeus parallel_rspec -n $p spec"
for x in spring rails phantomjs zeus; do
pgrep -fl $x;
for x in spring rails phantomjs zeus; do
pkill -fl $x;
Please let me know if this article helped you or if I missed anything!