In this post I’ll talk about the benefits of generating your Ruby on Rails routes more quickly, and how to implement it.

Background

Rails looks at the ./config/routes.rb file to generate the mappings between URLs in your application and what controller actions those URLs correspond to. In your app, you can say redirect_to new_post_url and Rails knows that you mean http://www.panozzaj.com/posts/new (for example.)

The problem: Slow Routes Generation

Generating routes in a medium-sized Rails application could take upwards of ten seconds. This is a bit slow for feedback. Sometimes you are just exploring the routes in an application, and it could take a while to get better information. Instead of tens of seconds, why not get it down to a few seconds, or – even better – milliseconds.

Speeding Up Rails Route Generation

I’ll take an app that I am currently working on that has only a hundred or so routes defined and a bunch of gems installed. Before any changes, rake routes takes about seven seconds to generate.

One method we can use to speed up our routes query is to use Zeus to speed up the output of rake routes. This is as simple as installing Zeus, running a Zeus server, and running zeus rake routes. This change alone reduced my time for the Zeus command to about 1.25 seconds, which is about a five-fold reduction.

A further step would be saving the output of route generation to a file and looking at that directly. Once this is done, lookups are essentially instantaneous (a few milliseconds to read the file and grep through it.) This can be tricky if you are actively modifying your routes, since you might be looking at a stale route output.

routes Script

Here is a script that handles all of this for you, including using a cache file and regenerating it if needed:

#!/bin/sh

# regenerate tmp/routes if it is not already generated or it is older than the routes config file
if [ ! -f tmp/routes ] || [ tmp/routes -ot config/routes.rb ]; then
  if [ ! -f tmp/routes ]; then
    generating='Generating'
  else
    generating='Regenerating'
  fi

  [ -S .zeus.sock ] && z="zeus"

  if [ -n "$z" ]; then
    echo "$generating routes - Running under zeus."
  else
    echo "$generating routes - No Zeus enviroment detected."
  fi

  $z rake routes > tmp/routes
fi

echo "Using generated routes file."

query=$1
if [ $query ]; then
  grep $query tmp/routes
else
  cat tmp/routes
fi

Basically it will first look at ./tmp/routes to see if there is cached output there. If that file exists and it is newer than the timestamp of the ./config/routes.rb file, then use the cached version (since the routes probably haven’t changed since we last generated.) I like storing it in the ./tmp directory since it is a generated file that should be ignored from source control anyway.

If there is no cache file or it is old, look to see if zeus is running, and if it is, use it since it is faster and should give us the same results. Worst case, use the standard rake command. Regardless, save the output in the cache file for future runs.

I often do a search on the routes output, so if you pass in a parameter (example: routes posts), the script does a search on the routes output. Otherwise it just prints it out for reading or piping to another command.

A Faster Way to Generate Routes

This post covered a faster way to generate routes. I found this handy enough to write up and used it to great effect when working with a larger application that needed a conversion from Rails 2 style routes to Rails 3 style routes.

I hope that this helps you work more quickly with your routes, and let me know about any other tips for faster generation that you have used!

Kyle Shipley notes:

@panozzaj You can also visit /rails/info/routes in Rails 4, or just type in a garbage path and look at the new error page.

— Kyle Shipley (@kyleashipley) May 16, 2014

You probably have a bunch of blog posts that you have already written that could be published today, but you may not be aware that they even exist. This post shows some ways to unlock this potential content for your business or personal blog.

Artifacts

In the course of doing business, there are a number of artifacts that you typically create that would make excellent material for blog posts. These artifacts could include:

• internal shared documents
• general project documentation
• workflow / process guides
• diagrams
• presentations
• lessons learned / retrospectives
• high signal emails
• new project steps

What took hours of work to write up and possibly days or months of hard-won experience or research can be shared with your customers or clients and help them. Obviously you should remove anything that is your company’s secret sauce, confidential, or detailed plans, but why not share your overarching conventions for others to benefit from and potentially improve?

Sometimes these can be used directly. Sometimes they might need a little cleanup. Sometimes explanation would help your audience understand the meaning or context of the document. But you may have ninety percent of a future post that you have already created.

Values

Internal artifacts also reveal something about company values. If in your software development process you explain: “We don’t track bugs as positive velocity” or something like that, that may be a useful thing to explicate. It may be that this belief comprises a core part of your company’s culture. Similarly, reading your internal documents through the eyes of an outsider might give insight into what things your company values and how you arrived at the beliefs you currently hold. One example might be why and how you use feature branches to break up work (or why you consider them harmful.) Perhaps you already have this written up, or it would be good to document to make things clearer for new people on the project.

Sharing the internals of how your company makes decisions aligns you with like-minded customers and potential employees.

Examples

There are a few examples of this process that I can point to on this blog. These came out of typical things that I did at work and that seemed useful enough to share:

Starting on an Existing Rails Project came from a brainstorming document that I wrote up to capture thoughts on how to get up to speed on an existing project more quickly. When I saw how much value there was there, I tweeted to ask others if they would use something like this, and they responded positively (derisking taking another few hours to polish it up.)

How to Run a Successful Brown Bag System was content that I wrote for a blog, but it came from listing out our thought process and timeline of iterating on a system to try to get lunch and learn / brown bags going. From what I know, this process is still going strong in some incarnation at the original company.

Taking action

Take a few minutes right now and go through Dropbox, Google Docs, and your project documentation and make a list which documents might have something that you can share with others. From this list, order the documents by how polished or informative they are. Finally, make a schedule to go through them one at a time and clean them up a little and publish them.

Seriously, go do this. I’ll bet you can find three posts that you can publish with relatively few changes that are valuable.

In this post I’ll cover why I set up a local Jenkins server, and what benefits you might gain by doing the same. This post is probably most suited for developer types, but you may try it if you have many repetitive tasks.

What it looks like in action

Here is what my current Jenkins dashboard looks like:

You may notice that it seems more aesthetically pleasing than other Jenkins installs. This is due to using the Doony plugin, a series of UI improvements to Jenkins that make it much better to use. My favorite is a “Build Now” button that will kick off a build and take you to the console page to watch the build (typically a three or four step process otherwise.) This is handy for testing out new or newly modified jobs. The console output itself looks much better as well. Doony even has a Chrome extension so if you work in a stuffy office, then you can get a sexier Jenkins just for yourself.

What I specifically use it for

There are many things that need to be done to keep a computer system running at its best, and I try to automate as many of them as possible. With a local Jenkins server, I can do this and easily keep tabs on how everything is running.

One easy application is to keep the system updated. On Linux, you’d sudo apt-get update && sudo apt-get upgrade -y. On Mac, running brew update && brew upgrade nightly keeps me running on the latest and greatest. Anything that needs backwards revisions of software (databases, etc.) should have its own virtual environment or be running on the latest anyway.

On a similar note, I pull the latest plugins for Vim. I have over forty git submodules in my dotfiles, most of which are Vim plugins. Running this command keeps me running on the bleeding edge to pick up bug fixes. It also ensures that if I try to clone my dotfiles on another computer that it will work as expected.

I blog with Jekyll and publish scheduled blog posts by running a script hourly that checks for posts that should be published and publishes them. This is pretty consistent and publishes posts within an hour of when they should be published assuming my laptop has an internet connection. If not, probably not a big deal, they will just be published the next time around. This is also handy since I use the AWS credentials stored on my local computer.

To keep things running fast, I have one job that runs git garbage collection on all of my repos nightly as well. (See this StackOverflow post for potential tradeoffs of doing this. I haven’t seen any negative effects so far.) I also have one job that goes through Rails projects that I have and clears out their logs and upload directories. This keeps searches faster and more relevant by trimming log files and attachments, many of which are generated when doing automated test runs.

Last, I run diagnostics. brew doctor ensures that I don’t have any dangling Homebrew installs or ill-fated link attempts. As is standard, the command returns a zero exit code if it succeeds, and something else if it reports anything, which makes it easy to script with Jenkins. I typically just fix the issue in the morning and kick off another build.

Why is it better?

The cron ecosystem is typically the standard way to schedule tasks. However, there are some shortcomings with using cron for this:

• hard to debug when things go wrong
• the job itself might fail silently
• lack of consistent logging or notifications
  • typically need to log console results to a random file
  • inspect that file randomly (remember where it is?)
• hard to link together multiple scripts
• tough to have RVM support
  • need to set up paths correctly or else…
• likely little history of the job and its configuration
• emails might get sent out… or they might not
• by default emails are only sent locally

With Jenkins, you get the entire history of the job with full logs, and can tack on notifications as needed. You have customizability by adding various plugins for interacting with Github repos, seeing disk usage, emailing specific users when builds fail or are unstable. There is a graphical interface for seeing and changing the jobs, and you can version this. Jenkins has some memory overhead, but not much. I think the improved interface is worth it.

I think automating all of this probably saves me time in the long run. I would do it even without Jenkins, now I just have a better interface.

I take the pain early on broken Homebrew packages or Vim bundles. I can more quickly change these, and other developers that I work with can benefit from me seeing the problems first. Running on the bleeding edge provides early warning when a Vim git repository changes or gets deleted, as I typically find out the day after updating. I’d rather have a few small fixes over time than a bunch of fixes when I need to update a critical package.

Lastly, I get better at administering Jenkins, which carries over to other domains like continuous integration / continuous deployment. Building automation muscles.

Setting it up

I covered installing a local Jenkins server in a previous blog post. Check it out for some more thoughts and why Jenkins LTS is likely the way you want to go for local Jenkins installs.

I also change the port from 8080 to 9090 since I sometimes test other Jenkins installs (for pushing up to other servers with Vagrant.) Since I am running on Mac I also use pow for setting it up so I can say http://jenkins.dev to point to localhost:9090:

# echo 'http://localhost:9090' > ~/.pow/jenkins

Some other automation ideas

I plan on outfitting Jenkins with a few more jobs, as time permits.

One job is a script that will go through my git repos and figure out which changes are not committed or pushed across them, along with checking stashes to see if there is code lingering in them. I would like to lessen the time it takes from value creation to value realization. Plus, having unpushed code is typically a liability in case something happens to my computer overnight. The job could email me a summary of this every day so I can take action on it in the morning.

I would like a “standup” script that summarizes what I did the previous day across all projects. Here are some code sketches that are good inspiration:

• prototype 1
• prototype 2
• prototype 3

I’d like to automatically calculate how many words were written across all git repos that are associated with writing. Basically doing a plus/minus/net words written in Markdown files. I have a semblance of a net word count script already, I would just need to wrap it with a list of repos and run daily at midnight. The script would also encourage me to commit my writing more regularly since I know it would affect the stats.

It might be nice to rebuild Ruby documentation for all installed gems in case I am without an internet connection, although I am not sold on the value of this yet. It seems rare these days that I don’t have a connection and need to look things up.

You could use Jenkins to automate things like:

• checking websites for broken external links
• pulling down data from servers or APIs
• sending you email reminders
• mirroring websites
• checking disk space
• running cleanup or backup scripts

The nice thing is that it shouldn’t take long to do most of these, and you can see the history.

This guide represents my current best understanding of how to get productive with an existing Ruby on Rails app. I currently work in a consulting / contracting role on various web development projects, and this general process would work with other frameworks similarly. I thought about the process that I go through (or would like to go through) when starting to work on an existing Ruby on Rails project, and wrote down those thoughts. Afterward, it seemed valuable enough to clean up a little and publish.

In this post I won’t cover getting a legacy project’s tests up to snuff. I also will not cover taking an existing Rails project and upgrading it to the latest version of Rails. There are existing resources that already cover this.

So what will I cover?

Let’s pretend you are starting to work on a Rails project today and this isn’t your first Rails rodeo. However, the project that you are starting on has been in development or on production for months or years. How do you quickly get up to speed to make a contribution, and how do you not break everything in the process?

This project could have been unmaintained for a little while. You could be joining up with an existing team. It could have been outsourced and in disrepair. It could have been insourced and in disrepair. :) It could be well-tested or have no test coverage. But someone, somewhere, cares about this project, or you wouldn’t be working on it. Where do you get started?

Overview

The first thing to do is to understand what the project does at a high level. This might seem obvious, but still important. Questions to answer include:

• What value does this project have to the business that owns it?
• What is the primary functionality of this project?
• Where does this fit in with the company’s other projects / products?
• Who uses this app on a daily basis?
• How many people use this app on a daily basis?
• What is the history of the project development?
• If the old developers are still available, can you contact them?

Once you have this information, you have a better understanding of the context of the project and those things most important to keep running.

Get Access

You can’t get very far without having access to anything important. You’ll eventually need access to:

• source control (Github)
• project management (Trello, Jira)
• team communication tools (Slack, Teams, Zoom)
• supplementary docs (Google Docs, Notion, wiki, Miro)
• deployment credentials (Heroku)
• error notifications (Airbrake, Exceptional, Honeybadger)
• storage credentials (AWS)
• end user message apps (Helpscout)
• performance monitoring (New Relic)
• continuous integration (GitHub Actions, Jenkins)

I would generally send out one email to whoever should have most of these and see if you can get credentials or get your account added. I would then make a document that has all of these services in one list, and then whenever you onboard a new developer you can add them more quickly the next time.

Reading the Fine Manual

After understanding the project, the next thing to do is to look at the clues previous developers have given you. Sometimes you get the dreaded Welcome to Rails documentation, but often your predecessors or teammates have left a breadcrumb trail of how to get up and running on the app.

Read through the README. Check out the doc folder, which I’ve seen contain very detailed and useful information. If the project is hosted on Github or the like, check to see if there is a wiki. Also, look at the Gemfile. This will give you a sense of the parts of the application and can be an effective way to date apps. Look for the following answers:

Does the documentation seem up to date? Compare the documentation to the Gemfile and see which elements are discussed and which aren’t.

Are there any gotchas or other things to watch out for? Maybe the project shouldn’t be deployed more than three times an hour or the app workers will cause too many external auth requests. (Side note: as a developer, if you know things like this about your project, document it so the next person knows to watch out for this!)

What versions of Ruby/Rails is the app using? If it isn’t yet documented in code, I use .ruby-version to hold the version (say, 2.0.0p123) and .ruby-gemset to hold an appropriately named gemset. RVM and other ruby environment managers look at these files and can automatically switch to the right Ruby version and gemset when you change to the project directory. This encapsulates your project environment better and ensures that you have only the things you expect actually accessible.

What database is the app using? You’ll need to have this installed, with the right version. If the database version is not clearly specified, look at what production is running. Note any differences in database versions between environments.

What external services does the app interact with? You may need to be aware of these and have at least development credentials for it to work correctly.

Get a local development server running

The next goal is to get an environment where you can see the results of changes that you make to the app. Ideally there would be a project setup script you can run or a Vagrant environment that you can run in to do all of this setup for you. However, I have found that this is not typically the case. You may need to:

• Copy the example database configuration and make any necessary changes
  • cp config/database.yml.example config/database.yml
• Create the databases, run migrations from scratch, look at the seeds and run them
• See what server is used in production by looking at Procfile

Document any additional dependencies required or gotchas. Also, note any differences between running the migrations from scratch and what your final database schema looks like. Sometimes there are database scripts that were not idempotent, run correctly, or were removed somewhere along the way. You may need to fix some migrations if they reference classes that have been renamed and have not been run since that happened.

Hopefully you can get a development server up and running and run through the app a bit. At this point you can also test out running workers and other things if they won’t impact other environments.

Get the tests running

Get the tests running locally if they are somewhat reasonably maintained. If there are no tests or the tests are all or mostly failing, this is something to be aware of and you may reconsider your project and/or life choices. Some failing tests may point to actual problems in your environment. More likely, the tests themselves were not maintained properly or are not deterministic. Generally if the tests at least pass at the file level, then you can proceed. Document any test environment setup that was needed to get things going.

If the tests are non-deterministic, consider introducing database_cleaner and timecop, and potentially doing other analysis.

Next, understand the code coverage. I personally have room to grow in this area, but saying “All of the tests pass, I have no idea why staging is broken!” is a poor substitute to understanding that the tests are weak in an area and beefing them up or doing manual testing to prevent regressions.

Install simplecov and run against it the app. Save this somewhere for future analysis / comparison. This is useful for being able to say, “we increased the code coverage of the app by 10% in the past three months.”

Understanding (or creating) the project workflow

Next, get one change into production through any other gates that are necessary (staging, demo, etc.). This could ideally be a minor change like a comment in one of the view files so you can see that it went through correctly without impacting any end user behavior. Pushing up a change at this point ensures that you can push up changes to the codebase so you can deliver the value that you create. This also ferrets out any issues you might have with deploying code sooner than later (insufficient privileges, production errors on push, etc.)

Figure out how you should interact with the current codebase to get your changes in. What is the merge workflow like for the project? Is it documented somewhere? Perhaps you should use pull requests or feature branches. Basically you don’t want to stomp on other people’s commits if the project is currently under active development, and want to put commits through a consistent code review.

What is the deployment workflow like for the project?

• What environments are this application deployed to, and who uses them?
• Where are the backups, if any, for each environment?? :)
• Know branches to deploy from, where it deploys to, git remotes, etc.
• Do I have any needed deployment keys or credentials?
• How is the performance monitored on each environment?

Push your minor change to intermediate environments and then production. When you see your change on production, you are done with basic setup of the app. Congratulations!

Intermission

The more this process is documented / automated, the less time will be required to onboard new developers. So be sure to document anything that you needed to do to set it up.

After getting the code running and successfully deployed, you may wish to make a few more improvements, which I’ll outline below.

Understand the application code

To get a better understanding of what is going on under the hood, make sure you review most of the code to see where the functionality lives. You can run rake stats to get a sense of how many lines of code are in each major section of the app.

I recommend running the Rails ERD tool or something similar (RubyMine comes to mind) to visualize the data model to get an understanding of what models interact with other models. Sometimes fairly basic data model problems can be uncovered just by looking at the output of this project.

Run through the app code quickly, starting with the models and lib files, moving to controllers, and finishing with views and JavaScript/CoffeeScript files (viewing helpers as needed.)

Are all of the gems in the Gemfile documented and used? Be sure to understand how all of the gems are or might be used. If project startup is extremely slow, consider using bumbler to get the project load times down by optimizing which gems are actually required and which ones can be loaded dynamically by your app.

Improve your working environment

Get quick iteration times by installing and using Zeus or the now-more-official-than-Zeus spring. This should speed up your testing iteration by a large factor.

Get faster testing times if the suite takes over ten minutes wall time to run. Install and get parallel_tests working as one way to solve this problem.

High level audit

Are there any project security issues to be aware of? You can add great value with fairly low effort by ensuring that the project is safe from known or silly security vulnerabilities.

Upgrade Rails

First, ensure the project is running on the latest point release of the current minor that you are on, with upgrading to a later Rails minor or major release if that is feasible. Upgrading a minor or major version is an undertaking that may take days or weeks, so you should probably note this and discuss with your client.

Example: if your project is running 3.2.12 (major 3, minor 2, point 12), you should upgrade to the latest in the 3.2.x line since point releases are likely backward compatible. Thereafter, look to upgrade to the highest minor release of the 3.x.y (if not there already), until you get to a major release (so 3.2.latest to 4.0.latest to 4.1.latest).

The theory is that point releases are most likely to contain security or other bug fixes, and upgrading a minor or major revision might take more work but is useful for keeping the app up to date. Again, I won’t cover this process in more detail.

Upgrade gems

Grab bundler-audit and run bundle-audit update and bundle-audit to see what known security problems are present in the current gemset. You should upgrade these at least minimally to fix the issue, and hopefully the app will have tests that break if the gem functionality changes in a breaking way.

While you are at it, you can get a sense of how outdated all of the gems are and how much time it would take to upgrade the app to use the latest versions of those gems. This is useful for security, increased functionality, and usually bug fixes and speed improvements. Run bundle outdated to see which gems are behind their current latest revision on RubyGems.

Other security issues

Check for SQL injection attacks by at least looking through the app and searching for connection or execute and inspecting these for potentially user-supplied values.

Look for other basic issues as explained in the following guides:

• The Official Ruby on Rails Security Guide
• The Inadequate Guide to Rails Security

Supporting infrastructure and services

You can add a lot of value by ensuring that you have the following services connected to your app:

• error reporting service (Honeybadger, etc.)
• performance monitoring (New Relic)
• continuous integration server (Jenkins, etc.)

These ensure that your app is running properly and that you can detect when things are not running well. The CI server will ensure that tests are running consistently and deterministically. Typically you can get hosted solutions for all of these problems which will save the project time and therefore money.

What did I miss?

Some people reading this post thought that there were a few more things that you might consider to help your understanding of the codebase.

Pro blogger and Ruby enthusiast Matt Swanson suggests the following tools:

@panozzaj two small hacks of mine: xray-rails (shows name of layouts/partials in browser) and pry (breakpoint debugging in your terminal)

— matt swanson (@_swanson) April 30, 2014

Indianapolis Ruby developer and Wicked PDF expert David Jones suggests looking at your codebase with some tools / services to get a sense of what is especially tricky:

@panozzaj @brakeman should be in your toolbox alongside bundler-audit. Also use metric_fu or @codeclimate to get an eye on the hairy bits.

— David Jones (@unixmonkey) May 1, 2014

What else did I miss or that you find especially helpful? Thanks!

Original tweet:

New blog post: Starting on an Existing Rails Project http://t.co/luZ7oVe1H9 #rails #ruby

— Anthony Panozzo (@panozzaj) April 30, 2014

I run the continuous integration server Jenkins on my local development machine to work as a better cron. I actually run the Jenkins LTS (long term support) version since it is designed to be more stable than the developer release line.

From the Jenkins LTS docs:

Jenkins produces a new release weekly to deliver bug fixes and new features rapidly to users and plugin developers who need them. But for more conservative users, it’s preferable to stick to a release line which changes less often and only for important bug fixes, even if such a release line lags behind in terms of features.

I’d rather have stability than blazing edge features for something that is just supporting infrastructure. Since I am using a Mac, it would be nice to install this using Homebrew. The homebrew-versions keg has a version of this, so you can install it from there with:

$ brew tap homebrew/versions
$ brew install jenkins-lts

Then you can run it with the commands listed at brew info jenkins-lts. By default, jenkins-lts will have a plist to control startup and teardown of the Jenkins process at ~/Library/LaunchAgents/homebrew.mxcl.jenkins-lts.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>homebrew.mxcl.jenkins-lts</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/bin/java</string>
      <string>-Dmail.smtp.starttls.enable=true</string>
      <string>-jar</string>
      <string>/usr/local/opt/jenkins-lts/libexec/jenkins.war</string>
      <string>--httpListenAddress=127.0.0.1</string>
      <string>--httpPort=8080</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
  </dict>
</plist>

I ran into issues where a newly started Jenkins server would intermittently stop responding after several requests. I traced this down with the Monitoring plugin to Jenkins running out of memory, and updated the plist file to include three lines about how to manage the memory of the Jenkins process:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>homebrew.mxcl.jenkins-lts</string>
    <key>ProgramArguments</key>
    <array>
      <string>/usr/bin/java</string>

      <string>-XX:MaxPermSize=1024m</string>
      <string>-Xms1024m</string>
      <string>-Xmx1024m</string>

      <string>-Dmail.smtp.starttls.enable=true</string>
      <string>-jar</string>
      <string>/usr/local/opt/jenkins-lts/libexec/jenkins.war</string>
      <string>--httpListenAddress=127.0.0.1</string>
      <string>--httpPort=8080</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
  </dict>
</plist>

After doing this, I found that the server issues went away, and confirmed it with the Monitoring plugin. I may have set the memory size way too high, but Jenkins still does not take up much memory, so it is fine for now.