Category Archives: Ruby

Unicorn and bundler issue

I am using unicorn with a rails3 app that uses bundler and I hit a weird error when trying to start a server:

`rescue in load_spec_files': git:// (at rails3) is not checked out. Please run `bundle install` (Bundler::GitError)

The background to this is that I am starting unicorn_rails as root then dropping privs in the after_fork block, this is the relevant section from my unicorn.conf file:

after_fork do |server, worker|
  # …
  uid, gid = Process.euid, Process.egid
  user, group = 'will', 'will'
  target_uid = Etc.getpwnam(user).uid
  target_gid = Etc.getgrnam(group).gid
  worker.tmp.chown(target_uid, target_gid)

  if uid != target_uid || gid != target_gid
    Process.initgroups(user, target_gid)

This is basically the same as the GitHub unicorn config.

Back to the problem. It works fine for all the standard gems, but was failing on the dependencies I had specified as ‘:git’ deps, for instance authlogic:

gem 'authlogic', :git => 'git://', :branch => 'rails3'

I checked where these gem/git dependencies were being stored and saw the problem:

[will@server current]$ bundle show activerecord

[will@server current]$ bundle show authlogic

The :git dependencies were being bundled to a subdirectory of my home dir (the same user that does the deploy, and therefore the initial bundle install. The unicorn_rails process starts as root and has already run Bundler.setup before it forks, so Bundler.setup gets run as root, and the root user has no idea where the :git dependencies are, hence the error.

I solved this by adding a call to Bundler.setup from within the unicorn.conf after_fork block:

after_fork do |server, worker|
  # …
  uid, gid = Process.euid, Process.egid
  user, group = 'will', 'will'
  target_uid = Etc.getpwnam(user).uid
  target_gid = Etc.getgrnam(group).gid
  worker.tmp.chown(target_uid, target_gid)

  if uid != target_uid || gid != target_gid
    Process.initgroups(user, target_gid)

It seems to work fine, better solutions welcome!

Tagged , ,

Basic zeroMQ Ruby example

Update: As Jake pointed out in the comments you obviously need zero MQ installed for this example to work. Just ‘brew install zeromq’ or ‘port install zmq’ on OS X, or use your Linux package manager.

I couldn’t find may examples of zeroMQ usage in Ruby so here is a basic sender/consumer I made to test it. First install the ‘zmq’ gem:

gem install zmq --no-ri --no-rdoc

Now start a worker, you can start as many as you want:

Now stick some messages on the queue:

You should get messages distributed to all the worker processes you started up. Pretty simple!

Tagged ,

map-reduce using mongoid

It took me a while to work out how to use map-reduce in Ruby using mongoid so I thought I’d share it here in-case it helps anyone else get there quicker. I start out with a model that includes Mongoid::Document:

To map-reduce across the collection I need to define a map and reduce function in javascript then run the on the collection:

I can roll this into my VisitorSession model:

This obviously makes it easier to call:

>> VisitorSession.first(:conditions => {:project_id => '2f5178'}).visits_for_project
=> 1

Tagged ,

Fixing the warden and rails3 TypeError (can’t convert nil into String) error

I’ve upgraded an app I run that uses warden to Rails3. I started getting “TypeError (can’t convert nil into String)” exceptions after the upgrade:

I tracked it down to the action name not getting set, so I added this in my Warden::Manager.before_failure block:

env['action_dispatch.request.path_parameters'][:action] = "login"

The complete block now:

There may be a better way of doing this, but it works for me.

Tagged ,

fixing the rspec2 and rails3 “there is already a transaction in progress” error

I upgraded an app to rspec2 and rails3 recently and ran into a problem. I run each test in a transaction which rolls back after each test has finished. This works fine normally, but It seemed that after the upgrade the first failing test would cause this per-test transaction to not get rolled back so all the subsequent tests would run in the same transaction.

This caused the tests so fail with validation errors as test data was getting re-inserted, and a validates_uniqueness_of validation was complaining each time, note the second failing test, this test should have passed but failed with a “Validation failed: Name has already been taken” error:

It turns out that what was actually happening was that an exception raised in the after(:each) block was causing the rollback to fail and the failing test was just masking the error. This was my after(:each) block:

This code fails when there is no directory to remove and though this worked fine in the older version of rspec (rspec just ignored the error) with rspec2 it caused the issues discussed above. The solution (because I don’t care if the file exists or not before I delete it) is as simple as adding a condition:

Tagged , ,

NWRUG June 17th – Mongo

I forgot to blog about this earlier, but Adam Holt will be talking about all things mongo on Thursday the 17th June (that’s tomorrow at the time of posting). There will be free pizza afterwards, you just need to sign up (soon) if you’re coming.

Next NWRUG meeting 21st January – UNIX: Rediscovering the wheel

The original announcement is available on the NWRUG site.

This month John Leach of Brightbox will be talking on UNIX: Rediscovering the wheel.

“Those who don’t understand UNIX are condemned to reinvent it, poorly.”

We in the Ruby Community seem to have a habit of re-inventing things. Sometimes this is for good reason, but in some cases we don’t know we’re even doing it! We’re wasting valuable time that could be spent learning Erlang!

UNIX-like operating systems have been around for decades and lots of problems have come and gone in that time. I’m going to talk about some of the tools available that can be used to solve common Ruby and Rails deployment and development problems.

Brightbox will also be sponsoring the event so there will be pizzas available after the meeting in Odder across the road from the meeting.


6:30pm :: Welcome & Pre-session bar visit.
7:00pm :: UNIX: Rediscovering the wheel by John Leach of Brightbox
7:45pm :: Pizzas at the Odder bar across the road sponsored by Brightbox

If you want more information email, call Will on 07939 547 962 or tweet @will_j.

Sign Up

If you would like to attend this event please sign up here as the BBC need a list of attendees before the event, and I need to arrange the correct amount of food.


This meeting is being held at one of our regular venues, the BBC Manchester main building on Oxford Road in central Manchester (Directions). If you get lost call Will on 07939 547 962.

Flushing memcached servers from Ruby

In Flushing memcached servers the easy way I highlighted a way to flush a memcached server without restarting it:

$ echo ”flush_all” | nc localhost 11211

However I almost never use the actual shell version of this, mostly I do the equivalent in Ruby by opening up a socket and communicating through that. Here’s a simple example:

socket = '', 11211 )
socket.write( "flush_allrn" )
result = socket.recv(2)
puts "Success!" if result == 'OK'


Optimising the Recycling Group Finder – Making a Ruby on Rails app faster

This is really just the ‘story’ of how I fixed a very slight performance issue with the Recycling Group Finder site that I run, but I figured it would be worth a post as an example or motivation to anyone else who needs to get started investigating their own Ruby on Rails app performance issues.

The performance problem

I’ve been very happy with the responsiveness of the Recycling Group Finder, so just out of interest, just to see what it would tell me, I installed the NewRelic RPM plugin and activated the free Bronze account available to EngineYard customers. The results were pretty satisfying as my average response time for the most popular page was 163ms maximum with the second most popular page at 90ms. Those are good response times and fall well within the 37signals response time rule:

Our general rule of thumb is that most pages should render their HTML on the server in less than 200ms and almost all in less than 500ms.

Suspicious looking

One of the great things about data visualisations is it can make it really easy to spot patterns. Take this New Relic graph for example:

Recycling Group Finder - graph, before optimisation

Recycling Group Finder - before optimisation

The yellow on the graph represents time spent in the database, the blue is time spent in Ruby, ie. rendering, controllers etc. Memcached accesses are on there too but they’re so fast they hardly appear. This graph looked suspicious to me, I’d normally expect database time to be a much smaller proportion of the overall request time. So it looks like there may be some optimisation that can be done, but in order to optimise I first need to know what to optimise.

The hunt

Google for “rules of optimisation“. Most rules are something like this:

  1. Don’t optimise yet
  2. If you need to optimise, profile first.

I’m never going to be able to optimise my code unless I know what to optimise. If I trawl through looking for places that might be slow and trying to make them faster the chances are I’m going to spend hours changing code for no benefit. I might even make it slower. I need to know exactly where the bottleneck is, I need to profile my code.

There are a bunch of ways of finding out where your code is slow and I’ve personally used ruby-prof before with good results. However I know that the issue here is in the database, and I know that Rack::Bug will show me SQL queries that have run for an action, and importantly how long they took, so that’s what I’m going to try first. I install the plugin, configure it and load it up. The issue is immediately obvious:

Recycling Group Finder - Rack::Bug SQL queries

Recycling Group Finder - Rack::Bug SQL queries

Almost all of the SQL that is executed is under 0.5ms per query, there are a few queries at ~4ms but he one query that really stands out is the third one down. At 44.75ms it is more than half of the overall SQL time. Bingo! Now I know what is slow I need to know why it is slow. Time to break out the query analyser.

Fixing it

I needed to dig deeper into that SQL statement to see what it was doing, so I opened up a postgres shell and ran an explain analyse on the query:

The issue seems pretty clear. There is a Sequential scan on groups:

Seq Scan on groups (cost=0.00..626.75 rows=4885 width=363) (actual time=0.038..26.495 rows=5126 loops=1)

A Sequential scan on a large table is going to sink performance. I can see that the sequential scan is definitely the issue in this case as the cost and time taken are significant proportions of the overall query time. I need to eliminate it. Here’s the code that generates that query:

@groups = Group.find(:all, :include => :group_page, :origin => [, @location.lng], :limit => 30, :conditions => ["defunct = false AND lat is not null and lng is not null and full_address is not null and full_address != '' and country_code = ?", @location.country_code], :order => 'distance ASC, num_members DESC')

I wrote this code ages ago and re-reading it now I can see that although I am limiting the returned results to 30 rows the query will have to hit every row in the table to determine which rows are in the returned 30 as there are no conditions to the query. Whoops. Looking over the Geokit docs I see there’s a :within condition so I added a :within => 100 to the find. Testing the resultant query in the postgres shell using explain analyse again and the query has dropped to 10ms. Not bad but it’s still using a sequential scan. Adding an index on the conditions speeds up the query further to ~1.2ms:

Not bad when it started out at nearly 45ms. Here is the result reflected in the New Relic graph:

Recycling Group Finder - After optimisation

Recycling Group Finder - after optimisation

I deployed the new code approximately in the middle of the graph, it should be pretty obvious where.


Before you can optimise your Ruby on Rails app (or your app in any other framework/language for that matter) you need to know know where to optimise. Tools like Rack::Bug and NewRelic allow you to do this effectively and easily allowing you to direct your attention only on those parts of your app that need the attention.

On the Recycling Group Finder I cut response times drastically in about half an hour. Without knowing exactly where to make the change I would have been left guessing and may never have made the optimisation I did.

Looking for a Web-Development job? Learn Ruby and Ruby on Rails

Seriously. Not only will you be able to develop web-applications faster and with more joy, but if you fill some of the many Ruby on Rails job vacancies there are going the recruiters might stop bugging me so often.

There are Ruby on Rails jobs out there

Or that’s what is seems like from talking to people at the NWRUG and Geekup meetings I go to and by the phone calls I get from recruitment agents. I know of companies worried about using Ruby and Rails because of concerns over the number of developers available. These companies need you and they need you to write web applications for them in Ruby on Rails! These are companies who want to use Ruby on Rails and they will hire you if you learn it.

I’m fine sticking with $some_other_language but thanks anyway

That’s fine, there are lots of jobs available using your programming language. Well, maybe not if that language is Coldfusion. But if you expand your horizons, teach yourself something new and can prove to others that you’re interested in and capable of learning then you’re going to be a more valuable asset. That’s going to translate into more pay and a more fulfilling job using a language as expressive as Ruby and a framework as labor-saving as Ruby on Rails.

Worst case scenario is that you learn Ruby on Rails and you can write your own web-apps a whole lot faster (you do write your own web-apps right?), your CV looks better and you have more time for the dull stuff that you fit around programming. Watching Buffy or something. You know, programmer social life stuff.

I tried Ruby and Ruby on Rails already but I prefer Python…


I tried Ruby and Ruby on Rails already but I prefer Cobol!

You don’t exist, go away.

You were thoroughly convincing, I’m sold

This post is so convincing that when I proof read it I nearly went and learned ruby on Rails myself, even though I already know it. If you want to you learn you can start here, and there’s going to be a local Ruby user in your area somewhere, sign up to their mailing list, we’re a pretty helpful bunch.

If you’re anywhere near Manchester, UK then come along to the next NWRUG meeting, it’s this Thursday and there’s free pizza. You need to sign up to attend.