Category Archives: Rails

Railsberry animated gifs

I put up some photos of Railsberry here, but Flickr doesn’t work with animated gifs, so here they are. Click the images to get the animated versions.

Riding the Railsberry Unicorn:

Jon Leighton getting closer:

Josh Kalderimis misbehaving:

Tagged

Fixing the Gemfile not found (Bundler::GemfileNotFound) error

I was working on an app (bundler, unicorn, Rails3) that had a strange deploy issue. Five deploys (using capistrano) after our unicorn processes had started unicorn would fail to restart, this is the capistrano output:

* executing `deploy:restart'
* executing `unicorn:restart'
* executing "cd /u/apps/dash/current && unicornctl restart"
servers: ["stats-01"]
[stats-01] executing command
** [out :: stats-01] Restarting pid 15160...
** [out :: stats-01] PID 15160 has not changed, so the deploy may have failed. Check the unicorn log for issues.

I checked the unicorn log for details:

I, [2011-08-02T15:59:32.498371 #11790] INFO -- : executing ["/u/apps/dash/shared/bundle/ruby/1.9.1/bin/unicorn", "/u/apps/dash/current/config.ru", "-Dc", "/u/apps/dash/current/config/unicorn.conf.rb", "-E", "production"] (in /u/apps/dash/releases/20110802155921)
/opt/ruby/lib/ruby/gems/1.9.1/gems/bundler-1.0.15/lib/bundler/definition.rb:14:in `build': /u/apps/dash/releases/20110802152815/Gemfile not found (Bundler::GemfileNotFound)
from /opt/ruby/lib/ruby/gems/1.9.1/gems/bundler-1.0.15/lib/bundler.rb:136:in `definition'
from /opt/ruby/lib/ruby/gems/1.9.1/gems/bundler-1.0.15/lib/bundler.rb:124:in `load'
from /opt/ruby/lib/ruby/gems/1.9.1/gems/bundler-1.0.15/lib/bundler.rb:107:in `setup'
from /opt/ruby/lib/ruby/gems/1.9.1/gems/bundler-1.0.15/lib/bundler/setup.rb:17:in `'
from :29:in `require'
from :29:in `require'
E, [2011-08-02T15:59:32.682270 #11225] ERROR -- : reaped # exec()-ed

Sure enough there’s the exception, “Gemfile not found (Bundler::GemfileNotFound)“, and the file referenced (/u/apps/dash/releases/20110802152815/Gemfile) didn’t exist. The directory that was being looked in (20110802152815) was from a previous deploy and had been rotated off the filesystem. We keep five historical deploys so that explained why the problem only happened five deploys after a full unicorn restart.

I suspected an environment variable was getting set somewhere, and never updated, so I added some debugging to our unicorn.conf.rb file:

ENV.each{ |k, v| puts "#{k}:tt#{v}" }

I then restarted the unicorns fully and tailed the unicorn log file while deploying the app. Sure enough one of the environment variables stuck out:

BUNDLE_GEMFILE: /u/apps/dash/releases/20110802165726/Gemfile

I deployed again, it remained the same, still pointing to /u/apps/dash/releases/20110802165726/Gemfile. I continued to deploy until release 20110802165726 was rotated off the filesystem and up pops the error again. This looks like the problem.

I committed a change to our unicorn.conf.rb that set the BUNDLE_GEMFILE variable explicitly in the before_exec block:

before_exec do |server|
ENV['BUNDLE_GEMFILE'] = "/u/apps/dash/current/Gemfile"

end

Over 5 deploys later and the env var is still set to /u/apps/dash/current/Gemfile and there are no more errors. Let me know if you found this useful!

Notes

  • There may be other issues that cause errors of this type, this was just the solution that worked for us, YMMV.
  • There may be better places to set the environment variable other than the unicorn.conf.rb, I’m open to suggestions (we’re using bluepill, I may be able to set it there for intance).
Update: I’ve changed this on our systems so the environment variable is set in bluepill, it works the same.
Tagged , ,

Setting up PostgreSQL for Ruby on Rails development on OS X

One of the reasons people used to give for using MySQL over PostgreSQL (just ‘Postgres’ from here on in) was that Postgres was considered hard to install. It’s a shame, because it’s a great database (I’ve been using it for personal and some work projects for years, like my current side project, sendcat). Luckily it’s now really simple to get it going on your Mac to give it a try. This is how you do it.

What this guide is

This is a guide to getting PostgreSQL running locally on your Mac, then configuring Rails to use that for development.

What this guide is not

  • An advanced PostgreSQL guide.
  • Suitable for using in production.
  • Anything to do with why you might want to use PostgreSQL over any other database.

Installation

You can get binaries for most systems from the Postgresql site, but it’s even easier if you’ve got homebrew installed, if you haven’t got homebrew it’s worth it, pick it up here. I’m going to assume you are installing from Homebrew for this post, but you should find the information useful even if you are installing directly or using Macports.

With homebrew just run:

$ brew install postgres

You will get a load of output, but the most important part is this:

If this is your first install, create a database with:
    initdb /usr/local/var/postgres

If this is your first install, automatically load on login with:
    mkdir -p ~/Library/LaunchAgents
    cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
    launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist

If this is an upgrade and you already have the org.postgresql.postgres.plist loaded:
    launchctl unload -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
    cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
    launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist

Or start manually with:
    pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start

And stop with:
    pg_ctl -D /usr/local/var/postgres stop -s -m fast

If you want to install the postgres gem, including ARCHFLAGS is recommended:
    env ARCHFLAGS="-arch x86_64" gem install pg

There’s a lot to read, but don’t worry, you don’t need most of the information there. You can get to that information again by running:

brew info postgres

As the instructions say, if this is your first install, create a database with:

$ initdb /usr/local/var/postgres

Do this now. You should see output like this:

$ initdb /usr/local/var/postgres
The files belonging to this database system will be owned by user "will".
This user must also own the server process.

The database cluster will be initialized with locale en_GB.UTF-8.
The default database encoding has accordingly been set to UTF8.
The default text search configuration will be set to "english".

creating directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting default max_connections ... 20
selecting default shared_buffers ... 2400kB
creating configuration files ... ok
creating template1 database in /usr/local/var/postgres/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the -A option the
next time you run initdb.

Success. You can now start the database server using:

    postgres -D /usr/local/var/postgres
or
    pg_ctl -D /usr/local/var/postgres -l logfile start

Again, there’s a lot of output, but you can pretty much ignore most of it.

Startup/Shutdown

Next, as the instructions suggest you can set Postgres to start and stop automatically when your mac starts. Run these  three commands to have this happen (Postgres will start when you run the last command so there is no need to manually start it if you do this):

mkdir -p ~/Library/LaunchAgents
cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist

I’ve done this because I use Postgres for all my personal projects. If you’re just experimenting and want to control when it is running you can start and stop Postgres with these commands (perhaps with a shell alias). EDIT: Someone on the Hacker News thread suggested Lunchy for managing launchctl stuff, I’ve not tried it, but it looks useful.

Start Postgres:

pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start

Stop Postgres:

pg_ctl -D /usr/local/var/postgres stop -s -m fast

That’s it, Postgres is up and running. You can see it in the process list. Run “ps auxwww | grep postgres” and you should see output like this:

$ ps auxwww | grep postgres
will     33206   0.4  0.0  2435116    528 s004  S+    6:52pm   0:00.00 grep postgres
will     33011   0.0  0.0  2445360    880   ??  Ss    6:41pm   0:00.14 postgres: writer process
will     33007   0.0  0.1  2445360   2412   ??  S     6:41pm   0:00.25 /usr/local/Cellar/postgresql/9.0.4/bin/postgres -D /usr/local/var/postgres -r /usr/local/var/postgres/server.log
will     33014   0.0  0.0  2441392    420   ??  Ss    6:41pm   0:00.03 postgres: stats collector process
will     33013   0.0  0.0  2445492   1460   ??  Ss    6:41pm   0:00.03 postgres: autovacuum launcher process
will     33012   0.0  0.0  2445360    504   ??  Ss    6:41pm   0:00.10 postgres: wal writer process

Create a user and database

Now that the Postgres server is running we need to create a database for use in our rails app. This is really simple using the shell commands that ship with Postgres. First lets create a new user. Running the createuser command you will get an interactive prompt asking some questions about the user, answering ‘n’ is OK for all of them:

$ createuser shawsome
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) n
Shall the new role be allowed to create more new roles? (y/n) n

Next create the two databases you will need, development and test. Here you can see the options are given on the command line, the -O specifies the owner of the database (the user we just created) and -U specified the character encoding scheme to be used in the database.

$ createdb -Oshawsome -Eutf8 shawsome_development
$ createdb -Oshawsome -Eutf8 shawsome_test

You can verify everything worked by connecting. Postgres ships with a shell just as MySQL does, it’s called ‘psql’. Run the following command, you should find yourself at a database prompt:

$ psql -U shawsome shawsome_development
psql (9.0.4)
Type "help" for help.

shawsome_development=>

That’s the DB all done with. Hit ctrl-d to exit the shell. Next install the postgres gem.

$ sudo env ARCHFLAGS="-arch x86_64" gem install --no-ri --no-rdoc pg
Building native extensions.  This could take a while...
Successfully installed pg-0.11.0
1 gem installed

For Macports you might have more luck with:

$ sudo env ARCHFLAGS="-arch x86_64" gem install pg -- --with-pg-config=/opt/local/lib/postgresql84/bin/pg_config

If you want to read further on these commands check out the docs for createuser, createdb and psql.

Create the Rails app

Now we need to create the app. Run “rails new” but specify –database=postgresql to get a database.yml pre-configured. We won’t need to edit the database.yml from the generated file, but it does contain some information that could be useful if you’re using Macports so open it up and see what got generated.

$ rails new shawsome --database=postgresql
… Much output

Head into the new app and create a scaffold. It’s not going to be anything fancy, just enough to get some data into the database:

$ cd shawsome
$ rails g scaffold Post title:string author:string body:text
      invoke  active_record
      create    db/migrate/20110528180734_create_posts.rb
      create    app/models/post.rb
      invoke    test_unit
      create      test/unit/post_test.rb
      create      test/fixtures/posts.yml
       route  resources :posts
      invoke  scaffold_controller
      create    app/controllers/posts_controller.rb
      invoke    erb
      create      app/views/posts
      create      app/views/posts/index.html.erb
      create      app/views/posts/edit.html.erb
      create      app/views/posts/show.html.erb
      create      app/views/posts/new.html.erb
      create      app/views/posts/_form.html.erb
      invoke    test_unit
      create      test/functional/posts_controller_test.rb
      invoke    helper
      create      app/helpers/posts_helper.rb
      invoke      test_unit
      create        test/unit/helpers/posts_helper_test.rb
      invoke  stylesheets
      create    public/stylesheets/scaffold.css

You will now have a migration. We’re going to edit it a bit from the default to add some sensible restrictions and an index.

Run your super shiny migration:

$ rake db:migrate
(in /Users/will/shawsome)
==  CreatePosts: migrating ====================================================
-- create_table(:posts)
NOTICE:  CREATE TABLE will create implicit sequence "posts_id_seq" for serial column "posts.id"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "posts_pkey" for table "posts"
   -> 0.0060s
-- add_index(:posts, :author)
   -> 0.0033s
==  CreatePosts: migrated (0.0097s) ===========================================

Now start up a Rails server.

$ rails s

Poking around in the database

Add a few posts and you can re-run the database shell (see above) and start poking around. You can use ? to get help in the shell, but we will jump straight to dt to get a list of tables:

shawsome_development=> dt
               List of relations
 Schema |       Name        | Type  |  Owner
--------+-------------------+-------+----------
 public | posts             | table | shawsome
 public | schema_migrations | table | shawsome
(2 rows)

Great, our posts table is there, lets take a look at the posts table in more detail:

shawsome_development=> d posts
                                     Table "public.posts"
   Column   |            Type             |                     Modifiers
------------+-----------------------------+----------------------------------------------------
 id         | integer                     | not null default nextval('posts_id_seq'::regclass)
 title      | character varying(255)      | not null
 body       | text                        |
 author     | character varying(255)      | not null default 'Anonymous'::character varying
 created_at | timestamp without time zone |
 updated_at | timestamp without time zone |
Indexes:
    "posts_pkey" PRIMARY KEY, btree (id)
    "index_posts_on_author" btree (author)

You can see we get a fair amount of detail here, including column type, null/not null flags, and default values, as well as any indexes on the table. Lets select some data. This should be familiar to anyone who has used a relational database before (hint: try tab completion, it’s really good in the Postgres shell)

shawsome_development=> select id, title, author, created_at from posts;
 id |     title      |  author   |         created_at
----+----------------+-----------+----------------------------
  1 | Book 1         | Anonymous | 2011-05-28 18:09:13.965425
  2 | Bobski's dream | Anonymous | 2011-05-28 18:09:30.122767
(2 rows)

Done!

What now?

Check out the Postgres docs, they’re really good, and go forth and develop excellent sites on top of PostreSQL!

Tagged , , ,

Rails 2 -> 3 undefined method `html_safe’ for nil:NilClass error

I am converting the Recycling Group Finder site from Rails 2 to Rails 3 and though it has mostly gone to plan I was temporarily held up by this error which I was getting on some pages:

The error was hard to track down as the error message wasn’t very descriptive, but in the end it turned out to be caused by a comment. I am using content_for blocks to generate sections of page content and for a long section I had added a comment to the end of the block to help be know which block was closing:

It turns out that this ‘# some_section’ comment was the problem, possibly because of the change to erubis in Rails 3. Removing the comment caused the page to start working again:

I hope this page helps short-cut the debugging for anyone else that is bitten by this issue.

Tagged , ,

Running Unicorn under supervise/daemontools

I’m running a Rails3 app under daemontools, this is how I did it.

First, install then start daemontools and Unicorn. This is left as an exercise for the reader, it was easy under CentOS 5. You should end up with a /service directory on your server.

Run “mkdir /service/my_service_name && cd /service/my_service_name && ls”. If you have the right processes running you should have a “supervise” subdirectory automatically created for you under your new directory.

You should be in the “/service/my_service_name” directory. In order to get anything to run you need to create a “run” script. Here’s mine, you can copy and modify it for your own use. If you’re running Unicorn and you improve the script I’d appreciate feedback in the comments:

#!/bin/sh
cd /home/www/foo/current && unicorn_rails -E staging -c /home/www/foo/shared/config/unicorn.conf

Make sure the script is executable. If you have done everything right you should have a Unicorn process running. You should see something similar to this in the output of “ps auxww -H”:

If that’s not the sort of thing you see, check the logs. Lastly I added this cap snippet to kill the process on deploy and let supervise handle re-spawning it:

Done!

Why daemontools?

There is some propaganda on the daemontools site, but I used it because it’s quick to get going and in my experience reliable. I use monit a lot at Engine Yard, but there are no recent monit packages easily available for CentOS that I have found. One drawback is that you don’t get any of the resource monitoring that monit provides, such as http checks, memory checks etc. You’d have to implement this yourself, but in this instance that’s the tradeoff you make for simplicity.

CentOS 5 update:

If you get an error that looks like this:

Follow these instructions to fix the issue:

Unicorn and bundler issue

I am using unicorn with a rails3 app that uses bundler and I hit a weird error when trying to start a server:

`rescue in load_spec_files': git://github.com/odorcicd/authlogic.git (at rails3) is not checked out. Please run `bundle install` (Bundler::GitError)

The background to this is that I am starting unicorn_rails as root then dropping privs in the after_fork block, this is the relevant section from my unicorn.conf file:

after_fork do |server, worker|
  # …
  uid, gid = Process.euid, Process.egid
  user, group = 'will', 'will'
  target_uid = Etc.getpwnam(user).uid
  target_gid = Etc.getgrnam(group).gid
  worker.tmp.chown(target_uid, target_gid)

  if uid != target_uid || gid != target_gid
    Process.initgroups(user, target_gid)
    Process::GID.change_privilege(target_gid)
    Process::UID.change_privilege(target_uid)
  end
end

This is basically the same as the GitHub unicorn config.

Back to the problem. It works fine for all the standard rubygems.org gems, but was failing on the dependencies I had specified as ‘:git’ deps, for instance authlogic:

gem 'authlogic', :git => 'git://github.com/odorcicd/authlogic.git', :branch => 'rails3'

I checked where these gem/git dependencies were being stored and saw the problem:

[will@server current]$ bundle show activerecord
/usr/local/lib/ruby/gems/1.9.1/gems/activerecord-3.0.0

[will@server current]$ bundle show authlogic
/home/will/.bundler/ruby/1.9.1/authlogic-a087ad0

The :git dependencies were being bundled to a subdirectory of my home dir (the same user that does the deploy, and therefore the initial bundle install. The unicorn_rails process starts as root and has already run Bundler.setup before it forks, so Bundler.setup gets run as root, and the root user has no idea where the :git dependencies are, hence the error.

I solved this by adding a call to Bundler.setup from within the unicorn.conf after_fork block:

after_fork do |server, worker|
  Bundler.setup
  # …
  uid, gid = Process.euid, Process.egid
  user, group = 'will', 'will'
  target_uid = Etc.getpwnam(user).uid
  target_gid = Etc.getgrnam(group).gid
  worker.tmp.chown(target_uid, target_gid)

  if uid != target_uid || gid != target_gid
    Process.initgroups(user, target_gid)
    Process::GID.change_privilege(target_gid)
    Process::UID.change_privilege(target_uid)
  end
end

It seems to work fine, better solutions welcome!

Tagged , ,

Fixing the warden and rails3 TypeError (can’t convert nil into String) error

I’ve upgraded an app I run that uses warden to Rails3. I started getting “TypeError (can’t convert nil into String)” exceptions after the upgrade:

I tracked it down to the action name not getting set, so I added this in my Warden::Manager.before_failure block:

env['action_dispatch.request.path_parameters'][:action] = "login"

The complete block now:

There may be a better way of doing this, but it works for me.

Tagged ,

fixing the rspec2 and rails3 “there is already a transaction in progress” error

I upgraded an app to rspec2 and rails3 recently and ran into a problem. I run each test in a transaction which rolls back after each test has finished. This works fine normally, but It seemed that after the upgrade the first failing test would cause this per-test transaction to not get rolled back so all the subsequent tests would run in the same transaction.

This caused the tests so fail with validation errors as test data was getting re-inserted, and a validates_uniqueness_of validation was complaining each time, note the second failing test, this test should have passed but failed with a “Validation failed: Name has already been taken” error:

It turns out that what was actually happening was that an exception raised in the after(:each) block was causing the rollback to fail and the failing test was just masking the error. This was my after(:each) block:

This code fails when there is no directory to remove and though this worked fine in the older version of rspec (rspec just ignored the error) with rspec2 it caused the issues discussed above. The solution (because I don’t care if the file exists or not before I delete it) is as simple as adding a condition:

Tagged , ,

Generating a plist file in rails

I recently wrote an iPhone app (Waiting for approval in the app store at the time of writing) that needed data exported from a website (recyclinggroupfinder.com). The simplest way of handling external data in an app it seems is using a plist file, so I wrote this to generate one for me.

First of all I made my action respond to the plist format:

Next I created a builder file to format the data:

Then register the MIME type at the bottom of environment.rb:

Mime::Type.register "text/plist", :plist

And that’s it! Well mostly. The XML file generated can be made significantly smaller by converting it into the binary plist format, run this on the command line in terminal after downloading the generated XML plist.

cat things_xml.plist | plutil -convert binary1 - -o things.plist

The resultant binary plist is almost half the size of the XML one, much better for inclusion in an iPhone app:

pleb:~ will$ ls -l things*
-rw-r--r-- 1 will will 1247300 20 Jan 18:50 things.plist
-rw-r--r--@ 1 will will 2110437 20 Jan 18:50 things_xml.plist

Of course it would be much better to generate the binary format directly, and the plist-official gem looks like it can handle that and I mean to investigate, but I wrote the XML version before finding the gem, and it works for me!

Edit: the plist_official gem seems to be gone, but check out the binary plist gem instead.

Tagged , ,

Next NWRUG meeting 21st January – UNIX: Rediscovering the wheel

The original announcement is available on the NWRUG site.

This month John Leach of Brightbox will be talking on UNIX: Rediscovering the wheel.

“Those who don’t understand UNIX are condemned to reinvent it, poorly.”

We in the Ruby Community seem to have a habit of re-inventing things. Sometimes this is for good reason, but in some cases we don’t know we’re even doing it! We’re wasting valuable time that could be spent learning Erlang!

UNIX-like operating systems have been around for decades and lots of problems have come and gone in that time. I’m going to talk about some of the tools available that can be used to solve common Ruby and Rails deployment and development problems.

Brightbox will also be sponsoring the event so there will be pizzas available after the meeting in Odder across the road from the meeting.

Schedule

6:30pm :: Welcome & Pre-session bar visit.
7:00pm :: UNIX: Rediscovering the wheel by John Leach of Brightbox
7:45pm :: Pizzas at the Odder bar across the road sponsored by Brightbox

If you want more information email nwrug@willj.net, call Will on 07939 547 962 or tweet @will_j.

Sign Up

If you would like to attend this event please sign up here as the BBC need a list of attendees before the event, and I need to arrange the correct amount of food.

Location

This meeting is being held at one of our regular venues, the BBC Manchester main building on Oxford Road in central Manchester (Directions). If you get lost call Will on 07939 547 962.

Follow

Get every new post delivered to your Inbox.