I put up some photos of Railsberry here, but Flickr doesn’t work with animated gifs, so here they are. Click the images to get the animated versions.
Riding the Railsberry Unicorn:
Jon Leighton getting closer:
Josh Kalderimis misbehaving:
I put up some photos of Railsberry here, but Flickr doesn’t work with animated gifs, so here they are. Click the images to get the animated versions.
I got the following error installing the latest pg gem on Lion:
Installing pg (0.12.2) with native extensions Unfortunately, a fatal error has occurred. Please report this error to the Bundler issue tracker at https://github.com/carlhuda/bundler/issues so that we can fix it. Thanks!
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:482:in `build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb
checking for pg_config... yes
Using config values from /usr/local/bin/pg_config
checking for libpq-fe.h... yes
checking for libpq/libpq-fs.h... yes
checking for PQconnectdb() in -lpq... no
checking for PQconnectdb() in -llibpq... no
checking for PQconnectdb() in -lms/libpq... no
Can't find the PostgreSQL client library (libpq)
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby
--with-pg
--without-pg
--with-pg-dir
--without-pg-dir
--with-pg-include
--without-pg-include=${pg-dir}/include
--with-pg-lib
--without-pg-lib=${pg-dir}/lib
--with-pg-config
--without-pg-config
--with-pg_config
--without-pg_config
--with-pqlib
--without-pqlib
--with-libpqlib
--without-libpqlib
--with-ms/libpqlib
--without-ms/libpqlib
Gem files will remain installed in /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/pg-0.12.2 for inspection.
Results logged to /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/pg-0.12.2/ext/gem_make.out
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:445:in `each'
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:445:in `build_extensions'
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:197:in `install'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/source.rb:90:in `install'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/rubygems_integration.rb:82:in `preserve_paths'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/source.rb:89:in `install'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:73:in `install_gem_from_spec'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/rubygems_integration.rb:97:in `with_build_args'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:72:in `install_gem_from_spec'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:56:in `run'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:55:in `run'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:12:in `install'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/cli.rb:220:in `install'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/task.rb:22:in `send'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/task.rb:22:in `run'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/invocation.rb:118:in `invoke_task'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor.rb:263:in `dispatch'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/base.rb:386:in `start'
from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/bin/bundle:13
from /usr/bin/bundle:19:in `load'
from /usr/bin/bundle:19
I fixed it by passing in a build option to the gem install command:
env ARCHFLAGS="-arch x86_64" gem install --no-ri --no-rdoc pg -- --with-pg-config=/usr/local/Cellar/postgresql/9.1.2/bin/pg_config
To make bundler use this option whenever it's installing a gem you can do:
bundle config build.pg --with-pg-config=/usr/local/Cellar/postgresql/9.1.2/bin/pg_config
See build options in the bundler docs for a description of this.
I was writing a Ruby script recently that needed to download 43 2GB chunks of a database backup from a remote source, then decrypt each chunk, then finally concatenate the decrypted files together.
I knew I wanted to use threads to do this as it would speed up the overall process a great deal, and the downloading and decryption can be done in any order, it doesn’t matter if chunk 5 is downloaded before or after chunk 35, and the same with decryption. Those processes all operate on discrete files on the filesystem.
Where order does matter however is when the script is concatenating the files together into the final output file (in this case an lzop archive).
While looking how to handle this I discovered Ruby’s Queue class which “…provides a way to synchronize communication between threads“. Great, that’s exactly what I needed.
In my script I set up two thread-pools, one for downloading and one for decrypting, each with it’s own queue. At the start of the script I push all the download jobs on the download queue. The download thread pool workers download them then push them onto the decrypt queue. The decrypt queue can then get to work. It flows a little like this:
[download queue] -> [download pool] -> [decrypt queue] -> [decrypt pool]
However one last step remained, the concatenation. I used a queue again for this but needed to handle the jobs in order or I would end up with a useless lzop archive, so I came up with the following code to help with this:
You can see from the output that though the work units appear on the queue in any order, they will always be processed in the correct order:
[1.9.2] ~ $ ruby queue.rb
popping the stack
vals is now [16]
popping the stack
vals is now [1, 16]
popping the stack
vals is now [1, 11, 16]
popping the stack
vals is now [1, 11, 16, 19]
popping the stack
vals is now [1, 6, 11, 16, 19]
popping the stack
vals is now [1, 6, 11, 16, 18, 19]
popping the stack
vals is now [1, 6, 11, 15, 16, 18, 19]
popping the stack
vals is now [1, 6, 8, 11, 15, 16, 18, 19]
popping the stack
vals is now [0, 1, 6, 8, 11, 15, 16, 18, 19]
Processing 0
Processing 1
popping the stack
vals is now [5, 6, 8, 11, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 8, 11, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 8, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 8, 10, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 7, 8, 10, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [2, 3, 5, 6, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19]
Processing 2
Processing 3
popping the stack
vals is now [4, 5, 6, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19]
Processing 4
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
Processing 10
Processing 11
popping the stack
vals is now [14, 15, 16, 17, 18, 19]
popping the stack
vals is now [14, 15, 16, 17, 18, 19, 20]
popping the stack
vals is now [13, 14, 15, 16, 17, 18, 19, 20]
popping the stack
vals is now [12, 13, 14, 15, 16, 17, 18, 19, 20]
Processing 12
Processing 13
Processing 14
Processing 15
Processing 16
Processing 17
Processing 18
Processing 19
Processing 208
Processing 19
Processing 20
I recently installed jRuby via RVM and got a weird error:
I tried installing jRuby via brew, and this worked, but I got the same error when installing a gem, so it looked like a Rubygems issue. On a hunch I figured it would be the https entries in my gem sources:
I removed them all and replaced them temporarily with one non-SSL “http://rubygems.org/” entry and I was then able to install jruby-openssl. Once that was installed I deleted the plain old http rubygems.org URL and added back in my SSL URLs. I can now install ruby gems with no error.
Just a little http ping program I wrote to check request latency during my testing for zero-downtime rails deploys with Unicorn.
Example:
pinky:~ will$ ruby httping.rb willj.net
200 : 0.000000 0.000000 0.000000 ( 0.761449)
200 : 0.010000 0.000000 0.010000 ( 0.535888)
200 : 0.000000 0.010000 0.010000 ( 0.800904)
200 : 0.000000 0.000000 0.000000 ( 0.530763)
200 : 0.000000 0.000000 0.000000 ( 0.811362)
200 : 0.000000 0.000000 0.000000 ( 0.557995)
200 : 0.000000 0.010000 0.010000 ( 0.774484)
I am using unicorn with a rails3 app that uses bundler and I hit a weird error when trying to start a server:
`rescue in load_spec_files': git://github.com/odorcicd/authlogic.git (at rails3) is not checked out. Please run `bundle install` (Bundler::GitError)
The background to this is that I am starting unicorn_rails as root then dropping privs in the after_fork block, this is the relevant section from my unicorn.conf file:
after_fork do |server, worker|
# …
uid, gid = Process.euid, Process.egid
user, group = 'will', 'will'
target_uid = Etc.getpwnam(user).uid
target_gid = Etc.getgrnam(group).gid
worker.tmp.chown(target_uid, target_gid)
if uid != target_uid || gid != target_gid
Process.initgroups(user, target_gid)
Process::GID.change_privilege(target_gid)
Process::UID.change_privilege(target_uid)
end
end
This is basically the same as the GitHub unicorn config.
Back to the problem. It works fine for all the standard rubygems.org gems, but was failing on the dependencies I had specified as ‘:git’ deps, for instance authlogic:
gem 'authlogic', :git => 'git://github.com/odorcicd/authlogic.git', :branch => 'rails3'
I checked where these gem/git dependencies were being stored and saw the problem:
[will@server current]$ bundle show activerecord
/usr/local/lib/ruby/gems/1.9.1/gems/activerecord-3.0.0
[will@server current]$ bundle show authlogic
/home/will/.bundler/ruby/1.9.1/authlogic-a087ad0
The :git dependencies were being bundled to a subdirectory of my home dir (the same user that does the deploy, and therefore the initial bundle install. The unicorn_rails process starts as root and has already run Bundler.setup before it forks, so Bundler.setup gets run as root, and the root user has no idea where the :git dependencies are, hence the error.
I solved this by adding a call to Bundler.setup from within the unicorn.conf after_fork block:
after_fork do |server, worker|
Bundler.setup
# …
uid, gid = Process.euid, Process.egid
user, group = 'will', 'will'
target_uid = Etc.getpwnam(user).uid
target_gid = Etc.getgrnam(group).gid
worker.tmp.chown(target_uid, target_gid)
if uid != target_uid || gid != target_gid
Process.initgroups(user, target_gid)
Process::GID.change_privilege(target_gid)
Process::UID.change_privilege(target_uid)
end
end
It seems to work fine, better solutions welcome!
Update: As Jake pointed out in the comments you obviously need zero MQ installed for this example to work. Just ‘brew install zeromq’ or ‘port install zmq’ on OS X, or use your Linux package manager.
I couldn’t find may examples of zeroMQ usage in Ruby so here is a basic sender/consumer I made to test it. First install the ‘zmq’ gem:
gem install zmq --no-ri --no-rdoc
Now start a worker, you can start as many as you want:
Now stick some messages on the queue:
You should get messages distributed to all the worker processes you started up. Pretty simple!
It took me a while to work out how to use map-reduce in Ruby using mongoid so I thought I’d share it here in-case it helps anyone else get there quicker. I start out with a model that includes Mongoid::Document:
To map-reduce across the collection I need to define a map and reduce function in javascript then run the on the collection:
I can roll this into my VisitorSession model:
This obviously makes it easier to call:
>> VisitorSession.first(:conditions => {:project_id => '2f5178'}).visits_for_project
=> 1
I’ve upgraded an app I run that uses warden to Rails3. I started getting “TypeError (can’t convert nil into String)” exceptions after the upgrade:
I tracked it down to the action name not getting set, so I added this in my Warden::Manager.before_failure block:
env['action_dispatch.request.path_parameters'][:action] = "login"
The complete block now:
There may be a better way of doing this, but it works for me.
I upgraded an app to rspec2 and rails3 recently and ran into a problem. I run each test in a transaction which rolls back after each test has finished. This works fine normally, but It seemed that after the upgrade the first failing test would cause this per-test transaction to not get rolled back so all the subsequent tests would run in the same transaction.
This caused the tests so fail with validation errors as test data was getting re-inserted, and a validates_uniqueness_of validation was complaining each time, note the second failing test, this test should have passed but failed with a “Validation failed: Name has already been taken” error:
It turns out that what was actually happening was that an exception raised in the after(:each) block was causing the rollback to fail and the failing test was just masking the error. This was my after(:each) block:
This code fails when there is no directory to remove and though this worked fine in the older version of rspec (rspec just ignored the error) with rspec2 it caused the issues discussed above. The solution (because I don’t care if the file exists or not before I delete it) is as simple as adding a condition:
