Category Archives: Ruby

Error installing libv8 on Mac OS Catalina

I got an error installing the libv8 gem on Catalina:

$ gem install libv8 -v '3.16.14.19'
Building native extensions. This could take a while...
ERROR: Error installing libv8:
ERROR: Failed to build gem native extension.

current directory: /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/ext/libv8
/Users/will/.rbenv/versions/2.4.5/bin/ruby -r ./siteconf20200217-34155-18bho1v.rb extconf.rb
creating Makefile
Applying /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/patches/disable-building-tests.patch
Applying /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/patches/disable-werror-on-osx.patch
Applying /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/patches/disable-xcode-debugging.patch
Applying /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/patches/do-not-imply-vfp3-and-armv7.patch
Applying /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/patches/do-not-use-MAP_NORESERVE-on-freebsd.patch
Applying /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/patches/do-not-use-vfp2.patch
Applying /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/patches/fPIC-for-static.patch
Compiling v8 for x64
Using python 2.7.16
Using compiler: c++ (clang version 11.0.0)
Unable to find a compiler officially supported by v8.
It is recommended to use GCC v4.4 or higher
Beginning compilation. This will take some time.
Building v8 with env CXX=c++ LINK=c++ /usr/bin/make x64.release ARFLAGS.target=crs werror=no
GYP_GENERATORS=make \
build/gyp/gyp --generator-output="out" build/all.gyp \
-Ibuild/standalone.gypi --depth=. \
-Dv8_target_arch=x64 \
-S.x64 -Dv8_enable_backtrace=1 -Dv8_can_use_vfp2_instructions=true -Darm_fpu=vfpv2 -Dv8_can_use_vfp3_instructions=true -Darm_fpu=vfpv3 -Dwerror=''
CXX(target) /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/vendor/v8/out/x64.release/obj.target/preparser_lib/src/allocation.o
warning: include path for stdlibc++ headers not found; pass '-stdlib=libc++' on the command line to use the libc++ standard library instead [-Wstdlibcxx-not-found]
In file included from ../src/allocation.cc:33:
../src/utils.h:33:10: fatal error: 'climits' file not found
#include
^~~~~~~~~
1 warning and 1 error generated.
make[1]: *** [/Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/vendor/v8/out/x64.release/obj.target/preparser_lib/src/allocation.o] Error 1
make: *** [x64.release] Error 2
/Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/ext/libv8/location.rb:36:in `block in verify_installation!': libv8 did not install properly, expected binary v8 archive '/Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/vendor/v8/out/x64.release/obj.target/tools/gyp/libv8_base.a'to exist, but it was not found (Libv8::Location::Vendor::ArchiveNotFound)
from /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/ext/libv8/location.rb:35:in `each'
from /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/ext/libv8/location.rb:35:in `verify_installation!'
from /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19/ext/libv8/location.rb:26:in `install!'
from extconf.rb:7:in `'

extconf failed, exit code 1

Gem files will remain installed in /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/gems/libv8-3.16.14.19 for inspection.
Results logged to /Users/will/.rbenv/versions/2.4.5/lib/ruby/gems/2.4.0/extensions/x86_64-darwin-19/2.4.0/libv8-3.16.14.19/gem_make.out

The Solution

The solution was as follows:

$ brew install v8@3.15
$ gem install libv8 -v '3.16.14.19' -- --with-system-v8
Building native extensions with: '--with-system-v8'
This could take a while...
Successfully installed libv8-3.16.14.19
1 gem installed

Fixing the ruby ‘CoreFoundation/CFString.h’ file not found install error

I hit this error installing the latest Ruby using rbenv:

$ rbenv install 2.6.3
ruby-build: use openssl from homebrew
Downloading ruby-2.6.3.tar.bz2...
-> https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.3.tar.bz2
Installing ruby-2.6.3...
ruby-build: use readline from homebrew

BUILD FAILED (OS X 10.14.4 using ruby-build 20190423)

Inspect or clean up the working tree at /var/folders/r7/kjzbwmx533b20hcf1_s9kc9c0000gn/T/ruby-build.20190501131413.33977
Results logged to /var/folders/r7/kjzbwmx533b20hcf1_s9kc9c0000gn/T/ruby-build.20190501131413.33977.log

Last 10 log lines:
compiling error.c
compiling eval.c
compiling file.c
compiling gc.c
file.c:23:10: fatal error: 'CoreFoundation/CFString.h' file not found
#include
^~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [file.o] Error 1
make: *** Waiting for unfinished jobs....

The problem is missing headers. To re-install just run this command:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Now, Ruby should install just fine:

$ rbenv install 2.6.3
ruby-build: use openssl from homebrew
Downloading ruby-2.6.3.tar.bz2…
-> https://cache.ruby-lang.org/pub/ruby/2.6/ruby-2.6.3.tar.bz2
Installing ruby-2.6.3…
ruby-build: use readline from homebrew
Installed ruby-2.6.3 to /Users/will/.rbenv/versions/2.6.3

Fixing errors installing the charlock_holmes ruby gem

This gem can be a PITA to install on a mac, this is what worked for me. If you get this error:

charlock_holmes -v '0.7.6'
Building native extensions. This could take a while…
ERROR: Error installing charlock_holmes:
ERROR: Failed to build gem native extension.
current directory: /Users/will/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/charlock_holmes-0.7.6/ext/charlock_holmes
/Users/will/.rbenv/versions/2.6.1/bin/ruby -I /Users/will/.rbenv/versions/2.6.1/lib/ruby/2.6.0 -r ./siteconf20190213-34122-e636v8.rb extconf.rb
checking for -licui18n… yes
checking for -licui18n… yes
checking for unicode/ucnv.h… no
***************************************************************************************
*********** icu required (brew install icu4c or apt-get install libicu-dev) ***********
***************************************************************************************
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
–with-opt-dir
–without-opt-dir
–with-opt-include
–without-opt-include=${opt-dir}/include
–with-opt-lib
–without-opt-lib=${opt-dir}/lib
–with-make-prog
–without-make-prog
–srcdir=.
–curdir
–ruby=/Users/will/.rbenv/versions/2.6.1/bin/$(RUBY_BASE_NAME)
–with-icu-dir
–without-icu-dir
–with-icu-include
–without-icu-include=${icu-dir}/include
–with-icu-lib
–without-icu-lib=${icu-dir}/lib
–with-icui18nlib
–without-icui18nlib
–with-icui18nlib
–without-icui18nlib
To see why this extension failed to compile, please check the mkmf.log which can be found here:
/Users/will/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/extensions/x86_64-darwin-18/2.6.0-static/charlock_holmes-0.7.6/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /Users/will/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/gems/charlock_holmes-0.7.6 for inspection.
Results logged to /Users/will/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/extensions/x86_64-darwin-18/2.6.0-static/charlock_holmes-0.7.6/gem_make.out

view raw
gistfile1.txt
hosted with ❤ by GitHub

First, make sure you have installed the required dependencies:

brew install xz icu4c

Now, install charlock_holmes:

gem install charlock_holmes -v=0.7.6 -- --with-opt-dir=/usr/local/opt --with-opt-include=/usr/local/opt/icu4c/include --with-opt-lib=/usr/local/opt/icu4c/lib --with-cxxflags=-std=c++11

How to find if two nodes are connected in an RGL graph

Say you have a graph like this:

graph

How do you find out if there is a path between any of the two nodes? By using a breadth-first search:

require 'rgl/implicit'
require 'rgl/traversal'
vertices = ["one", "two", "three"]
g = RGL::ImplicitGraph.new do |g|
g.vertex_iterator { |b| vertices.map{|v| b.call(v) } }
g.adjacent_iterator { |x, b| b.call( vertices[(vertices.index(x)1).abs] ) }
g.directed = true
end
t = g.bfs_search_tree_from("one")
puts t.has_vertex?("two") # true
puts t.has_vertex?("three") # false

view raw
graph.rb
hosted with ❤ by GitHub

Tagged , ,

First robot remote driving test

I programmed some remote control software using a Golang receiving program on the robot and a ruby control client using my gamepad ruby gem and an xbox1 controller. It worked OK. It was a bit jerky, there’s no PWM so no acceleration, it’s either go or stop; anything not totally rigid on the robot wobbles. Also the position of the camera doesn’t show enough of the robot so it’s hard to get a real idea of where the robot is.

I was filming, the robot was being controlled my my wife, Morwenna, from upstairs.

The robot is also prone to shed a track if the “half turn” is used too much, that is one track forwards or backwards, the other one stationary. I can fix this in software if I can work out a way to do PWM on the robot that doesn’t run the Raspberry Pi CPU.

Tagged ,

Railsberry animated gifs

I put up some photos of Railsberry here, but Flickr doesn’t work with animated gifs, so here they are. Click the images to get the animated versions.

Riding the Railsberry Unicorn:

Jon Leighton getting closer:

Josh Kalderimis misbehaving:

Tagged

Error installing the pg gem on Lion

I got the following error installing the latest pg gem on Lion:

Installing pg (0.12.2) with native extensions Unfortunately, a fatal error has occurred. Please report this error to the Bundler issue tracker at https://github.com/carlhuda/bundler/issues so that we can fix it. Thanks!
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:482:in `build_extensions': ERROR: Failed to build gem native extension. (Gem::Installer::ExtensionBuildError)

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb 
checking for pg_config... yes
Using config values from /usr/local/bin/pg_config
checking for libpq-fe.h... yes
checking for libpq/libpq-fs.h... yes
checking for PQconnectdb() in -lpq... no
checking for PQconnectdb() in -llibpq... no
checking for PQconnectdb() in -lms/libpq... no
Can't find the PostgreSQL client library (libpq)
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby
	--with-pg
	--without-pg
	--with-pg-dir
	--without-pg-dir
	--with-pg-include
	--without-pg-include=${pg-dir}/include
	--with-pg-lib
	--without-pg-lib=${pg-dir}/lib
	--with-pg-config
	--without-pg-config
	--with-pg_config
	--without-pg_config
	--with-pqlib
	--without-pqlib
	--with-libpqlib
	--without-libpqlib
	--with-ms/libpqlib
	--without-ms/libpqlib


Gem files will remain installed in /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/pg-0.12.2 for inspection.
Results logged to /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/pg-0.12.2/ext/gem_make.out
	from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:445:in `each'
	from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:445:in `build_extensions'
	from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/installer.rb:197:in `install'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/source.rb:90:in `install'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/rubygems_integration.rb:82:in `preserve_paths'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/source.rb:89:in `install'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:73:in `install_gem_from_spec'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/rubygems_integration.rb:97:in `with_build_args'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:72:in `install_gem_from_spec'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:56:in `run'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:55:in `run'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/installer.rb:12:in `install'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/cli.rb:220:in `install'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/task.rb:22:in `send'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/task.rb:22:in `run'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/invocation.rb:118:in `invoke_task'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor.rb:263:in `dispatch'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/lib/bundler/vendor/thor/base.rb:386:in `start'
	from /Users/will/.rvm/gems/ruby-1.9.3-p0@status/gems/bundler-1.1.rc.7/bin/bundle:13
	from /usr/bin/bundle:19:in `load'
	from /usr/bin/bundle:19

I fixed it by passing in a build option to the gem install command:

env ARCHFLAGS="-arch x86_64" gem install --no-ri --no-rdoc pg -- --with-pg-config=/usr/local/Cellar/postgresql/9.1.2/bin/pg_config

To make bundler use this option whenever it's installing a gem you can do:

bundle config build.pg --with-pg-config=/usr/local/Cellar/postgresql/9.1.2/bin/pg_config

See build options in the bundler docs for a description of this.

Ruby’s Queue class, and ordered processing

I was writing a Ruby script recently that needed to download 43 2GB chunks of a database backup from a remote source, then decrypt each chunk, then finally concatenate the decrypted files together.

I knew I wanted to use threads to do this as it would speed up the overall process a great deal, and the downloading and decryption can be done in any order, it doesn’t matter if chunk 5 is downloaded before or after chunk 35, and the same with decryption. Those processes all operate on discrete files on the filesystem.

Where order does matter however is when the script is concatenating the files together into the final output file (in this case an lzop archive).

While looking how to handle this I discovered Ruby’s Queue class which “…provides a way to synchronize communication between threads“. Great, that’s exactly what I needed.

In my script I set up two thread-pools, one for downloading and one for decrypting, each with it’s own queue. At the start of the script I push all the download jobs on the download queue. The download thread pool workers download them then push them onto the decrypt queue. The decrypt queue can then get to work. It flows a little like this:

[download queue] -> [download pool] -> [decrypt queue] -> [decrypt pool]

However one last step remained, the concatenation. I used a queue again for this but needed to handle the jobs in order or I would end up with a useless lzop archive, so I came up with the following code to help with this:

You can see from the output that though the work units appear on the queue in any order, they will always be processed in the correct order:

[1.9.2] ~ $ ruby queue.rb
popping the stack
vals is now [16]
popping the stack
vals is now [1, 16]
popping the stack
vals is now [1, 11, 16]
popping the stack
vals is now [1, 11, 16, 19]
popping the stack
vals is now [1, 6, 11, 16, 19]
popping the stack
vals is now [1, 6, 11, 16, 18, 19]
popping the stack
vals is now [1, 6, 11, 15, 16, 18, 19]
popping the stack
vals is now [1, 6, 8, 11, 15, 16, 18, 19]
popping the stack
vals is now [0, 1, 6, 8, 11, 15, 16, 18, 19]
Processing 0
Processing 1
popping the stack
vals is now [5, 6, 8, 11, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 8, 11, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 8, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 8, 10, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 7, 8, 10, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [3, 5, 6, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19]
popping the stack
vals is now [2, 3, 5, 6, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19]
Processing 2
Processing 3
popping the stack
vals is now [4, 5, 6, 7, 8, 9, 10, 11, 14, 15, 16, 18, 19]
Processing 4
Processing 5
Processing 6
Processing 7
Processing 8
Processing 9
Processing 10
Processing 11
popping the stack
vals is now [14, 15, 16, 17, 18, 19]
popping the stack
vals is now [14, 15, 16, 17, 18, 19, 20]
popping the stack
vals is now [13, 14, 15, 16, 17, 18, 19, 20]
popping the stack
vals is now [12, 13, 14, 15, 16, 17, 18, 19, 20]
Processing 12
Processing 13
Processing 14
Processing 15
Processing 16
Processing 17
Processing 18
Processing 19
Processing 208
Processing 19
Processing 20

The code to do something similar:

require 'thread'
LAST_VAL = 8
q = Queue.new
(0..LAST_VAL).to_a.shuffle.each {|i| q << i}
vals = []
next_val = 0
loop do
puts "popping the stack"
vals << q.pop
vals.sort!
puts "vals is now #{vals.inspect}"
while vals[0] == next_val
puts "Processing #{vals.shift}"
next_val += 1
end
break if next_val > LAST_VAL
end

Tagged

LoadError: OpenSSL::SSL requires the jruby-openssl gem

I recently installed jRuby via RVM and got a weird error:

I tried installing jRuby via brew, and this worked, but I got the same error when installing a gem, so it looked like a Rubygems issue. On a hunch I figured it would be the https entries in my gem sources:

I removed them all and replaced them temporarily with one non-SSL “http://rubygems.org/&#8221; entry and I was then able to install jruby-openssl. Once that was installed I deleted the plain old http rubygems.org URL and added back in my SSL URLs. I can now install ruby gems with no error.

Tagged ,

Simple http ping program in Ruby

Just a little http ping program I wrote to check request latency during my testing for zero-downtime rails deploys with Unicorn.

Example:

pinky:~ will$ ruby httping.rb willj.net
200 :   0.000000   0.000000   0.000000 (  0.761449)
200 :   0.010000   0.000000   0.010000 (  0.535888)
200 :   0.000000   0.010000   0.010000 (  0.800904)
200 :   0.000000   0.000000   0.000000 (  0.530763)
200 :   0.000000   0.000000   0.000000 (  0.811362)
200 :   0.000000   0.000000   0.000000 (  0.557995)
200 :   0.000000   0.010000   0.010000 (  0.774484)

Tagged