Mastodon

Category: ruby

Do or do not. There is no try. Wait, yes there is.

Cruising through Stackoverflow I noticed some syntax that I had never seen before:

@blog.user.try(:username)

Interested, I looked it up. As far as I can tell, it seems that one of the clever folks behind Github posted it as a helpful snippet on his blog.
From there it seems to have made its way into the Rails ActiveSupport library and then into Ruby 1.9 itself. Basically its the same as send except it swallows errors and just returns nil.
An example from the docs:

@person ? @person.name : nil

could be replaced with:

@person.try(:name)

Coolness! I know this will be making its way into my own code shortly. Check out Scott Harvey’s excellent explanation of the try method and also take a look at the documentation here.

Keeping credentials out of your code with environmental vars

I have a project that is up on Github and deployed to Heroku. As I was adding the user credentials file to my project it dawned on me: my application needs these credentials when I run it on Heroku but I don’t want them to be part of my project and therefore visible to everyone on Github.

Turning to the great minds at Stackoverflow, I was advised to use environmental variables.
Since my application is connecting to another site via SFTP I have a hash of all the usual stuff; username password and hostname. It turns out you can just reference environmental variables like this:

@sftp_credentials = {Rails.env.to_sym => {
:sftp_host => ENV[‘SFTP_HOST’],
:sftp_user => ENV[‘SFTP_USER’],
:sftp_password => ENV[‘SFTP_PASSWORD’]
}}

And then make sure you have the environment vars set in your .bashrc so they are available during local development:

export SFTP_HOST=’somesite.com’
export SFTP_USER=’someuser’
export SFTP_PASSWORD=’somepassword’

And then set those variables on Heroku so they are available when your app runs there:

mike@sleepycat:~/projects/myapp$ heroku config:add SFTP_HOST=somesite.com SFTP_USER=someuser SFTP_PASSWORD=somepassword
Adding config vars:
SFTP_HOST => somesite.com
SFTP_PASSWORD => somepassword
SFTP_USER => someuser
Restarting app…done.

no such file to load — json?

While upgrading a project of mine to Rails 3 I stumbled upon a json error. I don’t know what exactly caused it since I have been upgrading from Ubuntu Karmic to Lucid in the past few days. Since this took a little head scratching I thought I would write it down here so I don’t forget. Hopefully it will save someone else some fooling around as well.
The error is coming from the jsvars plugin which is trying to call require ‘json’.

mike@sleepycat:~/projects/todasaulas$ rails s
=> Booting WEBrick
=> Rails 3.0.0 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
Exiting
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:in `require’: no such file to load — json (LoadError)
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:in `require’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:225:in `load_dependency’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:591:in `new_constants_in’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:225:in `load_dependency’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:in `require’
from /home/mike/projects/todasaulas/vendor/plugins/jsvars/lib/jsvars.rb:1

It turns out that json is provided by libjson-ruby on Ubuntu so the fix was as easy as

mike@sleepycat:~$ sudo aptitude install libjson-ruby

I’m not sure why things were working before if that wasn’t there. Who knows all the stuff that gets changed in an upgrade. You learn something new everyday.

Download a zip file and extract using RubyZip

Yesterday I found myself needing to download a zipfile and extract the contents. While this is easy in a shell script, in Ruby its quite aggravating. After some Googling I ended up using RubyZip, but I hope there are better libraries out there. This one doesn’t seem very intuitive. I’ll let the code do the talking here. I’ve added comments so it make sense.
If there is a smoother/less painful|bad way of doing this I’m all ears.

require ‘rubygems’
require ‘open-uri’
require ‘zip/zip’
require ‘fileutils’

def download_zip file_name
url = ‘http://www.example.com/download/zip/’
#the website will drop the connection without the user-agent and other stuff.
open( url + file_name, “User-Agent” => “Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.9) Gecko/20100402 Ubuntu/9.10 (karmic) Firefox/3.5.9”, “From” => “foo@bar.com”, “Referer” => “http://www.foo.bar/”) {|zf|
#zf is an instance of class Tempfile
Zip::ZipFile.open(zf.path) do |zipfile|
#zipfile.class is Zip::ZipFile
zipfile.each{|e|
#e is an instance of Zip::ZipEntry
fpath = File.join(file_name, e.to_s)
FileUtils.mkdir_p(File.dirname(fpath))
#the block is for handling an existing file. returning true will overwrite the files.
zipfile.extract(e, fpath){ true }
}
end
}
end

In my searches I noticed a few mentions about how poor the documentation for this library is. Its a little surprising how difficult it is to come up with one clear example of this. Hopefully this can serve as a starting point for someone and save them some aggravation. Forgive the formatting. :)

Rubygems: Finding all available versions of a gem

Yesterday I was asked if I knew what the highest available version in the Rails 1.x line was. I knew it was going to be a variation of “gem list” but it took a little fiddling to get it right. This is just a little thing but its not something you end up doing often and it was useful so I thought I would write it down here.

Strangely “gem list” accepts a string argument (in this case “rails”) that it uses to search for any gem starting with that string. It would be nice to be able to be more specific than “rails*” but it does the job. So here we go.

Gem list –remote seems to return the curent version only:

mike@sleepycat:~$ gem list –remote rails
rails (2.3.8 ruby)
rails-action-args (0.1.1 ruby)
…

Including –all gets all the released versions:

mike@sleepycat:~$ gem list –remote rails –all

*** REMOTE GEMS ***

rails (2.3.8 ruby, 2.3.7 ruby, 2.3.6 ruby, 2.3.5 ruby, 2.3.4 ruby, 2.3.3 ruby, 2.3.2 ruby, 2.2.3 ruby, 2.2.2 ruby, 2.1.2 ruby, 2.1.1 ruby, 2.1.0 ruby, 2.0.5 ruby, 2.0.4 ruby, 2.0.2 ruby, 2.0.1 ruby, 2.0.0 ruby, 1.2.6 ruby, 1.2.5 ruby, 1.2.4 ruby, 1.2.3 ruby, 1.2.2 ruby, 1.2.1 ruby, 1.2.0 ruby, 1.1.6 ruby, 1.1.5 ruby, 1.1.4 ruby, 1.1.3 ruby, 1.1.2 ruby, 1.1.1 ruby, 1.1.0 ruby, 1.0.0 ruby, 0.14.4 ruby, 0.14.3 ruby, 0.14.2 ruby, 0.14.1 ruby, 0.13.1 ruby, 0.13.0 ruby, 0.12.1 ruby, 0.12.0 ruby, 0.11.1 ruby, 0.11.0 ruby, 0.10.1 ruby, 0.10.0 ruby, 0.9.5 ruby, 0.9.4.1 ruby, 0.9.4 ruby, 0.9.3 ruby, 0.9.2 ruby, 0.9.1 ruby, 0.9.0 ruby, 0.8.5 ruby, 0.8.0 ruby)

And if you wondered, as I did, “Where is Rails 3 in that list?”. You need to include a flag for prereleases:

mike@sleepycat:~$ gem list –remote –prerelease rails –all

rails (3.0.0.beta4 ruby, 3.0.0.beta3 ruby, 3.0.0.beta2 ruby, 3.0.0.beta ruby, 2.3.8 ruby, 2.3.8.pre1 ruby, 2.3.7 ruby, 2.3.6 ruby, 2.3.5 ruby, 2.3.4 ruby, 2.3.3 ruby, 2.3.2 ruby, 2.2.3 ruby, 2.2.2 ruby, 2.1.2 ruby, 2.1.1 ruby, 2.1.0 ruby, 2.0.5 ruby, 2.0.4 ruby, 2.0.2 ruby, 2.0.1 ruby, 2.0.0 ruby, 1.2.6 ruby, 1.2.5 ruby, 1.2.4 ruby, 1.2.3 ruby, 1.2.2 ruby, 1.2.1 ruby, 1.2.0 ruby, 1.1.6 ruby, 1.1.5 ruby, 1.1.4 ruby, 1.1.3 ruby, 1.1.2 ruby, 1.1.1 ruby, 1.1.0 ruby, 1.0.0 ruby, 0.14.4 ruby, 0.14.3 ruby, 0.14.2 ruby, 0.14.1 ruby, 0.13.1 ruby, 0.13.0 ruby, 0.12.1 ruby, 0.12.0 ruby, 0.11.1 ruby, 0.11.0 ruby, 0.10.1 ruby, 0.10.0 ruby, 0.9.5 ruby, 0.9.4.1 ruby, 0.9.4 ruby, 0.9.3 ruby, 0.9.2 ruby, 0.9.1 ruby, 0.9.0 ruby, 0.8.5 ruby, 0.8.0 ruby)

Don’t forget that when the output gets to long (like when you ask for details) you can always redirect it into a file:

mike@sleepycat:~$ gem list –remote –details –prerelease rails –all > gemlist.txt

Like I said… a little thing, but useful to know about.

Rubygems: Installing downloaded gems

I set aside some time this evening to move one of my projects forward. Just as I was getting into it I realised I needed to install some gems. The normally quick “sudo gem install blah” took forever.  After several frustrating minutes, rubygems coughed up the following error:

WARNING:  RubyGems 1.2+ index not found for:
http://gemcutter.org
http://gems.rubyforge.org/

RubyGems will revert to legacy indexes degrading performance.

Degrading performance it certainly was. Both aggravated and curious, I embarked on a flurry of pings, nslookups and GET requests which seem to indicate that Rubygems.org is, in fact, down. Interestingly, Heroku, which hosts rubygems.org was up, reachable and blazing fast. Not wanting to totally waste an evening I figured now was a pretty good time to look into installing gems manually. Luckily it was really easy, but rubygems generally makes it so you don’t need to consider any other options.

The gem I was after offered two options: download as a tar file or a gem file.

Installing from the tar file seems to be a matter of extracting it into /usr/lib/ruby/gems/1.8/gems/ and a quick look at the README file indicated I needed to run setup.rb. That might not hold for all gems but it worked for that one.

Just out of interest I also tried installing from the downloaded gem file. It turns out that its even easier. After downloading the gem file all it took was:

mike@sleepycat:~$ cd Downloads/
mike@sleepycat:~/Downloads$ sudo gem install ruby-openid-2.1.8.gem
Successfully installed ruby-openid-2.1.8
1 gem installed
Installing ri documentation for ruby-openid-2.1.8…
Installing RDoc documentation for ruby-openid-2.1.8…

Not difficult, but not something I had had to do before. Hopefully Rubygems.org will be back up shortly and I won’t have to do again any time soon.I can understand the urge to centralise like Rubygems.org has done with the gem hosting community. The problem is that while you might save some bucks and/or some hassle by centralising, you are multiplying the impact of the inevitable. At least tonight, it doesn’t feel worth it.

Image overlays with Rmagick and Rails

I knew it wouldn’t be long before I had another foray into Rmagick. This time is pretty similar to last time, except this time I needed to overlay one image on top of another image. There are a few people who have posted about how to do this but I found the information pretty sparse. As I was looking I noticed that some people were using the ImageList class and others the Image class. After ping-ponging between the various tutorials and finding the differences between them pretty confusing, I thought I would sort myself out by using both.

So here is the code that generated that stunning image overlay. I tend to figure this stuff out by sticking it in a controller and tinkering  with it. Once it works I will move it into a model. The code below is taken from my controller. First, the version using the Rmagick’s ImageList class:

def rubylist
ruby = ImageList.new(‘public/images/ruby.jpg’)
rails = ImageList.new(‘public/images/rails.png’)
ruby.gravity=EastGravity
ruby.geometry=’0x0+20+20′
rails_on_ruby = ruby.composite_layers(rails, Magick::OverCompositeOp)
rails_on_ruby.format = ‘jpeg’
send_data rails_on_ruby.to_blob, :stream => ‘false’, :filename => ‘test.jpg’, :type => ‘image/jpeg’, :disposition => ‘inline’
end

So what we have are two ImageList objects; ruby and rails. I am going to use ruby as my “destination” ImageList, meaning it will be the base image that  the other image is applied to. Setting the Gravity and Geometry attributes on the “destination” object means that Rmagick will position the “source” image according to those settings. Calling composite_layers on ruby and feeding it rails and a composite operator let Rmagick know what should be combine with the base image and how it should combine them. Since we already gave it the “where” part of the equation with the gravity and geometry, the rest is easy.

Doing the same thing with the Image class instead of ImageList is just different enough to trip you (read: Me) up:

def ruby
ruby = Image.read(‘public/images/ruby.jpg’)[0] #This returns an Array! Get the first element.
rails = Image.read(‘public/images/rails.png’)[0]
rails_on_ruby = ruby.composite(rails, Magick::EastGravity, 0, 0, Magick::OverCompositeOp) #the 0,0 is the x,y
rails_on_ruby.format = ‘jpeg’
send_data rails_on_ruby.to_blob, :stream => ‘false’, :filename => ‘test.jpg’, :type => ‘image/jpeg’, :disposition => ‘inline’
end

Rmagick pretty consistently works in a way that is the opposite of what I expect. Where I would expect Image.read to return, say, an image, instead it returns an Array. So after some head scratching and a little RTFM time I learned you could do what you see above. Alternately you could do “Image.read(‘public/images/rails.png’).first” if you think it’s prettier. The geometry stuff takes a little fooling around to get sorted out and to see how it works with the gravity. My understanding is that the x,y is just to offset the object from where the gravity setting alone would have placed it.

Like the image annotations before, its not difficult to do, but its a pain to figure out. While there are a bunch of other ways to write this code, this will at least give a running start.

Adding text to pictures with Rmagick and Rails.

Recently I needed to add some dynamically generated text to an image with Ruby on Rails. For some reason there do not seem to be many tutorials on how to work with RMagick so I thought I would share what little I now about it. Originally I looked at Image Science but it turns out that its very focused on simply resizing images. So for this I turned to RMagick. I was a little worried that the installation would be nightmarish but it turned out to be quite painless (at least on Ubuntu. I can’t imagine what it would be like on Windows. Probably bad.):

mike@sleepycat:~$ sudo aptitude install imagemagick librmagick-ruby libmagickwand-dev

Then install the gem:

mike@sleepycat:~$ sudo gem install rmagick

RMagick is a complex library that has tonnes of graphics functions and can transform images in many different ways.
It seems that RMagick works conceptually the same for both Vector graphics and bitmap images: create an image and add text objects to it or create a canvas and add vector objects to it (using Ruby Vector Graphics aka:RVG). So lets do that. So here is an image to start with:

To start playing with RMagick, you can stick this in one of your controllers:

require ‘RMagick’
include Magick

def lolcat

img = ImageList.new(‘public/computer-cat.jpg’)
txt = Draw.new

img.annotate(txt, 0,0,0,0, “In ur Railz, annotatin ur picz.”){
txt.gravity = Magick::SouthGravity
txt.pointsize = 25
txt.stroke = ‘#000000’
txt.fill = ‘#ffffff’
txt.font_weight = Magick::BoldWeight
}

img.format = ‘jpeg’
send_data img.to_blob, :stream => ‘false’, :filename => ‘test.jpg’, :type => ‘image/jpeg’, :disposition => ‘inline’

end

And here is the output of that script:

Although sticking that in a controller is a quick way to see the results of some tinkering and experimentation, this is the kind of stuff that should really be in a model. You would likely want to return img.to_blob from one of your model methods and then use send_data from there.

It’s pretty basic but it was all I needed to get my project working. RMagick does seem to be a little arcane. A lot of the things I tried along the way failed in pretty confusing ways and with pretty cryptic errors. While simple stuff like this is fine its going to take a lot more reading and fiddling to make me truly comfortable with RMagick. At least I have this written down for next time…

Postgres and Rails – for MySQL people

A little while ago I decided to switch from MySQL to Postgres for my development database. Although I love how command line friendly MySQL is and think group_concat is the bomb, I’ve been burned my MySQL’s loosey-goosey nature and had it corrupt some data on me. I decided that it was time to switch to Postgres. The transition was a little aggravating at times but it doesn’t take long to get used to it. Installing Postgres is easy enough on Ubuntu:

mike@sleepycat:~$ sudo aptitude install postgresql postgresql-server-dev-8.4

If you need to start at new Rails project you can specify Postgres right from square one:

mike@sleepycat:~/test$ rails myapp -d postgresql

After installing the database I started to look for the gems that provide the driver. I found myself sorting through a bewildering number of gems. “Postgres”, “postgres-pr”, a few minutes search turned up several options each of which proved to be the wrong thing. The gem you need is actually called ruby-pg and is written by a group of people that includes the actual creator of the Ruby language, Yukihiro Matsumoto. Although it is called ruby-pg if you ever want to find it on the internet, you install it using the name pg:

sudo gem install pg

The next stumbling block for me was my assumption that the naming of the Rails adapter would be similar to MySQL. Since adapter: mysql is what I have been using in my project already, I assumed adapter: postgres would be fine. It’s not.

development:
adapter: postgresql
database: myapp_development
username: postgres
password: password
host: localhost

Coming from MySQL, it took a fair bit of reading to collect the Postgres equivalent of all the commands I use most often. Just to save you some looking here is my “Cole’s Notes” version:

Setting a password for the default user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres psql postgres
[sudo] password for mike:
psql (8.4.2)
Type “help” for help.
postgres=# \password postgres
Enter new password:
Enter it again:

Create a database as user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres createdb myapp_development

Drop a database as user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres dropdb myapp_development

Get to the command line of the database myapp_development as user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres psql myapp_development

Load the file ‘pgdump.sql’ into the myapp_development database as the user ‘postgres’:

mike@sleepycat:~/$ sudo -u postgres psql -f pgdump.sql myapp_development

Dump a Postgres database to a file:

mike@sleepycat:~/$ sudo -u postgres pg_dump myapp_development > ~/myapp_dev_dump.sql

Show tables:

postgres=# \dt

show a table:

postgres=# \d tablename

quit:

postgres=# \q

Thats what worked for me. In general I feel like using postgres is forcing me to write more portable code. Pretty much anything I write in Rails that runs in postgres will run on MySQL, but the reverse is really not true. Little things like using ‘1’ for true. MySQL is fine with that but for Postgres with a real boolean datatype, true is true. 1 is not. I stumbled arcoss that when I was using validates_acceptance_of. I feel like its also forcing me to write better SQL as well. If you’ve written a bunch of “find_by_sql” in an existing app you might not appreciate its pickyness though. However once the initial pain of the switch passes, Postgres is pretty great.

Installing memcached gem on Ubuntu Karmic

After my recent fiddling with caching and a little preliminary reading for an interesting side project I’m considering, I decided to install memcached. As will all things on linux, its really easy to install… unless its not. Every time I try to get one of these things going it always takes a while to figure out what packages are needed so the gem can function. So here they are:

sudo aptitude install libmemcached-dev libsasl2-dev libmemcached-dbg

I’m not sure that you need the debug libraries in there but its better to have them and not need them, than need them and not have them. Right?

Now you are ready to install the gem :

sudo gem install memcached –no-rdoc –no-ri

If you miss that libsasl2 library like I did the first time, all manner of badness happens as you can see below. But now you know how to avoid it.

ERROR:  Error installing memcached:
ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb
Libmemcached already built; run ‘rake clean’ first if you need to rebuild.
creating Makefile

make
gcc -I. -I/usr/lib/ruby/1.8/x86_64-linux -I/usr/lib/ruby/1.8/x86_64-linux -I.  -fPIC  -I/usr/lib/ruby/gems/1.8/gems/memcached-0.19.2/ext/include  -L/usr/lib/ruby/gems/1.8/gems/memcached-0.19.2/ext/lib -fno-strict-aliasing -g -g -O2  -fPIC   -fno-strict-aliasing -g -g -O2  -fPIC   -c rlibmemcached_wrap.c

Just so you know, after completely screwing up my first install, and realising that the suggested “rake clean” in the memcached directory was failing as well, I sorted it out by deleting the entire /usr/lib/ruby/gems/1.8/gems/memcached-0.19.2 directory. Very ugly. Doing it right the first time is recommended.

So now that its installed, lets test it to see that everything is ok using the example from the README:

mike@sleepycat:~$ memcached -p 11211 &
[1] 15778
mike@sleepycat:~$ irb
irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ‘memcached’
=> true
irb(main):003:0> $cache = Memcached.new(“localhost:11211″)
=> #<Memcached:0x7feb57cf8b50 @not_found=#<Memcached::NotFound: Memcached::NotFound>, @options={:no_block=>false, :logger=>nil, :ketama_weighted=>true, :retry_timeout=>30, :timeout=>0.25, :auto_eject_hosts=>true, :support_cas=>false, :show_backtraces=>false, :rcv_timeout=>0.25, :buffer_requests=>false, :credentials=>nil, :prefix_delimiter=>””, :sort_hosts=>false, :server_failure_limit=>2, :verify_key=>true, :distribution=>:consistent_ketama, :tcp_nodelay=>false, :hash=>:fnv1_32, :default_ttl=>604800, :use_udp=>false, :cache_lookups=>true, :poll_timeout=>0.25, :default_weight=>8, :binary_protocol=>false, :connect_timeout=>4, :hash_with_prefix_key=>true}, @default_ttl=604800, @servers=[“localhost:11211:8”], @not_stored=#<Memcached::NotStored: Memcached::NotStored>, @struct=#<Rlibmemcached::MemcachedSt:0x7feb57cf8ad8>>
irb(main):004:0> value = ‘hello’
=> “hello”
irb(main):005:0> $cache.set ‘test’, value
=> nil
irb(main):006:0> $cache.get ‘test’
=> “hello”
irb(main):007:0>