mikewilliamson

Postgres and Rails – for MySQL people

A little while ago I decided to switch from MySQL to Postgres for my development database. Although I love how command line friendly MySQL is and think group_concat is the bomb, I’ve been burned my MySQL’s loosey-goosey nature and had it corrupt some data on me. I decided that it was time to switch to Postgres. The transition was a little aggravating at times but it doesn’t take long to get used to it. Installing Postgres is easy enough on Ubuntu:

mike@sleepycat:~$ sudo aptitude install postgresql postgresql-server-dev-8.4

If you need to start at new Rails project you can specify Postgres right from square one:

mike@sleepycat:~/test$ rails myapp -d postgresql

After installing the database I started to look for the gems that provide the driver. I found myself sorting through a bewildering number of gems. “Postgres”, “postgres-pr”, a few minutes search turned up several options each of which proved to be the wrong thing. The gem you need is actually called ruby-pg and is written by a group of people that includes the actual creator of the Ruby language, Yukihiro Matsumoto. Although it is called ruby-pg if you ever want to find it on the internet, you install it using the name pg:

sudo gem install pg

The next stumbling block for me was my assumption that the naming of the Rails adapter would be similar to MySQL. Since adapter: mysql is what I have been using in my project already, I assumed adapter: postgres would be fine. It’s not.

development:
adapter: postgresql
database: myapp_development
username: postgres
password: password
host: localhost

Coming from MySQL, it took a fair bit of reading to collect the Postgres equivalent of all the commands I use most often. Just to save you some looking here is my “Cole’s Notes” version:

Setting a password for the default user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres psql postgres
[sudo] password for mike:
psql (8.4.2)
Type “help” for help.
postgres=# \password postgres
Enter new password:
Enter it again:

Create a database as user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres createdb myapp_development

Drop a database as user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres dropdb myapp_development

Get to the command line of the database myapp_development as user ‘postgres’:

mike@sleepycat:~$ sudo -u postgres psql myapp_development

Load the file ‘pgdump.sql’ into the myapp_development database as the user ‘postgres’:

mike@sleepycat:~/$ sudo -u postgres psql -f pgdump.sql myapp_development

Dump a Postgres database to a file:

mike@sleepycat:~/$ sudo -u postgres pg_dump myapp_development > ~/myapp_dev_dump.sql

Show tables:

postgres=# \dt

show a table:

postgres=# \d tablename

quit:

postgres=# \q

Thats what worked for me. In general I feel like using postgres is forcing me to write more portable code. Pretty much anything I write in Rails that runs in postgres will run on MySQL, but the reverse is really not true. Little things like using ‘1’ for true. MySQL is fine with that but for Postgres with a real boolean datatype, true is true. 1 is not. I stumbled arcoss that when I was using validates_acceptance_of. I feel like its also forcing me to write better SQL as well. If you’ve written a bunch of “find_by_sql” in an existing app you might not appreciate its pickyness though. However once the initial pain of the switch passes, Postgres is pretty great.

How would you design the browser ballot?

“Microsoft’s tying of Internet Explorer to the Windows operating system harms competition between web browsers, undermines product innovation and ultimately reduces consumer choice.”

I recently stumbled across an old post by Mitchell Baker of Mozilla. The post is essentially an offer to the EC to advise on what steps might be appropriate to implement a remedy for Microsoft’s illegal behaviour. As he says “There are separate questions of whether there is a good remedy, and what that remedy might be”. It’s one of those scenarios in which it’s interesting to imagine what you would do if the responsibility for a judgement suddenly (inexplicably) ended up in your lap.

The problem in need of a remedy he nicely sums up as Microsoft “promoted IE through activities that the US Department of Justice and the U.S. Courts determined to be illegal. As result, Internet Explorer ended up with well over 90% market share.”

Its interesting to think through what one might expect from such a remedy. When a wealthy drug dealer is busted their assets are seized and sold since they are the proceeds of crime. The proceeds of crime in this case would be Internet Explorer’s market share. In my head any remedy should be aimed at ensuring the perpetrator does not benefit from their crime, otherwise the incentive is there to commit another crime. The difficulty is that you can’t just confiscate market share.

The remedy that was eventually decided upon was the browser ballot. There were a lot of different ways it could have been implemented but in the end a web page was created that would display the major browsers to the user in random(ish) order. The user would decide what to install from there.

While think I might have offered a similar solution myself in my hypothetical world, I don’t think I agree with the implementation. Displaying a randomly ordered list of browsers gives them all an equal opportunity to be seen but the other browsers a starting at a deficit; to the user the are looking at a list, that has the program they have been using (possibly for years), or a bunch of stuff they have never heard of. An equal chance a being first on the list is not likely to overcome that.

It is a fact that for many users Internet Explorer IS the internet. For many more, even if they are aware of difference between the browser and the website displayed within it, Internet Explorer is the only browser they have ever used. The familiarity that comes with that is also the proceeds of the crime. While the browser ballot is definitely the way it should be, the problem lies in the fact that its the way it ALWAYS should have been.

With Microsoft’s illegal actions creating a bias in the mind of the user, I would be very curious to know what the actual outcome of the browser ballot is. Thinking about it leaves me with more questions than answers:

At what rate are users installing other browsers?

Is this rate sufficient to deprive Microsoft of the “proceeds of the crime”?

If it isn’t, should the ballot favour the other browsers and how much?

If it is, on what timeline?

Is that timeline short enough to feel like an ethical solution to people who have to live with the consequences of this illegal action?

Is this rate high enough to feel like a punishment rather than a market fluctuation?

If the goal is shifting market share, what is the ideal we are shifting it towards? How would you know when to stop?

While there are probably no “right” answers to most of these, just better answers and worse ones. For me the browser ballot falls somewhere in the middle.

What would your remedy look like?

Ubuntu One’s new Music Store

Well I’m not sure why it’s here, but it is. Ubuntu launches its music store with the upcoming release of Lucid Lynx. It’s easy enough to find, as its integrated into Rhythmbox. I was lukewarm on the idea when I heard it mentioned a while back, and after playing with it a little, not much has changed. I hate to rain on the parade because Lucid Lynx is looking like a REALLY good release, but this music store feels really tacked on.

While I was able to find pretty near every obscure artist I searched for, searches for Radiohead and Madonna turned up nothing. Where services like Magnatune and Jamendo definitely reflect the values of the Linux community in the sense that both make their ethical business practices (like how they deal with artists) a centre-piece of their business and offer music in open, unpatented formats (lossless Flac and Ogg Vorbis as well as others), the Ubuntu service seems out of step.

Apart from presumably supporting Ubuntu (a worthy thing in and of itself) you can only guess to whom the money might go when you buy a track. All of the albums I looked at were only available in the patented MP3 format, a very strange choice for a company with such powerful a voice in the FOSS community. Doubly strange when Ubuntu doesn’t even play MP3s by default because of the patents involved.

On top of that, while the quality of the files varied between 320 and 192, the price remained steady at .99 euros. Shouldn’t the price and the quality be connected? My guess is that this is really just an Ubuntu front end on someone else’s music service (it turns out it is). If it isn’t, it sure feels that way. In the end I suppose any revenue stream is a good revenue stream, but in my opinion, Ubuntu really needs to rethink this one and bring it in line both with the values of the community and with the othewise excellent work they are doing in other areas.

Ubuntu Lucid Lynx Beta 2

Beta 2 was posted 5 days ago and I am just getting to giving it a spin. The amount of polish on this release is looking seriously impressive. There are still issues with ACPI when you run it in VirtualBox though. I was assuming those would be ironed out by now.

The UI design team at Ubuntu caused a bit of a kerfuffle when they chose to move the min/max/close buttons to the left side of the application windows. Its a change that I find somewhat annoying but one that it has been decided that will be included in the final release. Given that it is so easy to change back, its really not worth getting upset about. If you don’t like it, slap this in the nearest command line:

mike@sleepycat:~$ gconf-editor

The Gnome configuration editor will come up and under /apps/metacity/general/button_layout you can set the buttons up any way you like. The old setting was menu:minimize,maximize,close.

The new installer is looking really impressive:Over all everything is looking rounder, sleeker and sexier. The new purple colour scheme is really nice. I’m not sure yet what I think of the music store that is included in this version or the App store.

While I am happy to see Ubuntu taking this mainstream, I am a little wary of how well some of these commercial ventures will mix with the underlying philsophy. A big part of what makes Linux generally more secure than Windows is the fact that there is no monetary barrier to updating to the lastest version of whatever software. Can you sell applications without losing that? Not sure yet.

For those like myself running VirtualBox OSE 3.0.8; you will need to get a newer version of the Virtualbox Guest additions iso.

Apples gilded cage

Apple has stirred up a tempest in a teapot with its new restrictions on iPhone development. Their new iPhone OS 4 terms of service bans cross-compiled apps and forces developers to write their apps in C/C++/Objective-C.

Needless to say this has upset quite a few developers, and especially those at Adobe who were busy working on a cross-compiler so that flash apps would run on the iPhone.

While all software reflects the assumptions of the developer, Ruby on Rails is probably the only software that is explicit about being “opinionated”. While the “golden path” in Rails encourages you to construct your database and app following certain conventions people generally think this is a good thing.

Those who strongly disagree live a frustrating life or find another framework. Apple is no less opinionated than Rails. You can see it in every application they build. Music purchased from iTunes or ripped from a purchased CD and works seamlessly with your iPod. The software is designed to encourage exactly that easily monetizable workflow and discourage things that are difficult to monetize.

Want to get those lossless FLAC files you downloaded from PirateBay on your iPod? The average user is in for a painful experience. Stay on the path.

To me, Apples new terms of service really seems like Apple finally asserting a “golden path” for iPhone development, just as they have for digital media with iTunes. The difference between Rails “golden path” and the Apple “golden path” really seems to be one of magnitude. Rails is aimed at a niche, while Apple runs an entire platform. They make the language, the IDE, the OS, most of the software and the hardware. Each of them with their own paths, but collectively comprising a larger path. Surrounded by enough golden paths you will eventually have a gilded cage.

Want to make millions in the App store? Build an app in a way that benefits Apple or piss off and build a web app. Stay on the path.

My suspicion is, the App Store goldrush has brought a bunch of new developers to the Apple platform, ones for whom “Apple or piss off” probably doesn’t feel like anything than can rightfully be called a choice. When you start writing code for a certain platform, you are buying into the whole platform, philosophy and all. You are endorsing and encouraging its use by extending its capabilities. After embracing and extending the closed platform these developers seem offended by Apples move to shut down Adobe’s attempt to open up the platform a little.

To me it seems strange to complain when a restrictive platform restricts you. Apple strictly controls every other aspect of the platform, why not dictate what language developers “originally” write in? Or the tools they use? Or the colour underpants they can wear when coding? In a closed system, why is there even an expectation that these things WOULDN’T be dictated to you?

Considering you can’t even change a battery in an iPod, why would anyone expect Apple to let developers work in a non-Apple language or with a non-Apple tool? Expecting both seems like a big ask. Its a closed platform. After all a developer who has written a few iPhone apps in Flash has gained a lot of experience with Flash and related tools but probably very little about the Apple platform and its tools.

The skill-set a developer gains from repeated exposure to a language or platform is valuable.Very valuable. This is why Microsoft is so heavily involved in schools. Just another kind of vendor lock in. Why would Apple let Adobe (or any other company) prevent developers from tying themselves tightly to Apple?

For better or for worse, a gilded cage is what Apple is offering, both for its users and developers. I don’t want any part of it, but I can’t fault them for doing it. For those don’t mind bars, you really won’t find a nicer cage…

Installing memcached gem on Ubuntu Karmic

After my recent fiddling with caching and a little preliminary reading for an interesting side project I’m considering, I decided to install memcached. As will all things on linux, its really easy to install… unless its not. Every time I try to get one of these things going it always takes a while to figure out what packages are needed so the gem can function. So here they are:

sudo aptitude install libmemcached-dev libsasl2-dev libmemcached-dbg

I’m not sure that you need the debug libraries in there but its better to have them and not need them, than need them and not have them. Right?

Now you are ready to install the gem :

sudo gem install memcached –no-rdoc –no-ri

If you miss that libsasl2 library like I did the first time, all manner of badness happens as you can see below. But now you know how to avoid it.

ERROR: Error installing memcached:
ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb
Libmemcached already built; run ‘rake clean’ first if you need to rebuild.
creating Makefile

make
gcc -I. -I/usr/lib/ruby/1.8/x86_64-linux -I/usr/lib/ruby/1.8/x86_64-linux -I. -fPIC -I/usr/lib/ruby/gems/1.8/gems/memcached-0.19.2/ext/include -L/usr/lib/ruby/gems/1.8/gems/memcached-0.19.2/ext/lib -fno-strict-aliasing -g -g -O2 -fPIC -fno-strict-aliasing -g -g -O2 -fPIC -c rlibmemcached_wrap.c

Just so you know, after completely screwing up my first install, and realising that the suggested “rake clean” in the memcached directory was failing as well, I sorted it out by deleting the entire /usr/lib/ruby/gems/1.8/gems/memcached-0.19.2 directory. Very ugly. Doing it right the first time is recommended.

So now that its installed, lets test it to see that everything is ok using the example from the README:

mike@sleepycat:~$ memcached -p 11211 &
[1] 15778
mike@sleepycat:~$ irb
irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ‘memcached’
=> true
irb(main):003:0> $cache = Memcached.new(“localhost:11211″)
=> #<Memcached:0x7feb57cf8b50 @not_found=#<Memcached::NotFound: Memcached::NotFound>, @options={:no_block=>false, :logger=>nil, :ketama_weighted=>true, :retry_timeout=>30, :timeout=>0.25, :auto_eject_hosts=>true, :support_cas=>false, :show_backtraces=>false, :rcv_timeout=>0.25, :buffer_requests=>false, :credentials=>nil, :prefix_delimiter=>””, :sort_hosts=>false, :server_failure_limit=>2, :verify_key=>true, :distribution=>:consistent_ketama, :tcp_nodelay=>false, :hash=>:fnv1_32, :default_ttl=>604800, :use_udp=>false, :cache_lookups=>true, :poll_timeout=>0.25, :default_weight=>8, :binary_protocol=>false, :connect_timeout=>4, :hash_with_prefix_key=>true}, @default_ttl=604800, @servers=[“localhost:11211:8”], @not_stored=#<Memcached::NotStored: Memcached::NotStored>, @struct=#<Rlibmemcached::MemcachedSt:0x7feb57cf8ad8>>
irb(main):004:0> value = ‘hello’
=> “hello”
irb(main):005:0> $cache.set ‘test’, value
=> nil
irb(main):006:0> $cache.get ‘test’
=> “hello”
irb(main):007:0>

Fun with Ubuntu’s notifications in Ruby

Ubuntu has a pretty snazzy notification system and, as it turns out, its really easy to use it in your scripts. At its core is libnotify, which is already somewhere on each Ubuntu system. To talk to it in your Ruby scripts you need to install the bindings for Ruby and the gem.

sudo aptitude install libinotify-ruby libgtk2-ruby libnotify-dev;

And then:

sudo gem install libnotify

Once thats done, fire up IRB:

irb(main):018:0> require ‘date’
=> true
irb(main):019:0> require ‘rubygems’
=> true
irb(main):020:0> require ‘libnotify’
=> true
irb(main):021:0>Libnotify.show :summary => “#{Date.today}”, :body => “Happy #{Date::DAYNAMES[Date.today.wday]}!”
=> true

The “require ‘date'” is only there because IRB doesn’t seem to include Date (or doesn’t include all of it) for some reason. The result:

How awesome is that?

Dealing with a stale cache at the Rails console

Working on other peoples apps can be tough. In spite of all the “convention” that is supposed to come over configuration, people still do a hell of a lot of configuring. In the end it takes a while figure out what’s what in a new app. Different places to create the same effect, dozens of ways to write equivalent code. After scratching my head for a while over a page that had a bunch of links to various models that all resulted in 404’s I was getting a little frustrated. The production log was no help. After dumping the DB and copying it to my dev database I noticed the problem was gone when I ran the app on my local machine. WTF? Then it hit me: Its cached.

So the task was to clean a stale cache from the console. A little searching turned up these gleaming bits of awesomeness and I’m putting them up here so I never forget them. For a starter on caching you should definitely check out the Railsguide.

This will return the cache directory in case you need to “expire” it manually (defaults to the public directory):

ActionController::Base.page_cache_directory

Or clear a page from the cache with something like this:

ApplicationController.expire_page(:controller => “site”, :action => “index”)

or a fragment:

ActionController::Base.new.expire_fragment(:controller => “my”, :action => “index”)
ActionController::Base.new.expire_fragment(/\/productions.*/)
ActionController::Base.new.expire_fragment(‘sidebar’)

I’ll be remembering that one!

Rails 3 on Heroku

Heroku sent around an email a little while ago talking about their “experimental” Rails 3 support. After a night of tinkering with a Rails 3 project I thought I might try it out. When I pushed my project I got this:

Total 131 (delta 27), reused 0 (delta 0)

—–> Heroku receiving push
—–> Gemfile detected, running Bundler
Unresolved dependencies detected; Installing…
Fetching source index from http://gemcutter.org/
Resolving dependencies
Installing abstract (1.0.0) from system gems
Installing actionmailer (3.0.0.beta) from rubygems repository at http://gemcutter.org/
Installing actionpack (3.0.0.beta) from rubygems repository at http://gemcutter.org/
Installing activemodel (3.0.0.beta) from rubygems repository at http://gemcutter.org/
Installing activerecord (3.0.0.beta) from rubygems repository at http://gemcutter.org/
Installing activeresource (3.0.0.beta) from rubygems repository at http://gemcutter.org/
Installing activesupport (3.0.0.beta) from rubygems repository at http://gemcutter.org/
Installing arel (0.2.1) from rubygems repository at http://gemcutter.org/
Installing builder (2.1.2) from system gems
Installing bundler (0.9.12) from rubygems repository at http://gemcutter.org/ /usr/local/lib/ruby/site_ruby/

1.8/rubygems/installer.rb:192:in `install’: bundler requires RubyGems version >= 1.3.6 (Gem::InstallError)
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/source.rb:42:in `install’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/installer.rb:30:in `run’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/installer.rb:18:in `each’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/installer.rb:18:in `run’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/installer.rb:6:in `install’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/cli.rb:60:in `install’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor/task.rb:33:in `send’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor/task.rb:33:in `run’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor/invocation.rb:109:in `invoke’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor/invocation.rb:116:in `call’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor/invocation.rb:116:in `invoke’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor.rb:137:in `start’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor/base.rb:378:in `start’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/lib/bundler/vendor/thor.rb:124:in `start’
from /usr/local/lib/ruby/gems/1.8/gems/bundler-0.9.9/bin/bundle:11
from /usr/local/bin/bundle:19:in `load’
from /usr/local/bin/bundle:19
FAILED: Have you updated to use a 0.9 Gemfile?

Wha? I had filled out my Gemfile, what could it be? It turns out that you need to specify a “stack” to use when you are running Rails 3. The stack is basically all the a specific configuration of all the supporting software (the OS, Ruby, whatever else). Whatever stack I had been given by default (“aspen”) was not the one that contained Rubygems 1.3.6. Check out their stacks page on the Heroku site for more info.

The fix? Even Though I find Heroku’s documentation is usually pretty bad, this time they had exactly what I needed:

mike@sleepycat:~/projects/myapp$ heroku stack:migrate bamboo-ree-1.8.7
—–> Preparing to migrate
aspen-mri-1.8.6 -> bamboo-ree-1.8.7

NOTE: You must specify ALL gems (including Rails) in manifest

Please read the migration guide:
http://docs.heroku.com/bamboo

—–> Migration prepared.
Run ‘git push heroku master’ to execute migration.
Next time I pushed it worked fine. Yay!

Rails 3 finder syntax

Reading about Rails 3 and all the changes that were made is definitely exciting. Big changes to the ActiveRecord API, revamping routing, a tonne of new stuff. At the same time it felt kind of depressing because at first glance it all looked SO different. Even good changes mean that you have to remember the old stuff to work on your old apps and the new stuff for the new apps. Its not as though there isn’t enough learning involved in this occupation already…

Now that I am starting to play with Rails 3 in some of my personal projects, I am happy to report that there is plenty that is exactly the same in Rails 3 and that even the changes are pretty easy to get used to.

So here you go, some things are the same in Rails 3:

Model.find id

Model.find id, id

Model.all

Model.first

Model.last

Model.find_by_…..

Just avoid Model.find :first and Model.find :all. Especially since this syntax will not just deprecated but removed in Rails 3.2. Pratik from the core team has all the details.

The real changes are really just in how you handle giving options and conditions to the find. In fact the new syntax is very jQuery-like, which you are probably using anyway. It becomes natural very quickly.

Things like:

Model.where “id > 1”

Model.order(“id desc”).limit 1

There is lots more info over at Rails guides. Not so bad really. Now to figure out what they have done with the Routing!