Mastodon

Splitting Flac files on Ubuntu

There have been a few occasions where I have found myself with an album saved as a single flac file, a cue file and no idea what to do next. Well it turns out that splitting flac files according to a cue sheet is a pretty easy thing once you have the right software. The right software in this case is the rather unlikely-to-ever-turn-up-in-your-google-search program gCue2tracks.

Splitting files with gCue2tracks

Its simple enough that it really doesn’t warrant any explanation. Simply point it to the cue file and click convert. It works on flac, ape and several other files. An easy to install deb file can be found here. If it complains about needing “oggenc” when you try to use it just run “sudo apt-get install vorbis-tools” and it will install the ogg encoder gCue2Tracks is looking for. Likewise if you ask it to save as flac it may complain that it can’t find “flac”… the solution is “sudo apt-get install flac”.

For the moderately more adventurous, pulling the current version from the repository is also an option. It will get you the current version (5.2) but there are no any significant changes since 2010. Its easy to do in any case:

sudo apt-get install mercurial #so you have the hg command installed
hg clone http://trac-hg.assembla.com/gCue2tracks/
cd gCue2tracks
./gCue2tracks.py

Of course if you run it like that, make sure you have installed its dependencies with “sudo apt-get install cuetools shntool libav-tools”.

Scripting MySQL database setup for Rails with Bash

I am working a bash script to set up a complicated application. As part of that setup I need to run a series of SQL commands to create the initial users and databases this app will require.

It turns out you can get mysql to execute arbitrary statements from the command line using the -e switch but that would be both ugly and tedious for a lot of statements. Sounds like a job for heredocs! Lets try it out:

mike@railsdev:~$ read -d ” test <<‘EOT’
create database test;
select 1;
EOT

mike@railsdev:~$ mysql -u root -ppassword -e “$test”
+—+
| 1 |
+—+
| 1 |
+—+

Perfect! I love it when things work the way you expect. On to useful stuff:

read -d ” MYSQL_SETUP <<'EOF'
create database myapp_test;
create database myapp_development;
create database myapp_production;
CREATE USER 'appuser'@'localhost' IDENTIFIED BY 'myapp_password';
grant all privileges on myapp_production.* to appuser@localhost;
grant all privileges on myapp_test.* to appuser@localhost;
grant all privileges on myapp_development.* to appuser@localhost;
EOF

mysql -u root -ppassword -e "$MYSQL_SETUP"

And then use the same trick to write a nice new database.yml for the app to use those credentials:

read -d ” DB_CREDENTIALS <<'EOF'
production:
adapter: mysql
database: myapp_production
pool: 5
username: appuser
password: myapp_password

development:
adapter: mysql
database: myapp_development
pool: 5
username: appuser
password: myapp_password

test:
adapter: mysql
database: myapp_test
pool: 5
username: appuser
password: myapp_password
EOF

read -d '' DB_CREDENTIALS < /var/www/myapp/config/database.yml

Creating Rails users in Postgres on Ubuntu

Note: Since writing this I have revisited this issue and found a better way. read about it here.

I am getting started with a new greenfield Rails app and had a bit of an adventure setting up a user for my Rails app in Postgres. My attempts to connect or create a database would fail. I thought I had created a user properly but apparently I was wrong.
Now I know for next time.
Start with the create user command:


mike@sleepycat:~$ sudo -u postgres createuser --createdb --pwprompt desired_username

You can see we are using the createdb option in this call, allowing the user we are creating to create new databases. You can take a look at the databases available and the users that can access them with the command:


sudo -u postgres psql -l

So now lets assign the user we created rights on the database we just created with:


mike@sleepycat:~$ sudo -u postgres psql my_project_development -c "grant all privileges on database my_project_development to desired_username;"
GRANT

I had hoped that might be all that was needed but then my rake task crapped out:


mike@sleepycat:~$ rake db:create

Couldn't create database for {"encoding"=>"unicode", "username"=>"desired_username", "adapter"=>"postgresql", "database"=>"my_project_test", "pool"=>5, "password"=>nil}
FATAL:  Ident authentication failed for user "desired_username"

It turns out that Postgres is set to expect the user accessing the database to actually have a system account. That seems like a little much for a development environment where I would end up with tonnes of those accounts. To change that expectation you need to edit the host-based authentication configuration file pg_hba.conf:


sudo vim /etc/postgresql/8.4/main/pg_hba.conf

Press capital G to jump to the bottom of the file and change “local all postgres ident” and “local all all ident” to md5 as shown below:


# DO NOT DISABLE!
# If you change this first entry you will need to make sure that the
# database super user can access the database using some other method.
# Noninteractive access to all databases is required during automatic maintenance
#(custom daily cronjobs, replication, and similar tasks).
# Database administrative login by UNIX sockets
local   all         postgres         md5

# TYPE  DATABASE    USER        CIDR-ADDRESS      METHOD
# "local" is for Unix domain socket connections only

local   all         all         md5
# IPv4 local connections:
host    all         all         127.0.0.1/32    md5
# IPv6 local connections:
host    all         all         ::1/128       md5

Restart your Postgres server and you should be good to go.


mike@sleepycat:~/projects/key_master$ sudo /etc/init.d/postgresql-8.4 restart
* Restarting PostgreSQL 8.4 database server
...done.

Now your Rails users don’t need to have system accounts and your rake tasks should run just fine. Back to work!

Thanks to Mark Berry for his comments below! They have been incorporated into the post now.

Setting CLI resolution in Ubuntu

Setting the resolution for the console is not much of a concern normally. Your virtual terminals are set to a reasonable resolution by default on a normal Ubuntu desktop installation. If you run Ubuntu server in a virtual machine, however, you are stuck with 640×480 resolution.

Recent versions of Ubuntu all use Grub2 so the way to change the resolution is now:

sudo vi /etc/default/grub

Then add the following:

GRUB_GFXMODE=1024×768
GRUB_GFXPAYLOAD_LINUX=keep

Save the file and run:

sudo update-grub

On your next reboot, there it will be.

Mobile Apps and the “Death” of the Web

Wired magazine’s latest issue ran with the rather flame-baity cover story “The Web is Dead”.
Their interpretation of data from Cisco, is that web pages are down to just 23% of the total volume of data being transferred over the internet and therefore the web is dying. While I have some serious doubts that you can make such a pronouncement when a 5 kilobit webpage can serve potentially infinite megs/gigs of HD video, I’ll let others debate the numbers. Their central argument is that single purpose Apps are replacing the browser and essentially killing the idea of the open internet.

“Mobile traffic tends to be driven by specialty software, mostly apps, designed for a single purpose. For the sake of the optimized experience on mobile devices, users forgo the general-purpose browser.”
Chris Anderson – The Web Is Dead. Long Live the Internet – Wired Magazine

This is the part the interests me. In the process of populating a user generated content site that I created, I have seen A LOT of websites recently. Beyond just seeing them, I was looking for some specific information on them; upcoming event, event date, address and contact email. It has been a bit of a shock how difficult it was to find basic information like that on most websites. Reflecting on what I have seen, it seems that the embrace of single purpose apps is less an abandonment of the open web and more an indictment of the difficulty of extracting up-to-date, usable information out of it. At least a single purpose app HAS a purpose.
The purpose of a website in my experience is threefold:

1.To be found. If Google can’t read it, people can’t find it, so its useless. Stop with text in images. Stop using Flash.
2. To show signs of life. There should be some indication of recent activity and an indication of how recent. This is the web, not the cobweb.
3. To provide basic info and facilitate key tasks. It should be obvious how/where I can do what I came to do. Selling tickets? You better have an obvious “Buy tickets” button somewhere. What’s your address? Where can people reach you?

A majority of websites fail at at least two of those 3 items and leave the user wondering why the site exists at all. Even when you hit all three at least one or two are more difficult than they should be. Maybe they tell you their address, but it might be with text as part of an image, so you can’t copy and paste it into Google Maps or an email. Maybe there is a “contact us” section that has a form you have to fill out, instead of just giving an email address which will stay forever in my contact list. Maybe the link to buy tickets takes four clicks to even get to. All of these things seriously frustrate users and make it clear that the site doesn’t have any clear purpose at all. In my mind this is exactly the sort of thing pushing people to abandon the browser for an app which does have a clear purpose. The user knows what to do and is happier with it, which the article seems to acknowledge:

“Every time you pick an iPhone app instead of a Web site, you are voting with your finger: A better experience is worth paying for, either in cash or in implicit acceptance of a non-Web standard.”
Chris Anderson – The Web Is Dead. Long Live the Internet – Wired Magazine

Computers are complicated. Even with more than a decade of mainstream computer use behind us, the average user knows as little as they ever did. In frustration they are reaching for the simplicity of apps that know their purpose. Your site is not an obligation, its a necessity. Design it with purpose. Don’t let your site be one of the ones that is driving people away from the internet.

Wired Magazine – The Web Is Dead. Long Live the Internet

no such file to load — json?

While upgrading a project of mine to Rails 3 I stumbled upon a json error. I don’t know what exactly caused it since I have been upgrading from Ubuntu Karmic to Lucid in the past few days. Since this took a little head scratching I thought I would write it down here so I don’t forget. Hopefully it will save someone else some fooling around as well.
The error is coming from the jsvars plugin which is trying to call require ‘json’.

mike@sleepycat:~/projects/todasaulas$ rails s
=> Booting WEBrick
=> Rails 3.0.0 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
Exiting
/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:in `require’: no such file to load — json (LoadError)
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:in `require’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:225:in `load_dependency’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:591:in `new_constants_in’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:225:in `load_dependency’
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:in `require’
from /home/mike/projects/todasaulas/vendor/plugins/jsvars/lib/jsvars.rb:1

It turns out that json is provided by libjson-ruby on Ubuntu so the fix was as easy as

mike@sleepycat:~$ sudo aptitude install libjson-ruby

I’m not sure why things were working before if that wasn’t there. Who knows all the stuff that gets changed in an upgrade. You learn something new everyday.

Rendering templates in Rails Metal

One of the things that has caught my interest lately is Rack. After reading about it a bunch I was looking for a way to start integrating it into my projects somehow. Adam Wiggins of Heroku, had an interesting presentation that called Rails Metal the “a gateway to the world of Rack.” Its a good description, and one that gave me a starting place for using Rack in my project.

If you run the “script/generate metal nameofMetalApp” command, Rails will generate the code for a small Rack app and dump it in the app directory of your project in a folder called “metal”. Inside that app, there are no Rails helpers, no convenience methods, no redirects or renders. You really are in a new world.

The reason for that is that Rails won’t have been loaded when the a request hits this little app. Its only if the apps in the metal folder all return 404 that Rails gets loaded at all.
What is gained here is speed. If a couple lines of code is sufficient to handle the request, why load an entire framework? For simple actions that happen a lot, it can make a big difference. The first thing I wanted to know was how to render HAML or ERB pages in my Metal apps.

Since that is probably pretty common I thought I would post it up here. Its not complicated but it will save some searching. I decided I would create a Metal end point that would serve up a static “about” page. There are many useful things to do with Metal, and while an “about” page might not be one of them, its a good place to start:

# Allow the metal piece to run in isolation
require(File.dirname(__FILE__) + &quot;/../../config/environment&quot;) unless defined?(Rails)
#added HAML
require 'haml'
class AboutApp
  def self.call(env)
    if env[&quot;PATH_INFO&quot;] =~ /^\/about/
      @time = Time.now
      #Render a HAML template using the HAML engine
      #http://haml-lang.com/docs/yardoc/Haml/Engine.html
      template = File.read(File.join(&quot;app&quot;,&quot;views&quot;, &quot;static&quot;, &quot;about.html.haml&quot;))
      haml_engine = Haml::Engine.new(template)
      output = haml_engine.render(binding)
      #Alternately you can render an ERB template
      #http://ruby-doc.org/stdlib/libdoc/erb/rdoc/classes/ERB.html
      #template = File.read(File.join(&quot;app&quot;, &quot;views&quot;, &quot;static&quot;, &quot;about.html.erb&quot;))
      #output = ERB.new(template).result(binding)
      [200, {&quot;Content-Type&quot; =&gt; &quot;text/html&quot;}, [&quot;#{output}&quot;]]
    else
      [404, {&quot;Content-Type&quot; =&gt; &quot;text/html&quot;}, [&quot;Not Found&quot;]]
    end
  end
end

Notice that I am creating an instance variable @time which is available in the templates because I am passing in binding objects to the renderer for both HAML and ERB.
And a basic template so we can see that its working (excuse the lack of spacing, wordpress is messing it up):

!!!
%html
  %body
    %h1 The time is
    %p
      =@time

Then fire up your server and request “/about”. That’s it. The thing to remember is that these Metal apps are real Rack apps. While most of them will be pretty basic, they don’t have to be. They just have to conform to the Rack spec. I have been curious about Sinatra for a while now and this seemed like a good time to try it out and see if I could use it instead of Metal. I wrote a quick little Sinatra app and dropped it into the metal folder, requested “/test” and there it was. (Again, excuse the lack of spacing)

require 'rubygems'
require 'sinatra'
require 'haml'
class TestApp &lt; Sinatra::Application
  get '/test' do
    output =&lt;&lt;-EOHAML
    !!!
    %html
      %body
        %h1
          Hello from Sinatra!
    EOHAML
    haml output
  end
end

While the purpose of all this is performance, it also seems that this is a fairly nice way to add other apps to your existing one. Say, if I wanted to run a basic CMS alongside one of my Rails apps for marketing purposes. What’s extra cool is that using a Rack based authentication middleware like Warden would mean that both apps would share an authentication mechanism. There are a lot of possibilities to explore!

Download a zip file and extract using RubyZip

Yesterday I found myself needing to download a zipfile and extract the contents. While this is easy in a shell script, in Ruby its quite aggravating. After some Googling I ended up using RubyZip, but I hope there are better libraries out there. This one doesn’t seem very intuitive. I’ll let the code do the talking here. I’ve added comments so it make sense.
If there is a smoother/less painful|bad way of doing this I’m all ears.

require ‘rubygems’
require ‘open-uri’
require ‘zip/zip’
require ‘fileutils’

def download_zip file_name
url = ‘http://www.example.com/download/zip/&#8217;
#the website will drop the connection without the user-agent and other stuff.
open( url + file_name, “User-Agent” => “Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.9) Gecko/20100402 Ubuntu/9.10 (karmic) Firefox/3.5.9”, “From” => “foo@bar.com”, “Referer” => “http://www.foo.bar/&#8221;) {|zf|
#zf is an instance of class Tempfile
Zip::ZipFile.open(zf.path) do |zipfile|
#zipfile.class is Zip::ZipFile
zipfile.each{|e|
#e is an instance of Zip::ZipEntry
fpath = File.join(file_name, e.to_s)
FileUtils.mkdir_p(File.dirname(fpath))
#the block is for handling an existing file. returning true will overwrite the files.
zipfile.extract(e, fpath){ true }
}
end
}
end

In my searches I noticed a few mentions about how poor the documentation for this library is. Its a little surprising how difficult it is to come up with one clear example of this. Hopefully this can serve as a starting point for someone and save them some aggravation. Forgive the formatting. :)

Rails has_and_belongs_to_many

I had a pretty zealous database design teacher pounded into us at school that if many to many relationships are coming up a lot in your design, then its probably a bad design. That still seems like solid advice to me but there are definitely times when its totally legit to use them. Tags, categories, roles. That sort of thing. Its been a while since I did one and I stumbled through doing one just now. Because of that I thought I would write it up for next time I need to have my memory jogged but also because I remember struggling with relations when I started doing Rails. So here goes.

I have an “Account” model and a “Role” model and want an account to have a habtm relation with roles. I started with this in my migration (which turned out to be wrong):

create_table :accounts_roles do |t|
t.integer :account_id
t.integer :role_id
end

Which produces this in the database:

mysql> desc accounts_roles;
+————+———+——+—–+———+—————-+
| Field      | Type    | Null | Key | Default | Extra          |
+————+———+——+—–+———+—————-+
| id         | int(11) | NO   | PRI | NULL    | auto_increment |
| account_id | int(11) | YES  |     | NULL    |                |
| role_id    | int(11) | YES  |     | NULL    |                |
+————+———+——+—–+———+—————-+
3 rows in set (0.00 sec)

But the moment you try to save any associate a role with an account I got this:
ActiveRecord::ConfigurationError: Primary key is not allowed in a has_and_belongs_to_many join table (accounts_roles).
Which is actually one of the more helpful error messages I have seen, so thank you to whomever created it. A quick “rake db:rollback” and modify that migration:

create_table :accounts_roles, :id => false do |t|
t.integer :account_id
t.integer :role_id
end

“rake db:migrate” again and there we go. Now this is looking better:

mysql> desc accounts_roles;
+————+———+——+—–+———+——-+
| Field      | Type    | Null | Key | Default | Extra |
+————+———+——+—–+———+——-+
| account_id | int(11) | YES  |     | NULL    |       |
| role_id    | int(11) | YES  |     | NULL    |       |
+————+———+——+—–+———+——-+
2 rows in set (0.00 sec)

And don’t forget to add the relation to your models:

class Role < ActiveRecord::Base
has_and_belongs_to_many :accounts
end

class Account < ActiveRecord::Base
has_and_belongs_to_many :roles
end

Then you can save stuff in there using something like this:

>> me.roles.create :name => ‘admin’
=> #<Role id: 1, name: “admin”, created_at: “2010-07-02 20:31:01”, updated_at: “2010-07-02 20:31:01”>

Note that create implicitly saves the record so it already has a role id. The << operator works the same way so this would be equivalent:

me.roles<<Role.new(:name => ‘commenter’)

Build will add a new role object in there but won’t save it automatically so there won’t be a role id assigned until it is saved.

>> me.roles.build
=> #<Role id: nil, name: nil, created_at: nil, updated_at: nil>

That’s it!