mikewilliamson

Installing Ubuntu on a Dell Inspiron 1300

I was recently given an old Dell Inspiron 1300 that had been replaced with one of Dell’s shiny new laptops. The battery life is of course nearly non-existent but functionally it is still in great shape. I figure all it needs is a little love and I will find it a new home.
Installing Ubuntu is so painless that I was very surprised to find that when I was booting the machine from a usb stick (made with usb-creator and a 12.04 iso) it would hang while displaying the loading screen. A quick googling turned up an answer by the ever-helpful Jorge Castro on Ask Ubuntu. His advice; press F6 and add this to grub’s boot parameters:

b43.blacklist=yes

The trouble is the Broadcom wireless BCM4318 LAN card used in this model. The linuxwireless site describes this situation like this:

The Broadcom wireless chip needs proprietary software (called “firmware”) that runs on the wireless chip itself to work properly. This firmware is copyrighted by Broadcom and must be extracted from Broadcom’s proprietary drivers. To get such firmware on your system, you must download the driver from a legal distribution point, extract it, and install it.

So while blacklisting the b43 driver will get the system to boot to the USB stick and get to the installer, it doesn’t solve the problem. I plugged in an ethernet cable and made my way through the rest of the install process as normal. I needed to repeat Jorge Castro’s trick again to boot the laptop after the install. Once it was running and connected to the internet via my ethernet cable I was able to install the firmware by installing the package mentioned on the linuxwireless site:

sudo apt-get install firmware-b43-installer

After that I rebooted again and was thrilled to see that it came up without a fuss. Thinking it was now solved I looked at the network menu only to find yet another headscratcher. The message there: “wireless is disabled by hardware switch”. WTF?
It didn’t take long to figure out was actually going on; FN+F2 had been pressed at some point disabling the wireless entirely. FN+F2 one more time and suddenly wireless was working.

Removing restrictions from PDFs

My girlfriend recently sent me a PDF that she needed to print. She had tried to strip off the printing restrictions using the site I made for that purpose but it hadn’t worked. I quickly realised that my site, which is really a web front end on Ghostscript would not be able to deal with that particular PDF because it was encrypted.

Interestingly, it seems that the Ghostscript project has been making steady progress since I first made that site and can now remove encryption as well as the usual usability frustrations my girlfriend was dealing with; printing and copy/paste restrictions. After a bit of an update it is now cheerfully removing encryption too!

So in the spirit of “do things, tell people”, here is where to go next time you have a difference of opinion with a PDF file: http://pdf-permissions-remover.com/

If you are interested in the code you can find it on Github: https://github.com/sleepycat/pdfstripper

OsX annoyance

Whenever you talk User Interface with someone, somehow the Mac ends up being held up as a shining example of usability. For some reason I find the Mac deeply annoying. Take the “downloaded from the internet” warning for instance. I don’t stray from terminal very often, but a couple of times a week I launch MacVim and get this:

I’ve launched this app about twice a week for about a year, easily around a hundred times. Apparently OS X is capable of tracking when and how this app landed on my hard drive but not that I use it pretty frequently. Would it not be a reasonable thing to have some type of “stop asking me this question” option? Mozilla and many others have pulled this off so I am pretty sure it doesn’t require the GDP of Sweden to get the job done. In the vain hope that that the (?) might offer some helpful help (as opposed to the usual kind) I clicked and got this:

I can see how this self serving warning against “unlicensed software” is helpful to Apple. Do you suppose there is an option to specify who the help is supposed to be helpful to somewhere?

It turns out that you can drop to the terminal and enter:

xattr -d -r com.apple.quarantine ~/Downloads

People constantly criticise Linux for this sort of thing, but for some reason Apple can do the same and somehow maintain its reputation for ease of use. How does that work?

Opening the results of a grep in Vim

This week while removing some deprecated syntax from our old tests I found myself wondering how to take the results of my grep command and open the files returned in vim. Moments like this are great for getting to know your tools better so I decided to figure it out. Opening the files was not that difficult to figure out:

vim -o `grep -ril Factory.build test/`

This did exactly the right thing, a case insensitive (-i), recursive (-r) search of the test directory for the problematic syntax (Factory.build) returning only filenames (-l) and then opening the results in vim. I was able to quickly make my changes with a search and replace. With that done you can then write all buffers and close vim with:

:xa

I really love finding stuff like that. While that solved the immediate need, as written, it would not be able to deal with file names that have spaces in them. Since that sort of thing bugs me, I set up some test files and gave it a try:

mike@sleepycat:~/projects/vimtest☺ tail ./* ==> ./fileone <== blah blah cats blah
==> ./filethree <== blah rabbits blah
==> ./file two <== blah racoons dogs and cats blah

After a little tinkering I figured it out with a little help from the xargs man page. Its not something I would want to type often but this will make a very useful alias or script some time:

grep -Zril cats . | xargs -0 sh -c 'vim -O "$@" < /dev/tty' vim

Bash Boilerplate

I’ve ended up doing a lot of bash scripting lately and am really shocked at how difficult it is to sort out all most mundane things like handling long and short options and some other cases like ensuring the script is run as root. So here is my bash boilerplate; 45 lines of time and sanity saving bash script. It has been lovingly assembled in Franenstinian fashion from about a dozen blogs and stackoverflow answers and owes a great debt to missiondata’s excellent post on getopt. Thank you internet.

I have several scripts using variations on this now that I use reasonably regularly and its a real time saver to use this as my starting point. If you notice a mistake or something that is flat out a bad idea please let me know. I’d like to make this as solid as possible.

#!/usr/bin/env bash # set the script to exit immediately on error set -e
usage(){ cat <<'EOT' Call this script with... EOT exit 0; } # exit if there are no arguments [ $# -eq 0 ] && usage set -- `getopt -n$0 -u -a --longoptions "help dogs: cats:" "hd:c:" "$@"` # $# is the number of arguments while [ $# -gt 0 ] do case "$1" in -d|--dogs) dogs="$2"; shift;; -c|--cats) cats="$2"; shift;; -h| --help) usage;; --) shift;break;; *) break;; esac shift done cleanup_before_exit () { # this code is run before exit echo -e "Running cleanup code and exiting" } # trap catches the exit signal and runs the specified function trap cleanup_before_exit EXIT # if this script needs to be run as root # we can check like this: # if [ ! `id -u` -eq 0 ]; then # echo "You need to call this script using sudo." # exit 1 # fi
# OK now do stuff echo "$cats and $dogs living together, mass hysteria!"

Installing Ubuntu on a USB stick with QEMU-KVM

The more I learn about QEMU the more uses I find for it. Today I needed to install Ubuntu onto a USB memory stick. My normal process for doing such a thing is simply to create a bootable usb stick using usb-creator with the command:

usb-creator-gtk

What that gives you is a FAT32 formated usb with the Ubuntu installer and an option to “Try Ubuntu”. What I was after was to use the USB stick as a regular hard drive, formatted as normal with EXT4 and install Ubuntu on that. After a little headscratching I realized KVM would be perfect for this. Tell it to use the USB device file as the hard drive and the Ubuntu install ISO as the CDROM. Here is the command:

sudo kvm -hdb /dev/sdb -boot d -cdrom ~/Downloads/ubuntu-12.04-alternate-amd64.iso

The -boot flag identifies the device it should boot from with “d” being the shorthand for the first CD drive while -cdrom allows you to specify an iso to use as a disk. The really cool part of this is the -hdb flag that points to the device file for the USB drive. Slap the enter button and you are off to the races.

Having the flexibilty to do stuff like this is really awesome and saves me not only time but diskspace too, since I don’t need to keep creating Virtualbox VM’s. I’m going to have fun learning more about this.

The problem of selling software on Linux

Recently I listened Bryan Lunduke’s talk “Why Linux Sucks“. One of the arguments he made was that the open source/free software development model has produced a lot of great software but struggled to produce highly specialised and sophisticated software like Photoshop or some of the top audio/video editing apps.

The argument is certainly food for thought and he points out that the projects like the Linux kernel that are making progress very quickly are largely commercially backed. From there he talks about the need for the developers to get paid and lauds the Ubuntu Software Centre for offering the ability to sell software as a potential solution. I have been digesting this for a little while now and I think the concept of selling software can be deeply problematic.

The difficulty I have is that the economic incentives are for the developer to maximise the amount of money they collect by treating each version as a separate product. When you sell someone a particular version, the will keep that version until they feel that it is worth it to shell out for the next one. My Dad for instance ran Office 97 for almost 10 years, because he never felt the new features were worth the money.

The problem with this situation lies in the fact that old software lingering on your system is a vector for attack. Microsoft’s failure to entice my Dad into upgrading means that my Dad’s computer is now at risk of infection and compromise by all sorts of malware. Once he is infected and his computer starts attacking other people, this becomes everyone’s problem.

I think one of the strengths of the Linux platform is that all this “free as in speech” software has largely translated into “free as in beer” software which in turn means the barrier to staying up-to-date is nearly non-existent. Your package manager keeps installing the latest version for free and you don’t need to worry about virus’s and malware and I don’t need to worry that your computer is attacking mine. Throwing paid for software into this ecosystem seems like a way to compromise on of the best things about the platform. While Bryan Lunduke’s talk was good for stimulating some discussion I have come to feel that he has misdiagnosed the problem.

At a recent Android meetup I met a guy who was being paid by IBM to work on PhoneGap (now known as Apache Cordova). When I asked him why IBM would do this his answer was that IBM has hundreds of people that are trained and familiar with web technologies and almost none familiar with Objective C and the rest of the Apple technologies. With a need to make a bunch of apps on a variety of mobile devices and a a bunch of trained web developers on staff they looked around for a technology that could bridge that gap and found PhoneGap. After a little poking around they determined that it didn’t do everything they wanted but was close enough that paying someone to work on improving PhoneGap was cheaper than retraining all their web developers to learn Java and Objective C.

Coming back to the lack of good image/video/audio editors on Linux, it seems to me, then, that we have a chicken and egg type problem. The editors on Linux don’t attract financial backing because they are not good enough to be considered “good enough” for basic professional use. While that sounds a little circular, I think its sound. Just like IBM looking at PhoneGap and deciding that it was close enough to what they wanted that with minor investment they could make it fit their needs, GIMP and other programs could find that both monetary and code contributions could start rolling in once they cross that “close enough” threshold for their industry.

A good example is the GIMP. Much maligned for its interface, its real problem is that its lack of CMYK support has meant that anyone that deals with printing presses has been unable to use it. With the GIMP teams recent announcement that 90% of GIMP core has been ported to GEGL which allows them to operate in different colour-spaces like CMYK. This means that price sensitive people in the printing industry and can finally consider using GIMP in their daily workflow and may just find that its “good enough”. When they do, they too may find that throwing a few dollars towards the project to smooth out some rough edges will make the same kind of sense to them as it did to IBM.

For my part I took Bryan’s advice and took a look at the Ardour audio editor he used as one of his examples. He mentioned that it was getting some traction in the industry and was in need of donations to keep the developer working on it full time. I decided to donate monthly to the project, hoping that it too will cross that threshold into “good enough” and start attracting some investment from professionals looking for improvements. Here’s hoping!

Testing a live CD with QEMU-KVM

I read an article recently on Ars Technica about the recent design changes in Gnome. Over all it looks like Gnome is pushing forward with some interesting ideas and the article ended with a note that a live CD was available on the Gnome site that showed off the new interface. Creating a new virtual machine seems a little heavyhanded just to take a peek at the latest Gnome build so I am starting to use QEMU for that sort of thing. Here is all you need to do:

kvm -m 1024 -vga vmware -boot d -cdrom ~/Downloads/GNOME-3.4.iso

This kvm command comes from the qemu-kvm package which would obviously need to be installed. You may notice it resize a bunch of times before it settles down. This is a known bug, that is easily worked around by just giving it a few seconds to work itself out or resizing the window yourself. Anyway, that aside this a pretty useful way to test a live CD. Enjoy!

Update: 2014-06-14: Things have changed a little since I wrote this.
First, this works much better if your processor supports virtualization. You might need to turn this on in your BIOS. If your CPU supports it the following command will tell you. You can see ‘vmx’ highlighted in the output below:

mike@sleepycat:~☺ egrep '(vmx|svm)' /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms

This means we can pass -enable-kvm to the new qemu-system-x86_64 command to get it to use the Kernel-based Virtual Machine (KVM) which gets the kernel to take advantage of our hardwares virtualization capabilities:

qemu-system-x86_64 -m 6144 -vga std -enable-kvm -boot d -cdrom ~/Downloads/gnome-3.12.iso

You can see I am throwing more RAM at this thing (-m 6144) and using a different vga adapter to get more resolution (-vga std). Don’t forget that if you need to run a 32 bit OS you need to use qemu-system-i386.

Running a Sinatra app on Glassfish

I’m using JRuby more and more these days to deploy Rails apps. Typically that means creating a war file and deploying it on Glassfish. Yesterday I had request that seemed would be best fulfilled with a Sinatra app rather than Rails. It was my first experience getting Sinatra running on Glassfish and I thought I would share it.

The goal state here is a war file in the Glassfish autodeploy directory and the tool of choice is Warbler.

gem install warbler

With that installed we need to get ourselves set up to warble something. Warbler expects a specific directory structure it turns out, which is pretty typical for those familiar with the Ruby/Rails world:

mike@sleepycat:~/projects/myapp☺ find . ./Gemfile ./config.ru ./lib ./lib/myapp.rb ./Gemfile.lock

Of course you can add a views directory and other stuff in there as well. If you start doing things that Warbler does not expect you might have to add a warble.rb file but since I am using bundler and doing things that fall within warblers expectations I didn’t need one. I had a small number of very simple views, so I chose not to make a views directory and opted for named templates in the main app. To make this a little easier to digest, I’ve stripped out my code and made a basic hello world style app below thats going to be my starting point for next time I do this sort of thing. The three pieces you need are the Gemfile, the rackup file and the app itself.

The Gemfile:

source 'http://rubygems.org'

gem ‘sinatra’

group :development do
gem “sinatra-reloader”
gem “ruby-debug”
end

The config.ru

require 'rubygems' require 'bundler' Bundler.require require 'lib/myapp'

set :run, false #disable the built-in web server
set :environment, :production

run Sinatra::Application

And, sitting inside the lib directory, the application itself with a named template:

get '/' do @greeting = “Bonjour” status 200 erb :index end

template :index do
“<%= @greeting %> from Sinatra”
end
end

If you want to try it out at this point you can launch the app using rackup:

rackup config.ru

With those files in place you are ready to create a war file by running the “warble” command in the project directory. What that does is create a war file with the following structure:

./META-INF ./META-INF/init.rb ./META-INF/MANIFEST.MF ./WEB-INF ./WEB-INF/web.xml ./WEB-INF/Gemfile ./WEB-INF/config.ru ./WEB-INF/lib ./WEB-INF/lib/jruby-rack-1.1.5.jar ./WEB-INF/lib/jruby-core-1.6.7.jar ./WEB-INF/lib/jruby-stdlib-1.6.7.jar ./WEB-INF/lib/myapp.rb ./WEB-INF/gems ./WEB-INF/gems/specifications ...all the gemspecs from the gem dependencies end up here... ./WEB-INF/gems/gems ... All the gems that your project depends on are copied in to this folder ... ./WEB-INF/Gemfile.lock

And now you can drop that war in the Glassfish autodeploy directory and off it goes. My app was simple enough that I didn’t even need to use a database so you might need to add to this a little but it looks pretty simple. In fact every time I end up using Sinatra I come away impressed by how clean and simple it is. Now that I know how to run this on Glassfish, the next time may come sooner than later.

Making Virtual Machines with KVM and Virt-manager

I have used virtual machines for a quite a while now and have drifted back and forth between VMware and Virtualbox. While my initial experience of Virtualbox was good, between the acquisition by Oracle, the TAINT_CRAP thing and generally finding it slow and sometimes overly complicated, I thought I would explore some other options. Having recently discovered my laptop supports hardware virtualization, I thought I would try out KVM.

KVM stands for Kernel-based Virtual Machine and Ubuntu’s documentation sums up the expected uses nicely; “Ubuntu uses KVM as the back-end virtualization technology primarily for non-graphic servers”. So while this is definitely created with servers in mind, I figure I will explore using it as a Virtualbox replacement and see how that goes.

So after enabling virtualization in my BIOS I installed all the packages:

sudo apt-get install virt-manager qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils

The first thing I learned after starting virt-manager is that KVM is using QEMU as its processor emulator. This becomes obvious when you launch virt-manager and notice that it can’t connect to QEMU on localhost.
To get that working just add yourself to the libvirtd group. I did it manual style:

sudo vim /etc/group

and add yourself to the group like so:

libvirtd:x:128:mike

After that you will need to logout and log back in (restarting works too) and you should see localhost(QEMU) in the virt-manager GUI indicating that a connection has been made.

The process of setting up a virtual machine is similar to what you would find in VMWare or Virtualbox. Just click the “Create new virtual machine” button and its pretty self explanatory. One thing that stood out as a little concerning is an little red error icon when setting up the network config.

If you mouse over it, it says “could not initialize HAL for interface listing”. HAL of course has been deprecated and removed from Ubuntu (and replaced by Udev) so its not a surprise that it can’t be found. I wasn’t sure what to make of this but it turns out not to be an issue; the VM boots up happily and is able to sort out a network connection just fine.

I am curious to see what it will be like to work with this for a while. Particularly doing things like using the VM as a server of various kinds which is terribly clunky in with Virtualbox. Well I still have lots of tinkering to do but this is enough to get the experiment underway.