Mastodon

Tag: python

Why Virtualenv?

Virtualenv comes up often when learning about Python. It’s a Python library that creates a folder into which you install all the libraries your project will need. While its often stated that you should use it, its not often explained why. I recently stumbled upon a good intro that gives an example of creating an application that uses requests and then giving the scenario where running sudo pip install --upgrade requests while working on a separate project breaks the first application.
The idea that updating a library in one project would/could break some/all of my other projects that rely on that library is bizarre and kind of terrifying. It’s nice that the solution to the problem is apparently Virtualenv, but why is this a problem to begin with?

The root of this problem seems to be Pip. If I install version 1.0 of the testing library nose (because I am using pyenv) it gets placed in ~/.pyenv/versions/3.4.0/lib/python3.4/site-packages/. Looking in there, I can see folders for both the code and the metadata (the egg-info folder):

nose/                      nose-1.0.0-py3.4.egg-info/

If I run the pip install --upgrade command you can see the problem unfold:

mike@sleepycat:~/projects/play/python_play☺  pip install --upgrade nose
Downloading/unpacking nose from https://pypi.python.org/packages/source/n/nose/nose-1.3.1.tar.gz#md5=672398801ddf5ba745c55c6eed79c5aa
  Downloading nose-1.3.1.tar.gz (274kB): 274kB downloaded
...
Installing collected packages: nose
  Found existing installation: nose 1.0.0
    Uninstalling nose:
      Successfully uninstalled nose
  Running setup.py install for nose
...

Yup, Pip only installs a single version of a library on your system. A quick look back in the ~/.pyenv/versions/3.4.0/lib/python3.4/site-packages/ folder confirms what Pip’s insanely verbose output told us, our nose 1.0 is gone:

nose/                      nose-1.0.0-py3.4.egg-info/

This is pretty surprising for someone whose expectations have been shaped by Ruby’s package manager rubygems. You can see multiple versions of the same library coexisting in the interpreter’s gems folder, meaning that my old projects will still be able to use the old version while my new projects can use the newer without carnage:

ls ~/.rbenv/versions/rbx-2.2.5/gems/gems/arel-
arel-3.0.3/ arel-4.0.1/

Returning to the reason for needing Virtualenv, at first glance it seems you need Virtualenv to protect you from Pip’s inability to do multiple versions of a library. What’s interesting is that both Virtualenv and Pip where written by the same person, Ian Bicking, Virtualenv in 2007 and Pip in 2008. What this seems to suggest is that installing a single version is a design decision made because Pip assumes the existence/use of something like Virtualenv. This is especially true when you realize that Pip was aimed at replacing easy_install, an earlier tool which actually could do multiple versions of the same library as Rubygems had since 2003.

So if you have ever wondered why you need Virtualenv, its seems we have an answer. Pip has pretty much completely replaced previous package managers, and it was developed to assume Virtualenv or something similar is being used… and its assumptions essentially force you to use it.

For those of us starting out with Python, sorting out the ins and outs of the messy world of Python packaging is a pain. The old system seems to be broken, the current one using Virtualenv/Pip is hacky and far from ideal, and the future seems to be largely ignored. Fortunately the beginnings of a clean solution appear to be coming from the world of Docker, so we will have to watch that space carefully. In the meantime, I guess I will have to install Virtualenv…

Sikuli tips

Lately I have been working alot with Sikuli. In case you have not heard of it, its a project that uses the computer vision library OpenCV to automate GUI interactions. While Sikuli itself is new to me, this also marks my first exposure to the underlying language of Python.

For the most part its nice to use an has some decent documentation, but there have been a few things that I have tripped over; some related to Sikuli itself and some due to my lack of familiarity with Python. So here is a compilation of a bunch of those little things that I had to stop and scratch my head over.

Java 7:

Don’t. It crashes every time you click one of the image functions in the IDE. Use Java 6 and life is good.

Special Keys:

All the special keys can be accessed through the Key class.

type('Sikuli rocks' + Key.ENTER)

The rest of the keys follow the same pattern… Key.TAB, Key.BACKSPACE and so on.
All that leads you to believe that to copy some text is going to use Key.CTRL, but you actually need to use the KeyModifier class instead. So Ctrl+C looks like this:

type('c', KeyModifier.CTRL)

Killing a script:

You will run your script a lot and when it goes badly you need to kill it but there is no obvious way to do that since the IDE disappears when you run the script. Kill your script with this:

Alt+Shift+c

Screenshots:
Sometimes you need to capture a image of a menu (like a context menu) that disappears when you move away. To catch an image of the menu use the shorcut:

Ctrl+Shift+2

Launching an app:
Sure you could doubleclick some icon somewhere but its much faster to launch most apps with a command. To save yourself from having to escape all the slashes in a path you should use Pythons raw string (note the r in from of the path):


firefox = App(r'C:\Program Files\Mozilla\firefox.exe')

Sikuli has some pretty good documentation but there are some areas where it feels a little thin. Specifically finding a complete list of methods for a particular class. Fortunately Pythons dir() function does that job.


firefox = App(r'C:\Program Files\Mozilla\firefox.exe')
screenRegion = firefox.window()
print dir(screenRegion)

[‘ROI’, ‘above’, ‘autoWaitTimeout’, ‘below’, ‘bottomLeft’, ‘bottomRight’…]

Child windows:
Lets say that Firefox opens a child window, like the downloads window. To get a handle on that you just need to do this:


downloadsWindow = firefox.window(0)

Unfortunately what you will find in downloadsWindow is an object of Region class. I was expecting to be able to get an App instance so I could call methods like focus() on it. It does the job though.

One other thing is that sometimes you will want to check that a region actually contains what you think it does. For that you will probably want to capture an image of that area during execution. I drop in code like this when ever I need to get a visual of a region:


import shutil
captureimage = capture(resultsBox)
shutil.move(captureimage, r'C:\somefolder\bounds.png')

These are a few of the things I have bumped into so far and I wanted to put them up here so I don’t forget them. Hopfully I’ll get to do more work with Sikuli. I think with a little practice I could make some scripts that are pretty resiliant to a lot of the changes that normally break these types of scripts (unexpected popups for instance, or slow networks). I’m looking forward to learning more about it.