Living Code

Aspect-oriented Python

2009-02-25T06:09:18Z

A friend asked me for an example of AOP in Python. I started to write up my response, then realized it might be worth sharing more widely.

Briefly, AOP is about separating out Aspects which are are interspersed throughout your code (in AOP lingo, cross-cutting). The canonical example is logging, but there are others.

For Python, as long as you’re content with before/after aspects, the situation is good. In Python 2.5 and up there are two main tactics: decorators and context managers. Decorators use the @ syntax and context managers are used with the “with” keyword.

Decorators

Decorators are only usable on functions and methods in 2.4 and 2.5, but in 2.6, 3.0 and beyond they can be used on classes as well. Essentially they are callables (functions, methods, objects) that accept a function object and return a function object. They are called when the function is defined, so they get a chance to have their way with it: annotate it, replace it, or wrap it. The common case is to wrap it.

Quick example:

import logging
def before(fn):
    def wrapped(*args, **kws):
        logging.warn('about to call function %s' % fn.func_name)
        return fn(*args, **kws)
    return wrapped

def after(fn):
    def wrapped(*args, **kws):
        retVal = fn(*args, **kws)
        logging.warn('just returned from function %s' % fn.func_name)
        return retVal
    return wrapped

OK, those are three basic wrappers, you can use them like so:

@before
def foo():
    logging.warn('inside foo')

@after
def bar():
    logging.warn('inside bar')

@before
@after
def baz():
    logging.warn('inside baz')

foo()
bar()
baz()

This will result in the following output:

WARNING:root:about to call function foo
WARNING:root:inside foo
WARNING:root:inside bar
WARNING:root:just returned from function bar
WARNING:root:about to call function wrapped
WARNING:root:inside baz
WARNING:root:just returned from function baz

You will note that when we use two decorators on baz, the name of the function called by before is “wrapped.” This is because what before is called on is the result of after. The functools.update_wrapper function is useful in this case to make a wrapped function look more like the original function.

For more, please see PEP 318 Decorators for Functions and Methods: Examples and PEP 3129 Class Decorators. For convenience when creating new decorators, see the standard library functions functools.update_wrapper and functools.wraps.

Context Managers

Context Managers are used with the with statement, and are handy for resource aquisition and release. In Python 2.5 you have to “from __future__ import with_statement” to use them, but they are built-in in Python versions later than that. Also, objects such as files and locks are context managers now, so you can use patterns like

with open('example.txt') as example:
    for line in example:
        do_something(line)

This will automatically close the file when leaving the with block. And for locks the pattern is similar:

with myThreadingLock:
    do_something_threadsafely()

It is important to note that the lock will be release properly, or the file closed, even if an exception is thrown inside the with block.

If you want to create your own context managers, you can add two methods to your objects: __enter__(self) and __exit__(self, exception_type, exception_value, traceback). The return value from __enter__ will be passed to the optional as variable (seen in the file example). The __exit__ method will be called with exception info if there was an exception. If no exception is raised in the with block, then all three arguements will be None. If __exit__ returns True then any exception will be “swallowed”, otherwise the exception will be re-raised after any cleanup.

For more info, see PEP 343 The “with” Statement, especially the examples section, and also the helper functions in the standard library contextlib.

AOP and 80/20

Full-on aspect-oriented programming is beyond the scope of this post and involves join-points, code weaving, and other such arcanery. There are multiple Python libraries which target aspect-oriented coding styles, but for my money, the simplicity of the methods in the standard library, coupled with my impression that they cover at least 80% of the uses of AOP, make me favour these built-in techniques over any of the special purpose tools.

Unit Testing in Vancouver

2009-02-02T18:56:41Z

Just a quick reminder: Henry Prêcheur will be presenting Unit Testing in Python at the VanPyZ meeting tomorrow, February 3rd. The meeting will be at Workspace in Gastown (see map on VanPyZ page). Meetings are from 7-8:30, then we head out for beers afterwards.

Hope to see you there!

Tab Dumping with AppleScript and back to Python

2009-01-31T20:04:19Z

Goal: Iterate through all my (OS X Safari) browser windows and make a list of titles and urls which is then placed in the clipboard ready to be pasted into an email or blog post.

This is an update to Tab Dumping in Safari. That still works well as the basis for extending any Cocoa-based application at runtime, but it relies on SIMBL, which while it is a great bit of code, essentially is abusing the InputManager interface. Some developers and users shun such hacks, and at least one Apple application checks for them at startup and warns you from using them.

I have been running the WebKit nightlies, which are like Safari, but with newer code and features (most importantly to me right now, a Firebug-like developer toolkit). WebKit warns at startup that if you’re running extensions (such as SIMBL plugins) it may make the application less stable. I was running both Saft and my own tab dumping plugin, and WebKit was crashing a lot. So I removed those and the crashes went away. I miss a handful of the Saft extensions (but not having to update it for every Safari point release), and I found I really miss my little tab-dumping tool.

I toyed with the idea of rewriting it as a service, which would then be available from the services menu, but couldn’t figure out how to access the application’s windows and tabs from the service. So I tried looking at Safari’s scriptable dictionary, using the AppleScript Script Editor. Long ago, John Gruber had written about the frustration with Safari’s tabs not being scriptable, but a glance at the scripting dictionary showed me this was no longer the case (and probably hasn’t been for years, I haven’t kept track).

I am a complete n00b at AppleScript. I find the attempt at English-like syntax just confuses (and irritates) me no end. But what I wanted looked achievable with it, so I armed myself with some examples from Google searches, and Apple’s intro pages and managed to get what I wanted working. It may not be the best possible solution (in fact I suspect the string concatenation may be one of the most pessimal methods), but it Works For Me™.

In Script Editor, paste in the following:

set url_list to ""
-- change WebKit to Safari if you are not running nightlies
tell application "WebKit"
  set window_list to windows
  repeat with w in window_list
    try
      set tab_list to tabs of w
      repeat with t in tab_list
        set url_list to url_list & name of t & "\n"
        set url_list to url_list & URL of t & "\n\n"
      end repeat
    on error
      -- not all windows have tabs
    end try
  end repeat
  set the clipboard to url_list
end tell

I had to use AppleScript Utility to add the script menu to my menu bar. From there it was easy to create script folders that are specific to both WebKit and Safari and save a copy of the script (with the appropriate substitution, see comment in script) into each folder. Now I can copy the title and URL of all my open tabs onto the clipboard easily again, without any InputManager hacks.

I had some recollection that is a way to do this from Python, so I looked and found Appscript. I was able to install this with a simple easy_install appscript and quickly ported most of the applescript to Python. The only stumbling block was that I couldn’t find a way to access the clipboard with appscript, and I didn’t want to have to pull in the PyObjC framework just to write to the clipboard. So I used subprocess to call the command-line pbcopy utility.

#!/usr/local/bin/python
from appscript import app
import subprocess
tab_summaries = []
for window in app('WebKit').windows.get():
    try:
        for tab in window.tabs.get():
            name = tab.name.get().encode('utf-8')
            url = tab.URL.get().encode('utf-8')
            tab_summaries.append('%s\n%s' % (name, url))
    except:
        # not all windows have tabs
        pass
clipboard = subprocess.Popen('pbcopy', stdin=subprocess.PIPE)
clipboard.stdin.write('\n\n'.join(tab_summaries))

The remaining hurdle was simply to put the Python script I’d written into the same Scripting folder as my AppleScript version. For me this was ~/Library/Scripts/Applications/WebKit/. When run from the scripts folder, your usual environment is not inherited, so the #! line must point to the version of Python you are using (and which has Appscript installed). You should also make the script executable. Adding .py or any other extension is not necessary.

Overall, while I found AppleScript to be very powerful, and not quite as painful as I remembered, I found the Python version (warts and all) to be easier to work with. Combined with the fact that the script folder will run non-Applescript scripts, this opens up new worlds for me. I have hesitated in the past to write a lot of SIMBL-based plugins, tempting though it may be, because they are hacks, and they run in every Cocoa-based application. But adding application-specific (or universal) scripts, in Python, is pure, unadulterated goodness.

Processing Playground

2008-06-10T14:02:58Z

There is a lot of interesting stuff happening in Javascript land these days, even to the point of other languages targetting the browser as a runtime, but running on top of Javascript. You can run Scheme right in the browser, and by now everyone has probably heard of Objective-J (open-source coming soon), an Objective-C-like language used by 280 North to create their 280 Slides web application, inspired by Apple’s Keynote.

Since my last post about Processing, John Resig managed to port most of Processing to Javascript, so it is easier than ever to get started. Now instead of having to download the Java-based runtime, you can create Processing animations in your browser, within the limitations that it only targets very recent, beta browsers (Firefox 3, Opera 9.5, WebKit nightlies, no version of IE) and that not all of Processing is supported (no 3D, for instance, and my example from the earlier post does not run). Still, it is interesting and a lot of fun to play with. My seven-year-old son is fascinated with computer programming and looking to move beyond Scratch, so as part of that I stuck all the basic examples from Mr. Resig’s site into one page, with a menu to select them, and a button to run them. And I made them editable. You can write entirely new code too, of course, but the examples can help for getting started. I hope folks enjoy it.

Processing Playground

Of course, what my kids really want is a version of Scratch that we can extend to add some more features of our own. Scratch has been open-sourced, so we could possibly extend it, but it is built on Squeak Smalltalk, and I’ve never been very good at Smalltalk. Instead, I am porting it to Javascript. It is still in the early stages, but I’m making steady progress in my hour or so I have to code each evening, and my kids are eager to use it, so they keep me motivated and focussed.

Meme du jour

2008-04-11T16:05:39Z

Just for fun, and because I’m trying out two things:

Using Ecto for posting again
Blogging more often

Here are my results from the latest craze:


$ history|awk '{a[$2]++} END{for(i in a){printf "%5d\t%s\n",a[i],i}}'|sort -rn|head
139 ls
87 cd
54 python
43 /usr/bin/python
38 less
21 vim
16 clear
12 mate
11 rm
10 ssh

The first python is the one I built from source so I can include it in applications packaged with py2app, the second is the built-in python for Leopard. Both are version 2.5. The mate program is used to open files in TextMate from the command line.

MetaProgramming at VanPyZ

2008-04-01T04:01:59Z

Just a reminder for Pythonistas in the Vancouver area: The Vancouver Python and Zope user group (VanPyZ) is tomorrow (Tuesday, April 1). Paul Prescod will be talking about metaprogramming in Python. Details and directions are on the VanPyZ site. And, as usual, we’ll be heading out to the pub afterwards for more discussion.

Hope to see you there!

Saving PNG from PyGame

2008-04-01T03:21:31Z

The latest version, 1.8, of PyGame can save PNGs directly from a Surface: pygame.Image.save(mySurface, 'myimagefile.png'). But what if you want to support an older version of PyGame, such as the one available for the N800 or the XO? Well, assuming you have access to the Python Image Library, you can use that:

import Image # from PIL
import pygame

def pygame_to_pil_img(pg_surface):
    imgstr = pygame.image.tostring(pg_surface, 'RGB')
    return Image.fromstring('RGB', pg_surface.get_size(), imgstr)

def pil_to_pygame_img(pil_img):
    imgstr = pil_img.tostring()
    return pygame.image.fromstring(imgstr, pil_img.size, 'RGB')

Once you have a PyGame Image, you can save it to PNG easily: myImage.save('myfilename.png')

I’ve found myself looking for this code snippet more than once, now I can Google for it more readily, and maybe someone else will find it helpful too.

Drawing Hexmaps

2008-03-21T05:03:08Z

The other day, Thomas Guest talked about drawing chessboards, and ended with a challenge. I wanted to answer a different challenge, however. What if, instead of drawing on a rectangular grid, we wanted to draw on a hexagonal grid? The following is my slapdash answer. For real-world use I’d make nice classes and pass more parameters to the methods, but to demonstrate the math I’m just going to use global constants and functions.

Like Thomas’ article, I will show solutions using several different tools, in this case Apple’s Core Graphics, PyGame, Python Imaging Library (PIL), and SVG. All of these solutions will use the same constants and math:

# Constants used by each solution

from math import sin, cos, pi, sqrt
THETA = pi / 3.0 # Angle from one point to the next
HEXES_HIGH = 8 # How many rows of hexes
HEXES_WIDE = 5 # How many hexes in a row
RADIUS = 30 # Size of a hex
HALF_RADIUS = RADIUS / 2.0
HALF_HEX_HEIGHT = sqrt(RADIUS ** 2 - HALF_RADIUS ** 2)
IMAGE_WIDTH = int(RADIUS * (HEXES_WIDE * 3 + .5))
IMAGE_HEIGHT = int(HALF_HEX_HEIGHT * (HEXES_HIGH + 1))

# Functions (generators) used by each solution

def hex_points(x,y):
    '''Given x and y of the origin, return the six points around the origin of RADIUS distance'''
    for i in range(6):
        yield cos(THETA * i) * RADIUS + x, sin(THETA * i) * RADIUS + y

def hex_centres():
    for x in range(HEXES_WIDE):
        for y in range(HEXES_HIGH):
            yield (x * 3 + 1) * RADIUS + RADIUS * 1.5 * (y % 2), (y + 1) * HALF_HEX_HEIGHT

Now, given the above, what does the code look like to draw the hexes? Because each library handles colours slightly differently, we will need a generator for colours (and we will need more than just black and white as the chessboard used, because each hex borders on six others). I haven’t given a lot of thought to optimal colouring schemes: each colour generator simply produces red, yellow, blue, and green in a cycle. Here is the image produced by the Core Graphics solution, followed by the code:

def quartz_colours():
    while True:
        yield 1,0,0,1 # red
        yield 1,1,0,1 # yellow
        yield 0,0,1,1 # blue
        yield 0,1,0,1 # green

def quartz_hex():
    '''Requires a Mac with OS 10.4 or better and the Developer Tools installed'''
    import CoreGraphics as cg
    colours = quartz_colours()
    cs = cg.CGColorSpaceCreateDeviceRGB()
    c = cg.CGBitmapContextCreateWithColor(IMAGE_WIDTH, IMAGE_HEIGHT, cs, (0,0,0,.2))
    c.saveGState()
    c.setRGBStrokeColor(0,0,0,0)
    c.setLineWidth(0)
    for x,y in hex_centres():
        c.beginPath()
        c.setRGBFillColor(*colours.next())
        points = list(hex_points(x,y))
        c.moveToPoint(*points[-1])
        [c.addLineToPoint(*pt) for pt in points]
        c.drawPath(cg.kCGPathFill)
    c.restoreGState()
    c.writeToFile("quartz_hexes.png", cg.kCGImageFormatPNG)

Now for some cross-platform examples. Here is the image generated by PyGame, followed by that code:

def pygame_colours():
    while True:
        yield 255, 0, 0 # red
        yield 255, 255, 0 # yellow
        yield 0, 0, 255 # blue
        yield 0, 255, 0 # green

def pygame_hex():
    '''Requires PyGame 1.8 or better to save as PNG'''
    import pygame
    pygame.init()
    screen = pygame.display.set_mode((IMAGE_WIDTH, IMAGE_HEIGHT))
    colours = pygame_colours()
    for x,y in hex_centres():
        pygame.draw.polygon(screen, colours.next(), list(hex_points(x,y)))
    pygame.image.save(screen, 'pygame_hexes.png')

When you run the PyGame script, it will actually pop up a window very briefly, draw into the window, save the result, and close the window. I also didn’t get the PyGame script to add transparency for the background, although I think it could be added fairly easily. Now, for the web, here is a solution in SVG, with the image captured by screenshot in Safari, followed by the Python code, and the resulting SVG code:

def svg_colours():
    while True:
        yield 'rgb(255, 0, 0)'
        yield 'rgb(255, 255, 0)'
        yield 'rgb(0, 0, 255)'
        yield 'rgb(0, 255, 0)'

def svg_hex():
    out = open('svg_hexes.svg', 'w')
    print >> out, '''
    
    '
    out.close()

And here is the SVG created by the above script: SVG Hexes

Finally, one library which overlaps with the ones used by the Chessboard example: Python Imaging Library.

pil_colours = pygame_colours  # same format works, so we'll re-use it

def pil_hex():
    import Image, ImageDraw
    image = Image.new("RGBA", (IMAGE_WIDTH,IMAGE_HEIGHT), (0,0,0,0))
    colours = pil_colours()
    draw = ImageDraw.Draw(image)
    for x,y in hex_centres():
        draw.polygon(list(hex_points(x,y)), fill=colours.next())
    image.save('pil_hexes.png', 'PNG')

That’s it for my examples. Thomas ended with a challenge for displaying chess, and for describing the position. To describe the position, I would use a standard chess notation, such as described here. For my challenge, what other formats would be useful to create hex maps in? POVRay? Flash? Any other examples out there?

The 419 Economy

2008-03-20T05:25:17Z

Hello, my name is Sergio Arragones and I am writing to you for assistance in a matter that will be of great economic benefit to us both.

My late father was VICE PRESIDENT of the large American investment bankerage of Bear Stearns and he managed to put aside over two billion US dollars of bailout money from the puppet government of Mr. George Bush.

Unfortunately, he was killed in a incident of ROAD RAGE when his bullet-proof Mercedes convertible was driven into the bomb barricade of the New York UN Building by the SUV of an off-duty Police Officer.

While my family is still grieving over his tragic death, and attempting to get the suspicious circumstances investigated, I must act quickly to move these funds to your Nation for safekeeping. As a Citizen of the Monarchy of Canada, you may receive these funds for me until I may cross from New York State to the Province of Quebec, at which time you will transfer the funds over to me, keeping 10% (TEN PERCENT) for yourself, along with my Gratitude.

In order for me to transfer the Fund to you, you will need to create a PayPal account for this purpose. Do not use an existing PayPal account, as it may already have been compromised by nefarious hackers and BotNets. Also, you will need to download the Python libraries which allow you to code to PayPal directly. In addition, you will need libraries for Google Checkout the Atom Publishing Protocol.

On a server under your control (the server must be physically located in the Sovereignty of Canada. THIS IS IMPORTANT) you must set up a WSGI server. You may use Django or TurboGears for this, or use the reference WSGI implementation. I recommend you use PYTHON 2.5 or greater as many of the tools you need will be part of the STANDARD LIBRARY.

Once you have the Foundations of a Web2.0 server administered, contact me at the return address of this message and I will give you instructions on how to proceed for the transfer of funds.

My family thanks you.

Sincerly,

Dr. Rev. Sergio Arragones, Esq.
New York, NY
United States of America

Processing Critters

2008-03-20T17:49:53Z

This browser does not have a Java Plug-in.

Get the latest Java Plug-in here.

Source code: sketch_080314a

I’ve been playing around some with Processing, this is the result. As I mentioned in my earlier article, Processing and NodeBox are quite similar. Processing has some more interactivity (better keyboard and mouse handling) built in, and it can do basic 3D. NodeBox has better color handling (gradients) and remarkable libraries (WordNet). But what it really comes down to is that NodeBox is Python (much easier to extend) and Processing is Java (still easier than straight Java, but a lot more code to do basic extensions). Now all we need is a quick-start programming environment like these for Flash (and before you tell me the Adobe Flash tools are for artists–I have used the Flash IDE and it is possibly the worst IDE I’ve ever experienced. Which is odd, because the Flex IDE is one of the best, better than XCode/Interface Builder in some ways).

All of which is the long way to say that I enjoyed making this little animation in Processing, but I’m ready to go back to working in Python now, thank you.