package main
import fmt "fmt"
type Stringy func() string
func foo() string{
return "Stringy function"
}
func takesAFunction(foo Stringy){
fmt.Printf("takesAFunction: %v\n", foo())
}
func returnsAFunction()Stringy{
return func()string{
fmt.Printf("Inner stringy function\n");
return "bar" // have to return a string to be stringy
}
}
func main(){
takesAFunction(foo);
var f Stringy = returnsAFunction();
f();
var baz Stringy = func()string{
return "anonymous stringy\n"
};
fmt.Printf(baz());
}
This compiled and ran, giving the right output, so it looks to me like functional programming is indeed possible in Go, and on the Effective Go page we learn, “In Go, function literals are closures: the implementation makes sure the variables referred to by the function survive as long as they are active.” So that’s kind of cool.
Go also has rich, high-level concurrency support similar to Erlang, several features that prevent random memory from being read or written, and both strings and maps (dictionaries, hash tables) are built-in datatypes. It has a small, but powerful standard library.
One puzzle down, several to go. I still want to explore how I can talk to Cocoa from Go (for GUIs), how to wrap libraries such as libxml2 and cairo for Go, and basically how to map more Python/Javascript library goodness into Go’s world.
I have smaller questions too: can I add methods to existing types? How do I load the contents of a URL? It’s looking good so far! Tune in next time for more exploration.
]]>What do you get when you cross a clock and a computer: a computer. An
airplane and computer: computer. A VCR and computer: computer. [This argument I believe was my summary of something Donald Norman said in The Design of Everyday Things.]
Of course, computer here is used as a code word for Something which is
perverse and complicated, which most computer *are*, but they don’t have to
be. It doesn’t imply that you can program your clock radio with Python, for
instance.
These are examples of moped compromises. I owned a moped once. It was too
heavy to pedal and too underpowered to go up hills. It combined all the
worst features of bicycles and motorcycles, with few of the advantages of
either.
These bad compromises are everywhere. When I studied to be an EMT, we learned
some Greek and Latin, a little CPR, but mostly we learned how to cover our
asses in case we were sued. By becoming professionals we would no longer be
covered by Good Samaritan laws, and the nature of the job (trying like hell to
save somebody’s life, often when it’s too late) meant lots of times the
patients died or lost a limb or whatever. Whenever the outcome is less than
optimal, the American knee-jerk reaction is to sue. So we inadvertently
became lawyers.
What do you get when you cross a paramedic with a lawyer: A paralegal.
Now I’m in computing and it seems that just writing code, pretty simple and
harmless code, not viruses or anything like that, can get you thrown in jail
or sued in all kinds of remarkable and unpredictable ways. In fact, you can
break the law without even knowing it, because what you’ve created is already
a secret. Add that to the awesome array of software licenses and we’re
inadvertently becoming lawyers again (or outlaws).
What do you get when you cross a computer scientist with a lawyer: A lawyer.
]]>Briefly, AOP is about separating out Aspects which are are interspersed throughout your code (in AOP lingo, cross-cutting). The canonical example is logging, but there are others.
For Python, as long as you’re content with before/after aspects, the situation is good. In Python 2.5 and up there are two main tactics: decorators and context managers. Decorators use the @ syntax and context managers are used with the “with” keyword.
Decorators are only usable on functions and methods in 2.4 and 2.5, but in 2.6, 3.0 and beyond they can be used on classes as well. Essentially they are callables (functions, methods, objects) that accept a function object and return a function object. They are called when the function is defined, so they get a chance to have their way with it: annotate it, replace it, or wrap it. The common case is to wrap it.
Quick example:
import logging
def before(fn):
def wrapped(*args, **kws):
logging.warn('about to call function %s' % fn.func_name)
return fn(*args, **kws)
return wrapped
def after(fn):
def wrapped(*args, **kws):
retVal = fn(*args, **kws)
logging.warn('just returned from function %s' % fn.func_name)
return retVal
return wrapped
OK, those are three basic wrappers, you can use them like so:
@before
def foo():
logging.warn('inside foo')
@after
def bar():
logging.warn('inside bar')
@before
@after
def baz():
logging.warn('inside baz')
foo()
bar()
baz()
This will result in the following output:
WARNING:root:about to call function foo
WARNING:root:inside foo
WARNING:root:inside bar
WARNING:root:just returned from function bar
WARNING:root:about to call function wrapped
WARNING:root:inside baz
WARNING:root:just returned from function baz
You will note that when we use two decorators on baz, the name of the function called by before is “wrapped.” This is because what before is called on is the result of after. The functools.update_wrapper function is useful in this case to make a wrapped function look more like the original function.
For more, please see PEP 318 Decorators for Functions and Methods: Examples and PEP 3129 Class Decorators. For convenience when creating new decorators, see the standard library functions functools.update_wrapper and functools.wraps.
Context Managers are used with the with statement, and are handy for resource aquisition and release. In Python 2.5 you have to “from __future__ import with_statement” to use them, but they are built-in in Python versions later than that. Also, objects such as files and locks are context managers now, so you can use patterns like
with open('example.txt') as example:
for line in example:
do_something(line)
This will automatically close the file when leaving the with block. And for locks the pattern is similar:
with myThreadingLock:
do_something_threadsafely()
It is important to note that the lock will be release properly, or the file closed, even if an exception is thrown inside the with block.
If you want to create your own context managers, you can add two methods to your objects: __enter__(self) and __exit__(self, exception_type, exception_value, traceback). The return value from __enter__ will be passed to the optional as variable (seen in the file example). The __exit__ method will be called with exception info if there was an exception. If no exception is raised in the with block, then all three arguements will be None. If __exit__ returns True then any exception will be “swallowed”, otherwise the exception will be re-raised after any cleanup.
For more info, see PEP 343 The “with” Statement, especially the examples section, and also the helper functions in the standard library contextlib.
Full-on aspect-oriented programming is beyond the scope of this post and involves join-points, code weaving, and other such arcanery. There are multiple Python libraries which target aspect-oriented coding styles, but for my money, the simplicity of the methods in the standard library, coupled with my impression that they cover at least 80% of the uses of AOP, make me favour these built-in techniques over any of the special purpose tools.
]]>
Well, my Balding for Dollars event was a success. I have raised $717.37 for Children’s Hospital (there is still time to help bring it to my goal of $1000 if you’d like to donate). Thanks to everyone who helped with donations, words of support, and especially to Frank for his photography, to Berenice for helping with the cutting, to my wife Daniela for her encouragement and help, and to our kids for their patience. I couldn’t have done it without all of you.
There has been one change of plans. I had wanted to donate the hair to a Canadian foundation making wigs for kids with cancer, but couldn’t find it. Daniela’s google-fu skills were better and tomorrow my hair will go off to the Eva and Co. hair donation program. And to clarify, my daughter didn’t shave her head, we’re sending a ponytail from when she went from a long hairstyle to a short one.
Thanks also, to everyone who offered me a toque, but I’m well covered.
]]>
I usually use this site to try to give back to the community, with Python and Javascript tutorials, software, and the occasional joke. Today I’m going to ask for something in return. On Saturday, February 14th, 2009 (Valentine’s Day, at least in the US and Canada) I will be shaving my head to raise money for Vancouver Children’s Hospital as part of their Balding for Dollars drive.
The picture above was taken today by my friend Frank Lee. As you can see, my hair is quite long. I’ll be sending my ponytail to Locks of Love, who make a wigs for kids who have lost their hair due to medical conditions. This is my second balding for dollars and will be the fourth ponytail I’ve donated, and my daughter is also donating her ponytail. As I get older, my hair isn’t growing in like it used to. This may be my last good ponytail to give, so with your help, I’d like to make a difference.
You can make donations on the Balding for Dollars site here. Saturday is only two days away, so be my valentine and help some kids!
I appreciate any help you can give. Check back on Monday for before/after photos. Thank you!
]]>Hope to see you there!
]]>
Goal: Iterate through all my (OS X Safari) browser windows and make a list of titles and urls which is then placed in the clipboard ready to be pasted into an email or blog post.
This is an update to Tab Dumping in Safari. That still works well as the basis for extending any Cocoa-based application at runtime, but it relies on SIMBL, which while it is a great bit of code, essentially is abusing the InputManager interface. Some developers and users shun such hacks, and at least one Apple application checks for them at startup and warns you from using them.
I have been running the WebKit nightlies, which are like Safari, but with newer code and features (most importantly to me right now, a Firebug-like developer toolkit). WebKit warns at startup that if you’re running extensions (such as SIMBL plugins) it may make the application less stable. I was running both Saft and my own tab dumping plugin, and WebKit was crashing a lot. So I removed those and the crashes went away. I miss a handful of the Saft extensions (but not having to update it for every Safari point release), and I found I really miss my little tab-dumping tool.
I toyed with the idea of rewriting it as a service, which would then be available from the services menu, but couldn’t figure out how to access the application’s windows and tabs from the service. So I tried looking at Safari’s scriptable dictionary, using the AppleScript Script Editor. Long ago, John Gruber had written about the frustration with Safari’s tabs not being scriptable, but a glance at the scripting dictionary showed me this was no longer the case (and probably hasn’t been for years, I haven’t kept track).
I am a complete n00b at AppleScript. I find the attempt at English-like syntax just confuses (and irritates) me no end. But what I wanted looked achievable with it, so I armed myself with some examples from Google searches, and Apple’s intro pages and managed to get what I wanted working. It may not be the best possible solution (in fact I suspect the string concatenation may be one of the most pessimal methods), but it Works For Me™.
In Script Editor, paste in the following:
set url_list to ""
-- change WebKit to Safari if you are not running nightlies
tell application "WebKit"
set window_list to windows
repeat with w in window_list
try
set tab_list to tabs of w
repeat with t in tab_list
set url_list to url_list & name of t & "\n"
set url_list to url_list & URL of t & "\n\n"
end repeat
on error
-- not all windows have tabs
end try
end repeat
set the clipboard to url_list
end tell
I had to use AppleScript Utility to add the script menu to my menu bar. From there it was easy to create script folders that are specific to both WebKit and Safari and save a copy of the script (with the appropriate substitution, see comment in script) into each folder. Now I can copy the title and URL of all my open tabs onto the clipboard easily again, without any InputManager hacks.
I had some recollection that is a way to do this from Python, so I looked and found Appscript. I was able to install this with a simple easy_install appscript and quickly ported most of the applescript to Python. The only stumbling block was that I couldn’t find a way to access the clipboard with appscript, and I didn’t want to have to pull in the PyObjC framework just to write to the clipboard. So I used subprocess to call the command-line pbcopy utility.
#!/usr/local/bin/python
from appscript import app
import subprocess
tab_summaries = []
for window in app('WebKit').windows.get():
try:
for tab in window.tabs.get():
name = tab.name.get().encode('utf-8')
url = tab.URL.get().encode('utf-8')
tab_summaries.append('%s\n%s' % (name, url))
except:
# not all windows have tabs
pass
clipboard = subprocess.Popen('pbcopy', stdin=subprocess.PIPE)
clipboard.stdin.write('\n\n'.join(tab_summaries))
The remaining hurdle was simply to put the Python script I’d written into the same Scripting folder as my AppleScript version. For me this was ~/Library/Scripts/Applications/WebKit/. When run from the scripts folder, your usual environment is not inherited, so the #! line must point to the version of Python you are using (and which has Appscript installed). You should also make the script executable. Adding .py or any other extension is not necessary.
Overall, while I found AppleScript to be very powerful, and not quite as painful as I remembered, I found the Python version (warts and all) to be easier to work with. Combined with the fact that the script folder will run non-Applescript scripts, this opens up new worlds for me. I have hesitated in the past to write a lot of SIMBL-based plugins, tempting though it may be, because they are hacks, and they run in every Cocoa-based application. But adding application-specific (or universal) scripts, in Python, is pure, unadulterated goodness.
]]>What I did over winter vacation
We had an open discussion, starting with the nominal topic of the night, Python packaging. We discussed some of the pros and cons of distutils, setuptools, and easy_install. Our casual conclusion was that all of these tools work better if you first use Ian Bicking’s VirtualEnv to prevent library pollution in your main Python install. VirtualEnv puts a copy of Python, your required libraries, and any dependencies they have into a separate directory and then runs from that, keeping a project nicely sandboxed. The OS X Python application builder py2app, works similarly, but after the fact as a final build step so you can deliver a complete application without worrying what version of Python and libraries are installed on a user’s computer. Ian Bicking also has package installer you can use with (or without) VirtualEnv, called Pip, and if you check his two links here you may notice a pattern forming for his blog titles.
From there we wandered around, topic-wise, discussing XML/HTML parsing libraries such as Beautiful Soup and lxml, from there to screen scrapers like mechanize, to web spiders like scrapy.
Conversation drifted into 3D for a bit, touching on VRML/X3D and the open-source FreeWRL viewer (built by the Canadian Government, yay!). While it isn’t really a Python project, there is a Python library to generate VRML as part of the Scientific library (not to be confused with SciPy. And if you don’t want to go the XML route, you can stay in Python because Scientific can also generate VPython code (although VPython does not yet support things like transparency or texture mapping).
We kept coming back to web frameworks, discussing the tradeoffs between Django and Zope/Plone and the impact of “cloud computing” platforms such as Amazon Web Services (AWS) and Google App Engine. While the AWS services give a lot more flexibility, they require a substantial amount of planning and configuration. App Engine has the easy deployment of a PHP app, but with the convenience of Python and the power of Big Table, but the rather substantial disadvantage of still being a beta environment that you can’t actually buy services on yet (no scaling up).
There are some workarounds for these and other problems (the App Engine 10 app limit, for instance). Hadoop was mentioned as an open-source alternative to App Engine. Amazon Machine Images can give you a head start at deploying on AWS, although you will still need to make arrangements for data persistence. 10gen appears to be building using the App Engine model (more or less), but as a smaller player, they may be more responsive to user feedback. LowEndBox is not a provider, but a blog tracking ultra-cheap hosting with root access, so you could conceivably build your own AWS on a shoestring.
Since I spent part of the holidays building Lego Mindstorms models with my kids (a machine gun and a puppy), I kept trying to steer the conversation towards robotics. Unfortunately, the Python Robotics project does not support Mindstorms yet, but they do support Roombas, so I may still be in luck. I haven’t found a Python project for programming Mindstorms yet, but you could probably wrap Not Exactly C with Python fairly easily, and after the meeting I found this script to use Bluetooth to control Mindstorms from Python.
After that we continued the discussion at the pub and I don’t have any browser history to help me remember that part of the discussion. I do remember that before the meeting started, I did plug Scratch again as the best way to introduce 6-12 year olds to programming.
Finally, I want to thank everyone on the VanPyZ mailing list for helping to organize and to re-establish the group’s web site after recent hack attacks and crashing, especially Henry Prêcheur for restoring the content and moving it to the official Python wiki for the new VanPyZ page.
]]>Anyway, here are some examples showing how to draw polygons with variable opacity (or alpha). I’ve got examples for NodeBox, aggdraw, and pycairo. I was going to do some for the browser canvas, Processing, PyObjC, and SVG, but a) I wanted to get this post out, and b) PyCairo can already output to SVG and Cocoa (and Processing will be very similar to NodeBox). So if there is any interest in those examples, or others I may have forgotten, let me know in the comments.
While the images below look identical (at least they do to me), each was drawn with its respective library.
For each library I will present the drawing function and the main function that sets things up and drives it. At the end I will present the utility functions used which were the same for all libraries. No optimizations here, just the simplest thing I could do that worked.

NodeBox
I began with NodeBox, because it is so easy work with. In Processing they call programs “sketches” and in NodeBox that is very much what it feels like. The first thing I worked up was a way to draw “random” polygons in the same way for each libary. I could seed the random with a set value, but decided to use random to generate starting values, then just re-use those initial values. To get 50 different polygons, I used a list of primes, chose from the list at random, then used those selected values to increment each polygon vertex and color value (red, green, blue, and alpha). All of which was probably more work than was strictly necessary, but should be stable across different versions of Python and different operating systems.
The drawing routines all have the same signature. They take a tuple of point tuples (x,y) and a tuple of colors (r,g,b,a) as ints between 0 and 255.
def nodebox_poly(pts, clr):
fill(*ints2floats(clr)) # convert ints in range 0-255 into floats between 0.0 and 1.0
beginpath() # every time we want to change colours or linewidth we need to start a new path
moveto(*pts[-1]) # start and end at the last point in the list
for pt in pts:
lineto(*pt)
endpath()
def nodebox_main():
size(WIDTH, HEIGHT) # nodebox wants to know how big a canvas you are working with
draw_all(nodebox_poly) # No additional setup needed

PyCairo
The lines in cairo_main where we create the surface and where we save it are all we would have to change if we wanted to generate a Cocoa view, an SVG document, a PDF document, etc. Two of the main differences between this and the NodeBox example are that there is a bit more setup (Cairo doesn’t default to a white background) and we need to have a reference to the context for drawing, where NodeBox has an implicit drawing context.
def cairo_poly(pts, clr):
ctx.set_source_rgba(*ints2floats(clr))
ctx.move_to(*pts[-1])
for pt in pts:
ctx.line_to(*pt)
ctx.close_path()
ctx.fill()
def cairo_main():
# Setup Cairo
import cairo
global ctx
surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, WIDTH, HEIGHT)
ctx = cairo.Context(surface)
# fill background white
cairo_poly(((0,0),(WIDTH,0),(WIDTH,HEIGHT),(0,HEIGHT)),(255,255,255,255))
draw_all(cairo_poly)
surface.write_to_png('cairo_example.png')

aggdraw
The aggdraw example is very similar to Cairo. In fact, if I’d done either of these first, I would have passed the context object around and simply ignored it in NodeBox to eliminate a global variable. Aggdraw can draw in memory without using PIL, but doesn’t appear to have a way to write to a PNG image that way, so we draw to a PIL image (which requires us to flush all drawing operations before writing the file).
def aggdraw_poly(pts, clr):
import aggdraw
global ctx
b = aggdraw.Brush(clr, clr[-1])
pts2 = []
for p in pts:
pts2.extend(p)
ctx.polygon(pts2, b)
def aggdraw_main():
import aggdraw, Image
global ctx
img = Image.new('RGBA', (WIDTH, HEIGHT), "white")
ctx = aggdraw.Draw(img)
draw_all(aggdraw_poly)
ctx.flush()
img.save('aggdraw_example.png')
Utilites
These were the same for all of the examples. If you put it all in one file (or use mine, linked below) you can choose what library to use by switching which main function is called.
MIN_ALPHA = 50
MAX_ALPHA = 100
WIDTH = 500
HEIGHT = 250
#
# Utilities
#
def hex2tuple(hex_color):
return tuple([int(hex_color[i:i+2], 16) for i in range(1,9,2)])
def tuple2hex(tuple_color):
return "#%0.2X%0.2X%0.2X%0.2X" % tuple_color
def ints2floats(tuple_color):
return tuple([c / 255.0 for c in tuple_color])
def inc_point(p, dp):
return (p[0] + dp[0]) % WIDTH, (p[1] + dp[1]) % HEIGHT
def inc_triangle(t, dt):
return tuple([inc_point(t[i], dt[i]) for i in range(3)])
def inc_color(c, dc):
new_c = [(c[i] + dc[i]) % 256 for i in range(3)]
new_a = (c[3] + dc[3]) % MAX_ALPHA
if new_a < MIN_ALPHA: new_a += MIN_ALPHA
new_c.append(new_a)
return tuple(new_c)
def draw_all(draw_fn):
triangle = start_t
color = start_c
for i in range(50):
triangle = inc_triangle(triangle, dt)
color = inc_color(color, dc)
draw_fn(triangle, color)
#
# Starting and incrementing values
#
start_c = hex2tuple('#0xE6A20644')
start_t = (127, 132), (341, 171), (434, 125)
dt = (107, 23), (47, 73), (13, 97)
dc = 61, 113, 109, 41
As you can see, these libraries are all quite capable of producing nice vector-based, anti-aliased polygons with alpha blending. I have heard anecdotal evidence that AGG is faster than Cairo, but Cairo appears to be more widely used. Aside from that I don't know why one would be preferred over the other (unless you want to get into language wars: AGG is written in C++, Cairo is written in C). Both have bindings in a number of languages besides Python, if that's your thing. NodeBox is still my favourite tool for noodling around in with graphics programming. All of these are great fun, easy to use, and handy to have in your bag of tricks.
I tested this under OS X (required for NodeBox), using pycairo installed via fink (using fink's Python 2.5) and aggdraw installed via easy_install (using the builtin in Python 2.5).
Here is my test file containing all of the code above: Alpha polygons example code
]]>This is a collaboration that Daniela and I did together. Her words and voice, my scripting and adding some titles. The words to the poem can be found on her site here (or actually, on any page of her site since it is currently in her sidebar). The animation itself was done in NodeBox (so this example is Mac-only, but it could probably be adapted easily to Processing.
from math import radians, sin, cos
from random import seed
POEM = '''
in earth dreams
her limbs are all sky
her body inhabited
by wild birds and busy
mammals
an endangered species
she lost her head
where reason gives
the impression everything is
clear
she motions to dance
a rhythm spins
and turns attracted
to light
if she had eyes I would see
the stars she holds
dear
then there are all
the words she escapes
slips past their curved
seashell ears as they listen
to capture her
on the page my words
are already changing her
while she weaves gossamer
and soft moss to hang
in autumn woods
'''.split()
size(320,240)
speed(30)
class Vector(object):
def __init__(self, idx):
self._starting_angle = self.angle = radians(random(360))
self.rate = random(0.5,2.5)
self.distance = sin(radians(idx * 2) + 180) * 100
self.x = 0
self.y = 0
def step(self):
self.angle = self._starting_angle + FRAME * radians(self.rate)
self.x = cos(self.angle) * self.distance
self.y = sin(self.angle) * self.distance
return self.rate, self.x, self.y
class Word(object):
index = []
def __init__(self, idx, word):
self.red = random()
self.green = random(0.4, 0.75)
self.blue = 0.4
self.vector = Vector(idx)
self.word = word
self.size = 10 + 2 * len(word)
self.duration = 3 * (len(word) + 2)
Word.index += [idx] * self.duration
def step(self, alpha=1.0):
fill(self.red, self.green, self.blue, alpha)
rate, x, y = self.vector.step()
push()
rotate(rate * FRAME)
font('Georgia', self.size)
text(self.word, x, y)
pop()
def setup():
global words
seed(0)
words = [Word(idx, word) for idx,word in enumerate(POEM)]
def draw():
translate(WIDTH/2 -50,HEIGHT/2)
background(1.0, 0.97, 0.86)
# Add one word at a time, then show all words for 100 frames more
if FRAME < len(Word.index):
last = Word.index[FRAME]
for word in words[:last]:
word.step()
# Now fade out and have blank screen for titles
elif FRAME < len(Word.index) + 100:
alpha = 1.0 - 0.01 * (FRAME - len(Word.index))
for word in words:
word.step(alpha)
I used iMovie '08 to mix the voice and animation and to add the titles. I wasn't very happy with the changes to iMovie, I found the earlier versions were much more flexible and easy to use. On the other hand, I tried Norrkross Movie, that I actually had to buy, and was unable to do it at all. Next time I will just add the titles in NodeBox and only use iMovie to add the soundtrack.
My Python script could probably be cleaned up too. This was what I had after many iterations and experiments to get the timing, color, and rotation where we were both happy with it.
This was my first time uploading to YouTube and the resulting movie is very jumpy, in the original movie I uploaded was not. If anyone has suggestions how to avoid that (whether pre-processing or a better movie hosting service), please let me know in the comments. Overall I was happy with how it turned out and I'm looking forward to our next collaboration.
]]>