Living Code

The other day, Thomas Guest talked about drawing chessboards, and ended with a challenge. I wanted to answer a different challenge, however. What if, instead of drawing on a rectangular grid, we wanted to draw on a hexagonal grid? The following is my slapdash answer. For real-world use I’d make nice classes and pass more parameters to the methods, but to demonstrate the math I’m just going to use global constants and functions.

Like Thomas’ article, I will show solutions using several different tools, in this case Apple’s Core Graphics, PyGame, Python Imaging Library (PIL), and SVG. All of these solutions will use the same constants and math:

# Constants used by each solution

from math import sin, cos, pi, sqrt
THETA = pi / 3.0 # Angle from one point to the next
HEXES_HIGH = 8 # How many rows of hexes
HEXES_WIDE = 5 # How many hexes in a row
RADIUS = 30 # Size of a hex
HALF_RADIUS = RADIUS / 2.0
HALF_HEX_HEIGHT = sqrt(RADIUS ** 2 - HALF_RADIUS ** 2)
IMAGE_WIDTH = int(RADIUS * (HEXES_WIDE * 3 + .5))
IMAGE_HEIGHT = int(HALF_HEX_HEIGHT * (HEXES_HIGH + 1))

# Functions (generators) used by each solution

def hex_points(x,y):
    '''Given x and y of the origin, return the six points around the origin of RADIUS distance'''
    for i in range(6):
        yield cos(THETA * i) * RADIUS + x, sin(THETA * i) * RADIUS + y

def hex_centres():
    for x in range(HEXES_WIDE):
        for y in range(HEXES_HIGH):
            yield (x * 3 + 1) * RADIUS + RADIUS * 1.5 * (y % 2), (y + 1) * HALF_HEX_HEIGHT

Now, given the above, what does the code look like to draw the hexes? Because each library handles colours slightly differently, we will need a generator for colours (and we will need more than just black and white as the chessboard used, because each hex borders on six others). I haven’t given a lot of thought to optimal colouring schemes: each colour generator simply produces red, yellow, blue, and green in a cycle. Here is the image produced by the Core Graphics solution, followed by the code:

def quartz_colours():
    while True:
        yield 1,0,0,1 # red
        yield 1,1,0,1 # yellow
        yield 0,0,1,1 # blue
        yield 0,1,0,1 # green

def quartz_hex():
    '''Requires a Mac with OS 10.4 or better and the Developer Tools installed'''
    import CoreGraphics as cg
    colours = quartz_colours()
    cs = cg.CGColorSpaceCreateDeviceRGB()
    c = cg.CGBitmapContextCreateWithColor(IMAGE_WIDTH, IMAGE_HEIGHT, cs, (0,0,0,.2))
    c.saveGState()
    c.setRGBStrokeColor(0,0,0,0)
    c.setLineWidth(0)
    for x,y in hex_centres():
        c.beginPath()
        c.setRGBFillColor(*colours.next())
        points = list(hex_points(x,y))
        c.moveToPoint(*points[-1])
        [c.addLineToPoint(*pt) for pt in points]
        c.drawPath(cg.kCGPathFill)
    c.restoreGState()
    c.writeToFile("quartz_hexes.png", cg.kCGImageFormatPNG)

Now for some cross-platform examples. Here is the image generated by PyGame, followed by that code:

def pygame_colours():
    while True:
        yield 255, 0, 0 # red
        yield 255, 255, 0 # yellow
        yield 0, 0, 255 # blue
        yield 0, 255, 0 # green

def pygame_hex():
    '''Requires PyGame 1.8 or better to save as PNG'''
    import pygame
    pygame.init()
    screen = pygame.display.set_mode((IMAGE_WIDTH, IMAGE_HEIGHT))
    colours = pygame_colours()
    for x,y in hex_centres():
        pygame.draw.polygon(screen, colours.next(), list(hex_points(x,y)))
    pygame.image.save(screen, 'pygame_hexes.png')

When you run the PyGame script, it will actually pop up a window very briefly, draw into the window, save the result, and close the window. I also didn’t get the PyGame script to add transparency for the background, although I think it could be added fairly easily. Now, for the web, here is a solution in SVG, with the image captured by screenshot in Safari, followed by the Python code, and the resulting SVG code:

def svg_colours():
    while True:
        yield 'rgb(255, 0, 0)'
        yield 'rgb(255, 255, 0)'
        yield 'rgb(0, 0, 255)'
        yield 'rgb(0, 255, 0)'

def svg_hex():
    out = open('svg_hexes.svg', 'w')
    print >> out, '''<?xml version="1.0" standalone="no"?>
    <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
    <svg width="%spx" height="%spx" version="1.1" xmlns="http://www.w3.org/2000/svg">''' % (IMAGE_WIDTH, IMAGE_HEIGHT)
    colours = svg_colours()
    for pt in hex_centres():
        print >> out, '<polygon fill="%s" stroke-width="0" points="%s" />' % (colours.next(),  ' '.join(["%s,%s" % (x,y) for (x,y) in hex_points(*pt)]))
    print >> out, '</svg>'
    out.close()

And here is the SVG created by the above script: SVG Hexes

Finally, one library which overlaps with the ones used by the Chessboard example: Python Imaging Library.

pil_colours = pygame_colours  # same format works, so we'll re-use it

def pil_hex():
    import Image, ImageDraw
    image = Image.new("RGBA", (IMAGE_WIDTH,IMAGE_HEIGHT), (0,0,0,0))
    colours = pil_colours()
    draw = ImageDraw.Draw(image)
    for x,y in hex_centres():
        draw.polygon(list(hex_points(x,y)), fill=colours.next())
    image.save('pil_hexes.png', 'PNG')

That’s it for my examples. Thomas ended with a challenge for displaying chess, and for describing the position. To describe the position, I would use a standard chess notation, such as described here. For my challenge, what other formats would be useful to create hex maps in? POVRay? Flash? Any other examples out there?

Well, I’ve been promising this off and on here in my intermittent blog, but I’ve had the code up on Google code hosting for some time now, my kids have tested out the latest version, I’ve fixed some bugs introduced when PyObjC switched from distutils to setuptools. It is still pretty raw, unpolished, unoptimized, but I’m ready to let the world see it and let me know what they think.

Current features of Drawing Board:

• Freehand sketching: This is the main point of the software
• Onion-skinning: See previous frame dimmed behind current frame
• Create new frames: Either blank, or containing a copy of the current frame. Copied frames include the entire undo stack for the frame, so you can copy, and then undo a portion of the frame
• Change colours and line sizes
• Change the opacity of the window (this is a hack to allow you to trace images until I get image backgrounds implemented)
• Scale and translate the frame
• Remove current frame (not undo-able)
• Export as SVG
• Export frame as PNG or JPEG (PNG includes alpha for any area not drawn)

There is basic file handling, which may be useful as example code for learning Cocoa programming using Python. I’m still working at adding drawing tools besides freehand drawing, and I have ideas for a lot of other things, but the main idea is to keep the program from getting in your way–to keep as close to possible as sketching with a pencil on paper, but to make the process of creating simple animations easier.

Two features that are pretty close, and are important to the goal of the project, are export as Flash and export as Quicktime. Those will be coming sooner, rather than later.

The project page is at http://livingcode.org/project/drawingboard and you can find links there to the binary download, the source repository, and the bug/feature tracker. I’ve also set up Google groups for the Living Code projects: http://groups.google.com/group/livingcode-users and http://groups.google.com/group/livingcode-developer for ongoing discussions.

A few people have seen me demo this program at the Vancouver Python Workshop and at Bar Camp Vancouver and expressed an interest, so I hope it can be of use, both in learning to program OS X with Python, and for creating animations. Please let me know what you find useful and what could be improved!

The Problem

I first saw the term “tab dump” on Dori Smith’s blog, but I immediately recognized the concept. I keep Safari running all the time and with the help of Hao Li’s wonderful extension Saft I keep everything in tabs in one window. Among its many features, Saft will let you consolidate your windows into tabs of one window, and it can save the tabs you have open when you close (or crash) Safari, and re-open them automatically when you start Safari again. What it doesn’t do is give you a list of all the tabs you have open in text format, suitable for blog or email. I don’t currently put tag dumps on the blog because a) I’d feel guilty doing that without adding at least a short comment for each link, which would take too much time, and b) because this isn’t really a link blog, more a place for me to bash out example code and tutorials. At least, that’s how I think of it.

I do however, find Safari teetering on the brink of being unfunctionally slow because I have so many tabs open, and often they’re only open because I want to remember to do something with them later, or come back to them, or some other reminder-type function. So I send myself a tab dump on a more-or-less daily basis. Firefox has tools to help you do this, but I haven’t seen anything for Safari, possibly because you can’t really do it with a Safari plugin, but need to use an InputManager, which is fairly deep magic, and basically a hack, an abuse of the system.

On the other hand, I couldn’t keep using Safari if it wasn’t for Saft, and Saft is an InputManager. Another tool for blocking ads and such (which Saft also does) is PithHelmet, but the interesting thing to me about PithHelmet isn’t that it is a popular ad blocker, but that the Mike Solomon (who wrote PithHelmet) decided to not just make an InputManager, but to make the only InputManager you’ll ever need. You see, PithHelmet itself is not an InputManager, it is a plugin for SIMBL (also by Solomon), which is an InputManager that loads plugins based on the application(s) they claim to support. InputManagers get loaded by every application (Cocoa apps, at least), so you have to be careful you’re in the app you want to modify, and take steps not to break things. SIMBL takes care of the nasty business of being a well-behaved system hack, and your code can assume it is in the right app, because it doesn’t get loaded otherwise.

The Goal

Once I figured out that the only way I was going to get Tab Dumping behaviour into Safari (because Safari tabs don’t play well with Javascript, that turned out to be a dead-end), I decided to try writing an InputManager in Python. SIMBL is open-source, so at first I was looking at the code to see what I need to do to create an InputManager (remember, this is a hack, so Apple doesn’t document it very well). I also read Mike’s essay Armchair Guide To Cocoa Reverse Engineering. What I decided was that, rather than recreate the functionality in SIMBL using Python, I would just create a SIMBL plugin in Python.

Getting started wasn’t too bad, but I found one issue in the above essay that stumped me for awhile. Mike recommends you put your initialization code into a class method load() which gets called after your class is loaded. I don’t know if it is artifact of using PyObjC or what, but my load() method was never getting called. What I did instead was to run the command-line utility class-dump on another SIMBL plugin to see what they were doing. They were using the class method initialize() rather than load and when I switched to that things started working, where by “things” I mean, “I could print to the console to see that my class had loaded.”

The Solution

The next step was to actually do something once I had my code loading into Safari. The tab behaviour of Safari isn’t part of WebKit, so it isn’t documented anywhere. Once again, I used the handy class-dump utility. This is a fabulous tool which will read any Cocoa library, bundle, or application and produce a pseudo-header file showing all the objects and methods defined. I still had to try a few different paths to get to the tab information I wanted, but it was pretty easy, armed as I was with Python and the output of class-dump. Here is the result:

import objc

from Foundation import *

from AppKit import *

class TabDump(NSObject):

    # We will retain a pointer to the plugin to prevent it

    # being garbage-collected

    plugin = None

    @classmethod

    # the following is not strictly necessary, but we only

    # need one instance of our object

    def sharedInstance(cls):

        if not cls.plugin:

            cls.plugin = cls.alloc().init()

        return cls.plugin

    @classmethod

    def initialize(cls):

        app = NSApp()

        menu = app.windowsMenu()

        cls.item = NSMenuItem.alloc().initWithTitle_action_keyEquivalent_(

            'Dump tabs to clipboard',

             'tabdump:',

             '')

        # should be after "Previous Tab" and "Next Tab"

        menu.insertItem_atIndex_(cls.item, 6)

        cls.item.setTarget_(cls.sharedInstance())

    def tabdump_(self, source):

        output = []

        app = NSApp()

        for window in app.windows():

            if window.className() == 'BrowserWindow':

                controller = window.windowController()

                for browserWebView in controller.orderedTabs():

                    output.append(browserWebView.mainFrameTitle().encode('utf8'))

                    output.append(browserWebView.mainFrameURL().encode('utf8'))

                    output.append('')

        self.copyToPasteboard_('\n'.join(output))

    def copyToPasteboard_(self, string):

        pasteboard = NSPasteboard.generalPasteboard()

        pasteboard.declareTypes_owner_([NSStringPboardType], self)

        pasteboard.setString_forType_(string, NSStringPboardType)

As you can see, on my class being initialized, I create a new menu item and insert it into the Windows menu. This could be more robust, by testing menu item names to make sure I’m in the right place, but it works for me, and simple code is more maintainable code. I create an instance of my object and make it the target of the menu item. Pretty basic stuff.

When the tabdump method is called (by selecting the menu item in Safari), it walks through Safari’s window objects (of which there are many) until it finds browser windows, then it extracts the tabbed views from the browser windows to get the titles and URLs involved. When it has collected all the title/URL pairs, it turns it into a big string and puts the string on the pasteboard. Here is where we could be a lot fancier. I’m just putting title/URL pairs, separated by newlines in plain text, because that’s how I mail them to myself. You could easily create Markdown links or any other format here. You could turn them into HTML and put them on the Pasteboard that way. There’s a lot you can do, and the Firefox tool I used to use to do this offered so many options that I was never sure what most of them actually did. Here you can customize the code to do exactly what you need, and keep it simple.

Building the plugin

I haven’t tested this with multiple windows, or with a window with only one tab. It might work, might not. I don’t plan on using it that way, and if I do, it’s easy enough to fix. Now, there is one more thing you’ll need, which is the setup.py script to build it. Assuming you’ve saved the above code as TabDump.py, the following script should be what you need:

'''

    Minimalist build file for TabDump.py

    To build run 'python setup.py py2app' on the command line

'''

from distutils.core import setup

import py2app

plist = dict(

    NSPrincipalClass='TabDump',

    CFBundleName='TabDump',

    SIMBLTargetApplications=[

        dict(

            BundleIdentifier='com.apple.Safari',

            MinBundleVersion='312',

            MaxBundleVersion='420')],

)

setup(

    plugin=['TabDump.py'],

    options=dict(py2app=dict(

        extension='.bundle',

        plist=plist,

    )),

)

In the above file, MinBundleVersion and MaxBundleVersion can keep your code from being loaded if an untested version of the application is running. I have more-or-less dummy values there, don’t treat them as the right thing to do. The SIMBLTargetApplications key holds a list, so if you want your code to load in other applications, add more dictionaries to the list.

Also note that you can build your bundle with python setup.py py2app -A to create a development version (can’t ship it that way) that is all symlinks, so you can edit TabDump.py to make changes without having to rebuild the plugin. If you modify the MinBundleVersion or MaxBundleVersion you will have to rebuild to regenerate the property list (or move the property list to be an external file rather than generating it in setup.py), but that should be an infrequent operation. More importantly, you can put a symlink to your bundle in your ~/Library/Application Support/SIMBL/Plugins/ directory. Then you can make changes to the python code and test it by simply restarting Safari. WARNING: If you have a syntax error in your file, Safari will most likely hang on restart. Just force quit it and check your console for the error to fix.

The Promise

Now, if you’ve followed along with me so far, I’d like to point out a few things that are really freaking cool about this. Item the first: You now have Python running in Safari. Can you think of anything else you’d like it to do while you’re there? I bet you can. Item the second: You can do this in any Cocoa-based application just as easily. Problems in Mail.app? Frustrated by iChat? Just fix it. Take control of your own applications! Make the computer work for you, not the other way around. Item the third: dump-classes gives you the keys to the kingdom. Seriously, the combination of being able to embed Python and get a listing of the objects and methods at will is so powerful that when I got TabDump working late last night and realized what I’d just done (i.e., these three things), I was barely able to get to sleep after that. The possibilities are endless.

If you use this and do something cool with it, please drop me a line and tell me about it. I’m really looking forward to hearing about what kind of cool ways we can push our existing applications.

Correction [2006-08-30]

The class-dump utility rocks, and you should add it to your arsenal of Cocoa tools, along with Python and PyObjC. Since I’ve found it it has already become indispensable for examining existing applications that I want to, er, adjust. Here’s what I’ve learned so far.

First, I want to update my previous post to talk a little bit more about the command-line utility class-dump. This is a fine tool that lets you introspect a Cocoa bundle (plugin, library, or application) and prints out a header file describing all the objects and methods in that bundle. I didn’t mention where to get it, and at BarCamp this weekend I gave some mis-information by telling people it came with Apple’s developer tools, which is not true. I assumed that’s where it came from, because I didn’t remember hearing of it before reading Mike Solomon’s Armchair Guide to Cocoa Reverse Engineering, which refers to classdump without any explanation of where to get it. I tried it, found class-dump worked (tab-completion is your friend), and assumed it came with my system, when in fact I had installed it earlier after reading about it on another blog (I’m afraid I don’t remember where) meaning to try it out, then forgotten about it. So it was there, waiting for me, when I discovered a need for it.

So the truth is, class-dump is a utility written by Steve Nygard. He says it provides the same output as the developer tools command otool -ov, but formatted as a header file. Besides the basic output it can also do various kinds of filtering, sorting, and formatting.
So this is my Tool of the Week (and then some): class-dump. Use it, love it, thank Steve.

[Update: Thanks to Blake Winton for pointing out that the project page link to the Pastels download was broken, fixed now. Also added a link to the project page.]

Pastels is an example project for creating an OS X screensaver in Python using PyObjC. By extension it could be used as an example for building nearly any plugin or bundle for OS X. It started when I had an idea for drawing a simple squiggle, over and over, while cycling the colours and moving the squiggle around. I was very pleased with how it turned out.

Project page: http://livingcode.org/project/pastels/

It’s also my first attempt at hosting an open-source project at Google with their new hosting program. If it works out well I will add more of my projects there, which will save me trying to set up and configure Subversion on Dreamhost for public access (probably not difficult, but one more thing I don’t have to do means more time for writing example code and tutorials).

I’m working on the tutorial text to go along with this project, so ask any questions you have and I’ll try to get to them in the tutorial.

If you are seeing this on my site (as opposed to the Atom feed), there are some changes I’m making to the site that I’d like to point out. I’ve added pages for projects and mini-projects which use the same stylesheet and includes as the rest of the site. I know the stylesheet is uglyless than completely attractive right now–the first thing was to get everything factored and consistent, then to make it pretty. The projects page only has one item on it (Pastels), but that should be changing now that I have the infrastructure set up the way I want it. Nearly all the projects I mentioned in my presentation at the Vancouver Python Workshop will get their own pages soon. More about that in my next post.

Aaron Hillegass’ book, “Cocoa Programmig for Mac OS X” is a great way to learn Cocoa, Objective-C and Interface Builder, and I highly recommend you buy it to understand OS X programming (of course, you’ll be buying the new, improved Second Edition, and I’m working off the first edition here, so things may not completely match up). In order to demonstrate how compact, yet readable, Renaissance + Python can be, I’m going to convert the example programs from this book. Aaron’s company is called Big Nerd Ranch, and I was trying to find something to show which postings were examples from the book, so readers can follow along, but without implying in any way that Aaron has approved of these conversions in any way, so I’m going to call them the Small Nerd Examples (The small nerd being me).

Note that while you are developing, it is inconvenient to have to build the project every time you touch a file, so you can use python setup.py py2app –alias to build it once, then you can edit your files normally and the changes will be reflected when you run your application. Just remember to re-run this when you add a new file, and to re-run it without the alias flag when you’re building any version to be distributed.

Like the previous example, Hello World, this one contains four files. The setup.py file will look radically similar to the Hello World version. The program itself is simple: A window with two buttons, one which seeds the random number generator with the current time, and another which generates and displays a random number between 1 and 100 in a text field.

MainMenu.gsmarkup

<?xml version="1.0"?>

<!DOCTYPE gsmarkup>

<gsmarkup>

    <objects>

        <menu type="main">

            <menu title="ch02" type="apple">

                <menuItem title="About ch02" action="orderFrontStandardAboutPanel:"/>

                <menuSeparator/>

                <menu title="Services" type="services"/>

                <menuSeparator/>

                <menuItem title="Hide ch02" action="hide:" key="h"/>

                <menuItem title="Hide Others" action="hideOtherApplications:"/>

                <menuItem title="Show All" action="unhideAllApplications:"/>

                <menuSeparator/>

                <menuItem title="Quit ch02" action="terminate:" key="q"/>

            </menu>

            <menu title="Window" type="windows">

                <menuItem title="Minimize Window"

                    action="performMiniaturize:" key="m"/>

                <menuSeparator/>

                <menuItem title="Bring All to Front"

                    action="arrangeInFront:"/>

            </menu>

            </menu>

                <menu title="Help" type="help">

                <menuItem title="ch02 Help" keys="?"/>

        </menu>

    </objects>

</gsmarkup>

MainWindow.gsmarkup

<?xml version="1.0"?>

<!DOCTYPE gsmarkup>

<gsmarkup>

    <objects>

        <window delegate="#NSOwner" resizable="no">

            <vbox>

                <button target="#NSOwner" action="seed"

                    title="Seed random number generator with time"/>

                <button target="#NSOwner" action="generate"

                    title="Generate random number"/>

                <textField editable="no" id="text" align="center"/>

            </vbox>

        </window>

    </objects>

    <connectors>

        <outlet source="#NSOwner" target="text" key="outputNum"/>

    </connectors>

</gsmarkup>

ch02.py

'''

Hillegass Example, Ch. 02

'''

from Foundation import *

from AppKit import *

from Renaissance import *

import random

class AppDelegate(NSObject):

    outputNum = None

    def windowWillClose_(self, notification):

        NSApp().terminate_(self)

    def quit_(self, notification):

        NSApp().terminate_(self)

    def close_(self, notification):

        NSApp().terminate_(self)

    def applicationDidFinishLaunching_(self, notification):

        NSBundle.loadGSMarkupNamed_owner_('MainWindow', self)

    def seed(self):

        random.seed()

    def generate(self):

        self.outputNum.setStringValue_(str(random.randint(1,100)))

def main():

    app = NSApplication.sharedApplication()

    delegate = AppDelegate.alloc().init()

    app.setDelegate_(delegate)

    NSBundle.loadGSMarkupNamed_owner_('MainMenu', delegate)

    NSApp().run()

if __name__ == '__main__': main()

setup.py

'''

Run with:

% python setup.py py2app

or

% python setup.py py2app --alias # while developing

'''

from distutils.core import setup

import py2app

setup(

    data_files = ['MainMenu.gsmarkup', 'MainWindow.gsmarkup'],

    app = ['ch02.py'],

)

OK, this time out we’ve got about 52 lines of XML markup and 44 lines of Python. Of course, the program doesn’t really do much more than Hello World, but we’re getting somewhere. If anyone has questions about the code, or want more explanatory text around the Renaissance markup, please let me know in the comments or by email.