Weightless

Introduction

Weightless supports implementing complete Python programs as co-routines, including protocol stacks, such as the HTTP protocol. Weightless consists of three major parts:

compose: program decomposition with coroutines (a la PEP380).
observable: component configuration with the observer pattern (DNA)
gio: connecting file descriptors to a coroutine

Weightless is quite well refactored and small (<1000 loc), fully unit-tested and fast.

This example shows how to use Weightless to create a simple server.

There is a bit more background information motivating many design decisions.

Weightless is in use by Meresco. Meresco uses its own version for which free Debian and Redhat packages are provided.

Sources, Download and Install

(Currently being split into weightless-core and weightless-http)

* Browse Sources * RSS commit log

Check out, run the tests, install and run the example:

(Under construction while at EuroPython 2011)

    $ svn co http://weightless.svn.sourceforge.net/svnroot/weightless/weightless-core/tags/version_0.6.0.1 weightless-0.6.0.1
    $ cd weightless-0.6.0.1
    $ (cd test; ./alltests.sh)
    $ python setup.py install
    $ cd weightless/examples
    $ python httpserver.py
    $ wget "http://localhost:8080/" -O - --quiet
    Hello!
    $

Weightless at Conferences

EuroPython 2011

Beyond Python Enhanced Generators introduces the notion of backtraces, flow control, push-back and local on top of PEP 342 and 280, as implemented in compose().

EuroPython 2010

A Pythons DNA introduces the configuration part of Weightless: the DNA. It introduces a little bit of compose() and program decomposition with generators.

Software Practice Advancement Conference (SPA) 2008

Program Decomposition with Python Generators based on Weightless' compose() has been presented and explored on the Software Practice Advancement 2008 conference. For the results, look here.

Agile Open conference

On Agile Open 2008 the notion on decomposition with generalized generators was introduced in an Open Space session.

Dutch Linux Users Group (NLLGG) 2008

Many parts of Weightless are presented on Nederlandse Linux Gebruikersgroep 2008 in Utrecht. Most notably, coroutines and the way you can use them to escape from call-back hell are discussed. See the callbackvsgenerator stuff on github. (UPDATE July 2010: This resembles a more general example of 'taming callbacks with generators', as presented by Raymond Hettinger on EuroPython: Taming Twisted with Generators.)

Python Users Netherlands (PUN)

During the november 2008 meeting of the PUN in Rotterdam, the demonstration of co-routines is continued. Especially, the way you can do program decomposition using compose is demonstrated using this example code.

Related Work

Below are some other alternatives, initiatives and approaches, with a few short comments on them.

Spasmoidal - a very abstract task-dispatcher based on Python 2.5 generators which features a Pollster (Reactor) and a Socket Acceptor. It clearly suffers lack of program decomposition with generators, as the Spasmodoidal Tasks are very long single generators. Doug's approach is very generalized (it's more than only I/O) and I think that Doug Fort and I can learn a lot from each other!
Twisted - a popular asynchronous networking library that contains many useful ideas. I have had the most experience with this library, and learned how not to solve common problems. It probably is the most popular one in the Python scene, but, as often happens with software once accepted by broader public, it suffers from an old and insufficient architecture, which hinders development. As an example, the Twisted folks seem to talk about but never implement sendfile funtionality. Pitty we had to abandon this one.
Asynchio created by Vladimir Sukhoy contains some very interesting idea's. It implements the Asynchronous Completion Token pattern (Proactor) and it is truely asynchronous. It is a library that focusses on I/O and not on generators, and it is clear that the interface is a bit complicated to create networking programs every day. It might be more appropriate to (re)use parts of it under the Weightless cover.
Peers seems to be gone. I can't find it anymore at viral.media.mit.edu. Peers sources seem to be dead.
Active Objects Only a pattern, not a real implementation. Interesting though.
Thread Safe MultiQueue
Continuations Continuations! Yes! Though this is very interesting, the main reason why Weightless explores generators, is the fact that generators can make event drivven programming as easy (in fact: easier, if you take the problems of threading that appear later into account) as writing a sequential piece of code in a threaded environment. Continuations, I believe - unless properly embedded in langauges such as Smalltalk - will not likely make it because it is too abstract for many human beings (at least me) to understand. But I can't deny George Bauer's work is cool! UPDATE: This seems to be gone. I can't find it anymore. ;-(
Generator Coroutine Access to Its Own Handle (from Doug Fort's Spasmoidal) describes a solution to a common problem you'll hit when diving into this subject: a 'self' or 'this' for generators. Only a pattern description, the implementation is in Spadmoidal. And there is a much better way to do it, btw: use reflection to get 'this' from the callstack with a short and efficient this() method.
MultiTask - This is the only other initiative I have found that really solves the decomposition problem and it combines generators with I/O; two problems Weightless addresses as well. Very neat solution and good implementation!
Cogen
Trampolining Seems to be gone.
Eventlets Looks promising, still have to look closer.
gEvent A very thoughtful piece of work by Denis Bilenko. He presented it at EuroPython and he really knows a lot about this topic. gEvent will probably become widely adopted.
Monocle is a little framework for taming callbacks with generators. It seems compatible with multiple asynchrounous I/O frameworks. It does not do program decomposition though

By Seecr