By Seecr

Software Craftsmanship

Seecr - Software Craftsmanship By Erik J. Groeneveld,
This site was last updated on June 24th 2011

Component Configuration with DNA

Configuration consists of DNA describing a graph of observables in a Pythonic way.

Observable

Weightless uses the Observable Pattern to connect components.

Pythonic Implementation

Usually, in the traditional Observer Pattern, observables dispatch messages by calling:

            self.notify_observers('message', arg0, arg1, ...)

The observer receives this message by implementing:

            def notify(self, message, arg0, arg1, ...):
                #do something

A message with arguments is very similar to a method with arguments, and in Python we can implement the Observable pattern in a neat Pythonic way.

For an observer receiving message 'message0' with arguments 'arg0' and 'arg1', we would implement:

            def message0(arg0, arg1):
                # do something

An observable would dispatch this message by calling this method. However, since this method is implemented elsewhere and has to be dispatched by the Observable machinery, the observable calls it on 'self.do':

            self.do.message0('arg0', 'arg1')

The Observable machinery is behind 'do'. It will find the corresponding methods on the observers and call them.

Beyond Observable

Let's stretch up things a bit.

Return Values

When dispatching messages via 'self.all' instead of 'self.do', we get back a generator with the responses of the observers. For example:

            for response in self.all.message0('arg0', 'arg1'):
                print response

Streaming Data

The generator returned by 'self.all' can contain other generators in a recursive way. This tree of generators is flattened by compose in order to support program decomposition. Each observer is able to stream data back to the observable (as in the previous code sample), but each observable can also stream data to the observers using send():

            pipeline = self.all.message0('arg0', 'arg1'):
            for data in mysource:
                pipeline.send(data)

The tree of generators (coroutines) set up by the observers (and their observers recursively) is a pipeline that consumes data and yields a response.

Call Interfaces

In Weightless, an interface consists of a method name. Calling a single method 'doSomething' an getting a single response is done by using 'self.any':

            response = self.any.doSomething(arg)

This will return the response of the first observer that understande 'doSomething'.

Do, Any and All

'All' is the generic form on which 'do' and 'any' are based. 'Any' is equivalent to:

            response = self.all.message0(arg0, arg1).next()

And 'do' is equivalent to:

            for ignore in self.all.message0(arg0, arg1):
                pass

Dynamic Messages: 'unknown'

Sending or receiving notifications without knowing the messages in advance is possible with 'unknown'. An Observable can send messages using:

            self.all.unknown(message, arg0, arg1)

This will transparently call the method 'message' with the given arguments on all the observers.

Similarly, an observer can implement 'unknown' to receive messages dynamically:

            def unknown(self, message, *args, **kwargs):
                # do something

'Unknown' will only be called when no method with the name 'message' is available. This brings us back to the original notify/notify_observers, but with different names.

Labeled Messages

te be determined

Connecting Observables: DNA and be()

The Observable Pattern defines the communication between components. The DNA and be() define the relations between the components.

DNA

Like nature defines structure with DNA, so does Weightless define the structure of an application with its own DNA.

Graph of Observables

Weightless DNA defines a graph of Observables. For example:

            dna = (Component1(),
                      (Component2(),
                        (Component3(),)
                      ),
                      (Component4(),)
                  )

Component 2 and 4 observe component 1 and component 3 observes component 2. Formally, DNA is defined recursively as:

            dna0 = (component0, dna1, ..., dnan)

The components from dna1 to dnan become observers of the component0. The recursive definition allows copy-pasting DNA strands easily, so refactoring is possible.

Graps can be created by extracting a single component or a complete (sub) strand of DNA and assigning that to a variable. This variable can then be used in different places.

Since DNA is normal tuples, and the whole thing is normal Python, you can do anything you like.

Configuration Data

The prefered way of feeding configuration data to components is by passing it to their constructors. Take this simple configuration with two components sharing path information:

            path = '/starthere'
            dna = (Component1('myhost.org', 80, path),
                      (Component2(),
                          (Component3(path),)
                      ),
                  )

Note that sharing configuration data is straight forward, and that Component2, although in the path to Component3, does not need to know path.

Avoid passing through configuration data, and avoid using the environment for it.

Context: transactions, logging and security

Crosscutting concerns like logging, transactions and security all come down to sharing context information between components. Context information is scoped in one thread of control which, in Weightless, coincides with a (tree of) generators.

Observables can set context information by setting attributes on self.ctx. An log component, for example, could look like:

            class Logger(Observable):
                def unknown(self, message, *args, **kwargs):
                    self.ctx.log = []
                    yield self.all.unknown(message, *args, **kwargs)
                    for line in self.ctx.log:
                        print line

It forwards any message transparently using 'unknown', but creates a generator-local log object. This log object collects log messages from all observers as follows:

            class Storage(Observable):
                def store(self):
                    self.ctx.log.append('store')
                    # do work here

Please be aware that self.ctx has per-generator scope much like thread-local variables have.

Meresco-core has a nice transaction implementation, although it uses the now deprecated __callstack_var_ instead of self.ctx.>/p>

Be!

The observables in the tuples are connected by calling 'be'. This will call addObserver to create the appropriate relations:
            appl = be(dna)

The return value 'appl' refers to the root component of the graph. Applications are often started by calling appl.main() or something like that. See also the example.

Component Initialisation

Components may require initialisation after all observers are registered. This can be done by using 'once' or, for example:

            dna = ...
            appl = be(dna)
            appl.once.init()

Unlike 'all', 'any' and 'do', 'once' dispatches the message to all observers and to their observers recursively. It ensures that each component receives the message once and only once.