Tango has an optional package for enabling high-performance clustered
designs. The package supports distributed instances of cache, queue, and
task (rmi/rpc). All activity on a cluster operates via a named 'channel',
akin to a publish/subscribe channel i.e. once a channel is opened, cluster
operations can be performed upon it. Channels are often named based upon
the content flowing through them. For example, a common idiom is to use the
name of a class, and retain that channel purely for appropriate serialized
instances.
Data held within a cache is temporal, and the content is spread out across
the cluster using a DHT (distributed hash table) approach. Queue instances
are distributed in a similar manner, and are persisted to disk. Execution
of task/rpc is generally synchronous, though queues can be used to house
asynchronous task execution where a decoupled design is more suitable.
Either way, task execution is spread across the cluster in a manner akin to
queuing.
Throughput is far beyond any other similar approach known to us, and the
consistent interface across each of queue/cache/task makes the design
simple to use. We expect to make this functionality available in the next
release of Tango.