Posted by Phil Tomson, Saturday January 26 2002 @ 04:06AM EST
I've developed a task distribution system I call TaskMaster.
TaskMaster distributes 'runnable' task objects to a set of client machines on a network or in a cluster. Task objects are 'runnable' so long as they define a 'run' method in their class definition - it should be noted that TaskMaster is written in Ruby a dynamically typed, object oriented scripting language. Due to Ruby's being dynamically typed, it doesn't matter what type your task object is, only that it's public interface defines a 'run' method. TaskMaster distributes these task objects to available clients and the clients invoke the 'run' method of the task objects. Users of TaskMaster can create their own runnable task objects so there is no limit to the types of tasks which can be distributed. After tasks complete clients are 'harvested' (results and data are returned from the client) which makes them available to receive more tasks.
TaskMaster was originally developed as the heart of a distributed test system in which complex testcases were distributed to multiple client machines. Since there were thousands of testcases in the testsuite, distributing testcases to multiple clients greatly increased throughput.
TaskMaster is well suited for many types of applications where tasks need to be distributed to multiple clients, such as:
* testcase distribution to multiple clients in order to speed up regression testing
* distributed builds (with a networked filesystem)
* building and installing software on multiple machines simultaneously (using TaskMaster's broadcast mode)
TaskMaster is very flexible and can be configured by programming in Ruby (see:
http://www.ruby-lang.org for more info on Ruby. )
You can download TaskMaster from:
Please note that TaskMaster is currently usable, but that the API is still subject to some change. I also need to create more documentation and some examples. If you want to try it out you'll need to install Ruby (see the Ruby link above). Within the next week or two I'll be including some demo applications which can be used as examples for building your own distribution apps.
Finally, TaskMaster is open source software available under GPL and Ruby's license.
Video - The Road to PetaFlop Computing
Explore the Scalable Unit concept where multiple clusters of various sizes can be rapidly built and deployed into production. This new architectural approach yields many subtle benefits to dramatically lower total cost of ownership.
White Paper - Optimized HPC Performance
Multi-core processors provide a unique set of challenges and opportunities for the HPC market. Discover MPI strategies for the Next-Generation Quad-Core Processors.
Appro and the Three National Laboratories
[Appro delivers a new breed of highly scalable, dynamic, reliable and effective Linux clusters to create the next generation of supercomputers for the National Laboratories.