Re: Harnessing Unused Computer Power Over A Network

[BARTHOLDI Paul <Paul.Bartholdi@obs.unige.ch>]

Hello,

Using a "farm" of machines to solve a large problem has been with us for
quite long. Two main products, both PD, are: pvm and mpi . They are 
available through netlib.

But the question is: can you use one or the other for TASS? 
The answer depends on:

- can you split the work in small, more or less independant pieces?
- what are the relations between these pieces?
- how much data are to be transfered to and from the remote machine 
  (input and output) ?
- how much computation is needed for every one piece?

The ideal case is when the amount of transfered data is very small and
the computation time quite long. Then even a slow email through phone
lines can be used. If to the contrary you have a small computation time
on a large data set, then a symetric multiprocessor is probably the
only acceptable solution. The DES Challange, the finding of new larger 
prime numbers (or Mersens prime numbers etc) are good example of the 
first case (widely distributed network). In the past, we ran Monte-Carlo
simulations in galactic dynamic involving more than 30 Sun and Vax stations
situated 500 Km apart, and linked through 64 kbps phone lines.

In the extreme case where all small pieces are not related to each other,
then an other PD product can be used: NQS (Network Queuing System), whose
goal is to "balance" the load between many different processors on a network.
But again the computation should be significant compared to the transfer 
of data. 

With NQS, every station has the same status, any station can
accept work form any station, and the balancing takes into account the 
relative power of each one.

With pvm and mpi, the work is distributed from one central station to
the various "slaves".

In both cases, the number of stations involved is essentially unlimited.

As a simplified, dedicated version of NQS, look at crack, the PD program
used to crack password files (the goal really is to find weak passwords).
The work is also distributed to a set of machines using rsh only and 
according to their relative power. Data are passed to other machines 
using files. The data transfer is very small compared to computation 
time. Tass could develop a similar scheme.

Best regards and whishes for your fascinating project!
                                                          Paul

    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    +  Paul Bartholdi                 Paul.Bartholdi@obs.unige.ch  +
    +  Observatoire de Geneve    tel  +41 22 755 2611              +
    +  CH-1290 Sauverny          fax  +41 22 755 3983              +
    +  Switzerland               www  http://obswww.unige.ch/      +
    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++