Random sampled data and real-time data processing

[Chris Albertson <chrisalbertson90278@yahoo.com>]

Two topics here:

1) Random sampled data.

Any references (hopefully web based URLs) on time-series analysis
of random sampled data.  I understand what to do with data sampled
at uniform intervals.  All I really want to know is a bit about
the algorithms how they work and how much computation is required.


2) real-time data processing.

You are right about real time data reduction. There is no reason
to farm out the processing when one PC can do the entire job.

Setting it up is not hard either.  If you can process one image
you can process images in real time

One neat trick I saw once to process a bunch on image-like data
was to make use the the "standard" printer queue.  The system
that had all the data would simply "print" the image to the "first
available printer" of a certain class. (Many OSes allow you to send
a job to for example, "the first color printer on the fourth floor".
The "printers" were UNIX machines with the data processing program
specified as the input filter in their /etc/printcap files.  You
get a lot for free by doing it that way.  You get commands to
start and stop a queue, assign a queue to a device, show status.
e-mail notifications, logging and on and on.  Just one line in the
/etc/printcap does it all.  Now days printing can be done over the
Internet too. 

The total "compute power" required to turn an image into a list of
objects is not large.  We don't need thousands of PCs.  One PC per
camera would do it.  I've always argued for moving the compute
power to where to the data is.  You only have to move the computer
once.  If you move the data you have a lifetime job. 


Doug Welch wrote:
> 
> Tom et al,
> 
> I think that one can very probably keep up with the photometry
> at a single site with a single modern computer. The task
> of time-series analysis on *every* available star would be
> unprecedented and would likely lead to unanticipated findings.
> I am therefore more in favor of farming out lightcurves, perhaps
> in groups of dozens, for distributed time-series analysis.
> 
> Cheers,
> Doug

-- 

--
   Chris Albertson             
   Redondo Beach, California
   home:   310-376-1029   chrisalbertson90278@yahoo.com
   cell:   310-990-7550
   office: 310-336-5189   Christopher.J.Albertson@aero.org