Re: Correspondence with Bohden Paczynski

[Tom Droege <tdroege2@earthlink.net>]

Chris and all,

I think the bigger bottleneck is always going to be taking the data.  That 
is good.  Last nights data is being processed this morning. It will be done 
in an hour or so.  I could have it merged into one big "collected" list by 
nightfall if I wanted to do that.  By my dumb methods, just using cat, 
sort, and collect stars it takes about an hour to update my big collected 
list.  This presently has 20 million or so measurements.  So I am not all 
that far from a billion.

OK, there is no data base.  Just a big long list.  But if someone asks me, 
I can manually sort out data for a star in a few minutes.  I think most on 
this list (not me) could write a script in a few minutes that would replace 
me.  Of course finding all stars with some characteristic is a different 
matter and the proper subject for a data base.

Tom Droege

At 08:59 AM 9/4/02 -0700, you wrote:
>My opinion is that while we don't really _need_ real-time, (this
>is not a millitary system) doing it in real time with the level of
>automation that it implies is the only way to "keep up"  Just wait
>'till there are six operational Mk IV systems.  We need to use
>processing methods that scale up to on order of 10E9 observations.
>I think we will want to look at the millitary model of data reduction
>and publishing vs. the typical scientic mode of collect data, sit on
>data, publish paper.  OK if you don't like "millitary" think
>"Coca Cola Bottleing plant".  Same thing.  Either way it is
>"non-batch" or continous processing.  One more month 'till luanch...