Re: Data Last Night

[Tom Droege <tdroege2@earthlink.net>]

Michael,

My net access is terrible.  I live in a land that time forgot.  Near the 
big city for light pollution, but I might as well be on Mars for high speed 
data access.  I am too far from the exchange for DSL and have a telco that 
does not want to provide DSL.  OK, I could get satellite internet but I 
don't like what I think they are giving me.

The tass data does not compress very well.  You might do .8 or so.  There 
is much discussion in the archive on this.  There is even a tech note that 
reviews all the compression discussion.

Best is to first reduce it to star lists.  This is a factor of 10 or more 
gain.

This has always been one of our goals.  We just don't know how to do it yet.

Tom Droege

At 09:20 AM 4/27/03 -0700, you wrote:
>I don't know what your Net access is like, but could we forgo intermediate 
>media and just transfer it over the Net to Mike S's database? 15 CD's is 
>about 10gig or so. I'm assuming that is uncompressed, so if we gzip'd it 
>we could probably get the transfer time down to a reasonable level. If you 
>assume you could transfer about 100kb/s and that gzip would reduce the 
>size by a factor of 2 or 3, it would be about 10-15 hours. Grab data all 
>night and transfer it all day. If you could run the pipeline and transfer 
>in real-time, in the morning the data would be already in the database?
>
>Probably not practical but something to think about. If we could get a 
>mirror of Mike S's database, then we wouldn't have to worry about the 
>hard-media-as-backup.
>
>Cheers,
>Michael
>
>On Sunday, April 27, 2003, at 01:01 AM, Tom Droege wrote:
>
>>All three telescopes ran all night.  15 CD's were mostly full of 
>>data.  About 1100 images.  Probably over a million measurements.  There 
>>are still some problems with TOM3, the data does not look right so I will 
>>have to test it further.  But it is mostly OK.  Possibly more hanging chad.
>>
>>It took about 3 hours this morning 5 AM to 8 AM to shut down the 
>>telescopes and write everything to disk.  I will patiently wait for DVD's 
>>to get to the point where I can dump a whole night of data from one 
>>telescope to one CD.  It is presently a pain because I have to sit by the 
>>computer feeding it CD's every 10 minutes or so.   Anyone know of the 
>>state of software that would allow writing fits files to DVD?