Lots of data

["Creager, Robert S" <CreagRS@LOUISVILLE.STORTEK.COM>]

Hey folks,

I started importing lots of data last night.  It was easier than I thought
to re-import the same data over and over and over... again.  Currently I'm
set up to import the data twice (520k stars total), then crunch it, then
repeat 240 times (yeah, right).  Currently, I'm literally just reusing the
same data, with a random filename (one of my unique parameters). But I'm not
sure if this is the right way.

Then, as I was sitting down to type this, I thought of  using the same data,
but adding a random offset in ra/dec to the data so the whole frame moves
some random location (for one import session, not per frame).  This way, I'm
not dealing with truly random data, and I'm not re-using the same area of
the sky, as it were, so the dB will have a larger variety of data.
Thoughts?

While doing the import/crunch work, I'm CPU bound (no disk activity, user at
93%, system 7%).  When vacuuming the dB, I'm disk bound (2-7Mb/s).  I've
15Gb on the disk the dB's currently on, and can move to another disk with
35Gb left if I run out of space.  I'm betting that before the 160 days (16
hours per run, and this will get longer I'm sure) are up, I'll have a new,
faster computer up and running :-)

Robert Creager
Senior Software Engineer
ATS Library Engineering
303.673.2365 V
303.661.5379 F
888.912.4458 P
StorageTek
INFORMATION made POWERFUL