Re: [TASS] New Data Handling Paradigms?

[Peter Mount <peter@RETEP.ORG.UK>]

On Mon, 20 Sep 1999, Tom Droege wrote:

> As a reality check, on the 17th I ran all night and got 75 frames in V and
> I.  The raw data fits on 2 CD ROMS.  In the past raw data/star lists = 10.
> So we might get 120 Mbytes of star lists a night per location.
>
> On the average, I get 6 good nights a month.  This is one CD ROM of star
> lists every 3 weeks or so to mail off to a data base.

Looking at these figures, do you think I should look at stress testing
PostgreSQL to make sure it can handle this amount of data?

I did fix the 2Gb problem for PostgreSQL 6.5, but it might be an idea to
test it again while 6.6 is under development (I suspect that 6.6 will be
out when we start getting some data).

I know there are some large users out there (one with something like
60-Gig of data), but it's still uncharted territory.

Just out of interest, what is the disk useage of the current database?

Peter

> At 06:26 PM 9/20/99 -0400, you wrote:
> >aah@nofs.navy.mil wrote:
> >
> >> I need to look at FITS tables, but I thought they were in ascii.
> >> Note that ascii can be easily compressed, so zip'd versions of
> >> the starlists will be much smaller than the original.  I don't think
> >> you will get hundreds of megabytes of starlists from each site
> >> each night.  The maximum data rate I see is 20 frames/hr or
> >
> >  If we generate ~200 frames per night (and each star-list) is about 3
> >megabytes in size that will generate ~600 Megabytes of uncompressed
> >star-lists per night. The data rate out of each Mark IV camera appears
> >to be about 10x the data rate of the Mark III cameras....
> >
> >Glenn G.
> >
> >
>

--
       Peter T Mount peter@retep.org.uk
      Main Homepage: http://www.retep.org.uk
PostgreSQL JDBC Faq: http://www.retep.org.uk/postgres
 Java PDF Generator: http://www.retep.org.uk/pdf