Re: Data Processing Switch

[Stupendous Man <richmond@a188-l009.rit.edu>]

  Chris wrote:

> There may be a mis-understanding here.  What I thought Tom
> proposed to save was a FITS file with un-processed pixels
> (except for trimming the overscan) and corrected values
> in the header. If that's true: raw pixels and "corrected"
> headers then nothing is lost.  Such files should be able
> to be re-processed.
 
  Nope.  Tom's proposal saves images in which the pixels
have been corrected, too. 

> I read Micheal's pipeline documentation over lunch.  Looks
> like he writes only the image center and image scale into
> the FITS header.  

  True.  I don't even write the image scale, actually.

> I think this should be expanded to include
> a full astrometric solution including the rate of change in
> image scales in X and Y and rotation.  This is required for
> any off-the-equator image.

  I know where the documentation is for the WCS system.  When I considered
writing additional information into the headers, I realized that it
would become complicated (how does one specify a linear vs. quadratic
vs. cubic solution?), so I quit and worked on other things instead.
I guess I ought to go back and see if I can figure out how to 
place more accurate and detailed information into the FITS header,
so that the proper display software will give a position good
an arcsecond or two as one moves the cursor around.

  It's a pain to have image 4 degrees on a side :-(

  Arne wrote:

> My feeling is to save the raw images and discard the processed
> images after you have done the star extraction.  That way you
> save the same amount of data, but can recreate anything.

  Let me put some words in Tom's mouth, if he'll let me.

       a) it is a rare occasion that one wants to go back and
             look at an image

       b) on almost every such occasion, looking at the processed
             image (plus associated flat) is just as good as
             looking at the raw image.  For example, Tom discovered
             that some strange events were caused by satellite trails;
             obvious on the processed as well as the raw.

       c) the time it takes to look at a processed (or raw) image
             is, oh, the time to locate the appropriate CD-Rom, 
             find the right file on that CD-Rom, and display it.
             If the header is (nearly) correct, then one can move
             to a given (RA, Dec) in a few seconds

       d) the time it takes to reprocess a raw image is in the hours

  If these are accurate reflections of reality, then I would
keep the processed images.  

  It is often the case that it takes less work to go and re-observe
a given part of the sky than to locate old data and interpret it
properly.  In some situations, such as verifying a "flash", that's
no good; but in other situations, such as checking the magnitude
or position of a particular star, it's just fine.  We should not
forget that we can use the sky as our archive, in some cases.

                                          Michael