Re: Common and uncommon goals and software

[Stupendous Man <richmond@a188-l009.rit.edu>]

  Robert asks a question that many of us have pondered:

> Shouldn't there be an effort to generate a generic data reduction pipeline
> which everyone runs on the data taken from their camera?

  It surely would be nice to have such a beast!  In my opinion, the _main_
reason this hasn't appeared yet (either within the TASS project, or 
within the astronomical community as a whole) is because different 
people have different computing environments.  The generic pipeline --
     
       - in what language would it be written?
       - under what platforms would it run?
       - what sort of graphics would it support?
        
  Within the TASS project, we have members running (various flavors of) Linux 
and members running (various flavors of) Windows, and probably other
OSes, too.  Mike Gutzwiller, to make one specific example, wrote a very
nice piece of software to reduce Mark III scans.  He distributed both
Windows executables and the source code.  I have examined (profitably)
the source code, but never tried to build it on my Linux box; it would
take a LOT of work to port it to this environment.

  There may also be secondary considerations which prevent us all from
settling on one piece of software:

       - preferences for specific algorithms (e.g. PSF-fitting versus
                 aperture photometry)
       - different reference catalogs
       - different interests (e.g. variable stars vs. an all-sky 
                 photometric survey)
       - general pigheadedness: we don't believe that software written
                 by someone else is as good as our own

  I suspect that these are less important than the environment issue.

  Finally, suppose that there were one generic pipeline that satisfied
all of us in its performance on Mark IV data.  The Mark IV has images
4.2 degrees on a side, with pixels about 7 arcsec wide.  Joe Astronomer
at Podunk U. has a different setup: a telescope with a CCD camera which
produces images 0.2 degrees on a side with pixels 0.56 arcsec wide.
His telescope goes down to mag 19 and saturates at mag 12, whereas
the Mark IV covers the magnitude range 8 - 14, say.  Will Joe Astronomer
be able to use the generic pipeline for _his_ data, too?  It might
be possible to write a pipeline controlled by a few tunable parameters
that would take these changes into account .... but that would
be pretty difficult.  It would be much easier to write a pipeline 
which ASSUMES a number of properties about the dataset.  That's what
most of us are doing, I would guess.

  There is a software package used by many different astronomers
to reduce and analyze data from many different instruments.  It is 
called "IRAF", and you can read about it, and download it, from

               http://iraf.noao.edu

I have used IRAF in the past to reduce optical images, and I do have
a copy installed on my computer here at work.  But I shrink from the 
prospect of using it to reduce Mark IV data.  Why?  Well, mostly
because I don't understand exactly what it does.  IRAF is so big 
that it acts like a "black box" to most users.  I haven't ever looked
at the source code (which originally was written in a special
FORTRAN-like language designed specifically for IRAF -- it have been
replaced by some standard language now).  On the other hand, I _do_
know exactly what my own code does, and I can easily modify it 
to do something slightly different, if necessary.

  Another reason IRAF hasn't taken over the world is its steep
learning curve.  One normally runs IRAF from its own command shell, 
which has its own peculiar syntax and conventions.  

  
                                        Michael Richmond