Re: 1999 Mark III Data Collecting Season

[Chris Albertson <chrisja@jps.net>]

I seem to have written too much here.  Subjects are
1) I'm going to have to fix my Mk III camera
2) I agree with Nick we need a "Hands off" data reduction pipeline.
   I have a few ideas about how to do it.
3) How to do better FITS--> JPG conversion

Nick Beser wrote (with edits):
> 
> On Sat, 20 Mar 1999, Tom Droege wrote (with edits):
> 
> > Three Cheers for Glenn!!!   It is great that you are still running the Mark
> > III.  Anyone else?  

I think mine may be broken.  Tomorrow I'm going the take it down off the
roof and bring it and it's control computer in the house and try to
figure out what's wrong.  I hope it is something simple.  One thing
about Redondo Beach California is that we don't have seasons.  Good and
bad nights happen pretty much at random all year round.  What I'd like
for the Mk IIIs is automated software that can toss out the bad frames.
Then I'd run every night in hope of a few good hours.
 
> 
> We have been running the JHU/APL system (typically 4 nights per month
> weather permiting). The system is now on automatic data collection (read
> that I start a script file on a solaris system that turns on the pump
> power, and enables the doors, and then starts data collection and then
> move the data to an archive in the morning. The doors to the telescope
> are automatic. I have a large backlog of data that needs to be processed.
> 
> If you look on our club webpage (http://www.jhuapl.edu/APLastronomy), you
> will see our call for help for data crunching. Unfortunately since the
> TASS analysis software is all Windows 95/NT based, I can't setup an
> automatic starlists generation chain.

If I had time, I'd start work on an IRAF based data reduction pipeline.
two reasons for this:
1) It would be nice to have.  As Nick says he wants a batch oriented
   "hands off" system.
2) It is best (IMO) to have two independent data reduction pipelines.
   They serve to check each other.  
3) The Mark IV data will be reduced this way.  We could get a jump on the
   development of a Mk IV pipeline by setting up a Mk III pipeline.

I noticed that IRAF's "ccdproc" task knows how to reduce drift scan
images.  This will compress dark and flat frames into vectors and apply
them just like STAR.EXE.  It can also strip the non-image pixels too.
After this there are a number of ways to extract objects and do a catalog
match.  The good thing is that because the system knows about drift scan
images we can implement a second reduction pipeline with no real programing
just a bit of scripting and much trail and error.

To feed the above real-time pipeline I thought of a real kludge/hack
that would work.  Linux (and other UNIX) systems have a file that defines
how printing is done.  It is /etc/printcap.  One of the things you can
specify there is the name of a program that all files to be printed much
first be "run through".  They call this the "printer input filter".
Another thing that you can specify is that print jobs are to be sent to
some other computer on the network for printing.  My printers are called
"HP_Laser" and "Epson_Stylus".  I could define a third called "TASS_Reduce".
The input filter associated with this printer would run the data reduction
pipeline on the data and never actually send it to a real printer.  This
filter would live on the data-muncher computer.  It could even be a Sun
SPARC and DEC Alpha.  On the real-time computer the "TASS_Reduce"
printer would be set up to pass the files off to other computer. 

Given the above, when the driver closes a FITS file it would only have to
issue the command "lpr -P TASS_Reduce foobar.fits" and the file would be
sent off to the data reduction machine.  If that machine where down the file
would wait in the local queue area.  Once the data-muncher came up all the
files would move to that machine's queue area to be worked on one at a time.
The neat thing is that queue management is "free".

Still we'd need an automated "hands off" data reduction pipeline.

 
> My latest problem is that the FITS to jpeg software does not do a
> particularly impressive job of converting the image. Somehow the other
> FITS viewers transform the 16 bit binary image into a better looking star
> image. The code that I have does about the same job as the public domain
> fitstopgm utility (i.e. really poor).
> 
>  The
> image display is the last function that I need to prior to setting up the
> javascript on the web page.

Are you by any chance using that program I wrote called "fits2raw"?  If so
all it does is a very simple linear mapping of gray scale values.  It finds
the FITS pixel with the minimum and maximum ADU values and maps that range
onto the range 0...255.  It would be easy apply a log function or to use
say, +/- 2.5 sigma in place of min and max.  If you want I could do it or I
could point out the place in the code where I'd change the mapping function.
You may want to try it yourself so you can experiment and get the "look"
to your liking

One other thing that I could do that is specific to TASS Mk III images is
to use only the central pixels as we have non-image pixels on the edges.

-- 
   --Chris Albertson             home: chrisja@jps.net        
     Redondo Beach, California   work: chris@topdog.logicon.com