Re: latest news on Mark IV pipeline

[Tom Droege <tdroege@veriomail.com>]

Michael,

I just checked about 4 I and 4 V random headers on disk 18e and they all 
had   FILTER =  'I       ' or FILTER =  'V   '

Great progress.  You should get 18j in the mail soon.  It has a bias frame 
as the first item, and a readme.txt file that describes what is 
different.  Possibly it will be another good guy.

Tom

At 06:00 PM 5/28/01 -0400, you wrote:

>   I've spent the past few days adding one more major feature to the
>Mark IV pipeline: "bad regions".  I noticed that some of the
>nights on Disk Set 18 feature nasty ice crystals in the I-band
>camera (though some do not).  The crystals show up very nicely
>in the flatfield images, so I've written a new XVista program ("mask")
>which finds them, and modified another program ("phot") so that
>it checks each star against a list of bad regions and marks those
>which fall too close to one.  This should allow me to
>
>        a. see how large errors caused by ice crystals are
>        b. discard any measurement near an ice crystal, if desired
>
>   Also, one minor change.  There was a problem with the "master"
>flatfields I created: because I used the true median of a (small)
>number of night sky images, the master flatfield always ended up
>with a significant fraction of pixels which were IDENTICAL to
>the same pixels in the night sky images.  For example, if 10
>images were used to create a flatfield, then about 1/10 of the
>pixel values in the master flat would be the same as the pixel
>values in each target image.  Later, when the master flat was used to
>flatten the target images, about 1/10 of the pixels would end
>up being divided by themselves (in effect), and the resulting
>corrected frame would have a VERY sharp peak in the distribution
>of pixel values: about 1/10 would all have exactly the same
>value.  It looked okay to the eye, and (probably) didn't make
>any difference in stellar positions or magnitudes, but really
>screwed up my attempts to find a "sky" value.
>
>   So, I modified the pipeline so that it now creates median
>flat (and dark) images by using the "interquartile mean"
>rather than the median.  That is, it finds the 25'th and 75'th
>quartiles of the pixel values at some position, and then calculates
>the mean of all the values in between those limits.
>The big benefit is that the output value is (in general) _not_
>the same as any input value.  When I use this routine to create
>master flats, and then flatten target images with them, the
>distribution of pixel values in the corrected frames is nice
>and smooth.
>
>   It's time for me to stop fiddling with software and start
>reducing some data -- especially since I leave for the AAS meeting
>in, um, five days :-/  I won't have time to put together a kit
>of the new pipeline before I leave, sorry.  But I'll do it after
>I shall have returned, around mid-June.   Warning: there are some
>minor changes to the output file formats.
>
>   Thanks very much to the people who have been testing out the
>current version of the pipeline: Michael Sallman and John McKendry
>have both sent me bug reports.
>
>   I've looked at data from each of the Disk 18 CDs, and it looks
>to me as though the best nights are
>
>            18b, 18e, 18g, 18h, 18i
>
>I _think_ that some of the images on disk 18e may be missing
>FILTER values in the FITS headers; could someone else check on this?
>
>                                               Michael Richmond