Re: Analyzing Data

[Arne Henden <aah@nofs.navy.mil>]

So if I understand correctly, you are not using the noise in
each frame at all, but rather just the median sky value, in
determining whether a frame is to be included or rejected?
How do you handle the moon, for example, or different declinations?
Remember that Mike G. used the noise within the frame to decide
whether an image had clouds or not.  This is a more robust test.
Such a test is what I was referring to with <sky> * n*noise
(<sky> is the mean/median value of the sky).  I see your latest
email confirms that you are using n*median, and not n*noise.
Read Mike's old Mark III emails and see if they make more sense.
   Personally, I'd leave cloudy nights, moonlit nights, etc.
in the database.  As more data is collected, you will be able to
tell whether a prior night had higher noise and either delete it
then or give it lower weight.  Sometimes such nights work fine for
measuring bright stars; you just lose the faint ones.  If you try
to decide in more-or-less real time whether data is good, then
you have to work too hard at it. :-)  If good vs. bad is an issue,
then just don't observe when clouds are visible or when the forecast
is for clouds (that will cut down the data to ~50-100 nights per year!).
When I use the 1.0m in its automated mode, I check on the number
of detected stars per frame in a given field, and if it drops below
a certain threshold based on the normal number of detected stars
for that field, then I throw the frame out in the pipeline analysis
step.  Databases are great for finding deviants.
   Regarding use of data below the threshold when performing photometry:
you need to use all data within your aperture for the star measure,
and all data surrounding your aperture for sky measure.  If you use
"zero" then your results will get skewed dramatically.  I may be
misunderstanding, as usual.
Arne

Tom Droege wrote:
> Nomenclature is always a problem for me since I don't have enough 
> contact with the field to learn the language.
> 
> Arne writes:
> 
>> Are you rejecting all points
>> below <sky> + noise, or <sky> + n*noise?  The usual threshold is
>> something like n=3 for star finding
> 
> 
> I look at a bunch of data and note the lowest median values for the sky 
> for the V and I frames.  This is about 1500 ADU for the V channel and 
> 2500 or so for the I channel.  I have the limits set at 3500 and 6000.  
> So any median value above these values get trashed.  This puts me at an 
> n of 2.4 per the above.  I have just been guessing and am interested in 
> wisdom on this subject.
> 
> I am also trying to look for clouds.  The idea is to divide the image 
> into 36 sub images and throw out any image where the median of any sub 
> image is 3 sigma *below* the median of the whole image.  This works for 
> patchy clouds.  I also throw out images where there are more than 3 sub 
> images where the median of the sub images is above the median of the 
> whole frame.  This catches things like sticky shutters without losing 
> very many frames with multiple bright stars.
> 
> This patch rejection should also work for the water vapor if the 
> variation is large enough??
> 
>> though of course all data is
>> used when you go back and do photometry of the detected objects.
> 
> 
> Why?  Seems to me that this increases the noise of the measurements??  I 
> certainly have enough frames to throw away those that might be 
> questionable.
> 
> Tom Droege
> 
> 
> 
> 
> 
> At 12:06 PM 2/8/03 -0700, you wrote:
> 
>>   The water vapor is not a uniform sheet; it looks like cirrus.
>> Ic sees this nicely, especially with wide field cameras; you can
>> watch the vapor bands move across your images.  On the other hand,
>> why your Ic frames are getting rejected is strange to me.  You probably
>> just have the threshold set too high.  Are you rejecting all points
>> below <sky> + noise, or <sky> + n*noise?  The usual threshold is
>> something like n=3 for star finding, though of course all data is
>> used when you go back and do photometry of the detected objects.
>>   Clouds, on the other hand, will affect both V and Ic about the
>> same since cloud extinction is basically "grey".
> 
> 
> 
>