Re: DS24 spatial variations

[Tom Droege <tdroege2@earthlink.net>]

Stupendous Man writes,

At 05:30 PM 1/16/03 -0500, you wrote:
>   I agree with Andrew that we should use light-box flats.  However,
>we differ slightly on the next step.  I would say
>
>       a. subtract median darks (which are taken every night)
>       b. use light box flats (which are taken frequently -- I don't
>                 know if nightly is best, but Andrew's idea of
>                 averaging several sets sounds good to me)
>       c. apply a correction factor based on (x,y) position on the
>                 chip; this will be derived from grid tests

For the data sets reported previously that are in the Michael Sallman data 
base, I used sky flats.  We can now see that this produces a scaling error 
if there is a gradient in the sky.  This is a fact for me here in 
Batavia.  It may be less of a problem at a good location but that does not 
help me here.  So sky flats are out if they are used as a division.

So I agree with a. and b. above.

I will take light box flats at intervals and see how much they change.  I 
will also take darks at intervals and see how much they change.  I went to 
some trouble to try to temperature control the darks so that frequent darks 
would not be required.  My real suspicion is that the sky brightness 
dominates everything and its noise is much larger than any changes in darks 
or flats.  But the work has to be done to confirm this.

What do do after a. and b.?  If I understand the star finding algorithm 
that comes next, it will be biased if the field is not flat.  We could:

1) Generate a flat from the dark subtracted and light box flat fielded 
image generated by a. and b.  Then we could **subtract** it from the image 
to flatten it.  The problem in doing this is that we somehow need enough 
images to get a median flat.  I suspect that the sky gradient is constantly 
changing as things happen like a local car dealer (a mile or two away) 
switching off their lights.  It certainly always changes with declination, 
so several images would have to be taken at the same declination - and in 
the same direction to avoid reflections from buildings and the like.  A 
real pain, and I think not practical.

2) Force the individual image fields to be flat.  Just continuously 
subtract the background after doing a. and b.  Then find stars and complete 
the pipeline process.

Needless to say, I am promoting 2).  I presently do it by dividing the 
image up into 8 x 8 squares.  (16 x 16 does not take too much time but 
gives little improvement.) I get the median of the whole image and medians 
for each sub square.  I then subtract the sub square median from the over 
all median and add it to the image sub square.  The result is an image that 
is piece wise flat.  One can see the squares in the image, but only because 
the eye is sensitive to such things.  When one looks locally at the square 
boundaries, the noise overwhelms the change at the border.  In practice I 
select the limit below so that the border change is significantly smaller 
than the noise.

I am doing something a little more complicated.  First I get the median of 
the whole image and check it against a limit by filter.  If the level is 
too high I throw out the image.  This gets rid of images with wholesale 
cloud cover and images taken at dawn and dusk.  Next I look at the 
difference between the median and each sub square median.  If this is too 
large a positive number (it can be  negative due to a bright star) I throw 
out the image.  Too small indicates general brightness somewhere else, like 
a cloud patch.  This does a good job of removing images with cloud 
streaks.  Since I want to do the cloud and brightness rejections anyway, 
the smoothing comes almost free.

Next I have set the star finding program to look at stars that are 3 sigma 
instead of 5 sigma over background.  This is now possible because of the 
smoothing.  This generates more dim stars without any apparent increase in 
"noise" stars.  I am still studying this.  The later two filter matching is 
a powerful tool to remove stars that are noise.

OK, what does all this do?  Not much.  There is a small decrease in 
noise.  But it is just discernable.  This is at least not a loss.  I will 
comment on this in another post.  There is a factor of two or so more stars 
found.  This seems to be OK and real so far.  After performing these steps, 
I can no longer see any pattern in the star data taken on the 7x7 
grid.  There may still be some left, but it will require better tools than 
I have to discover it.  I estimate that c. above will be found not to be 
necessary.

I am setting up to process a large data sample by this scheme.  Stop me now 
if this process horrifies any of you.

Tom Droege