RE: Calculation of mag errors

["Michael Gutzwiller" <deepsky@fuse.net>]

I think there are other terms in the sky background noise that are being
left out here.  In addition to the photon noise from the sky background
there is the noise associated with the dark current and in the dark image
used to subtract it out.  There is also a small contribution from the read
circuitry itself.  These all add to the final instrumental uncertainty.

In programs I have written I have not tried to determine all the sources of
background noise but instead measured it in the dark areas of the image
itself.  What I found worked well was to simply assume the following:
* Most of the image was dark, not stars, anything more than 70% works well
* The background noise was reasonably Gaussian

I then calculated the noise in the image by doing the following
* create an array of the differences between adjacent pixel counts
* sort the array
* set minusOneSigma = array value at index 0.1587*(size of array)
* set plusOneSigma = array value at index 0.8413*(size of array)
* calculating the background noise as (plusOneSigma - minusOneSigma)/2.828

This will calculate the actual variance in the background whatever the
source of that variance.  The large differences between adjacent pixels due
to stars are in the tails of the difference array which we ignore in
selecting only the +- one sigma levels.

Calculating the background noise this way worked well on the Mark III
images.  It should work well in determining the instrumental uncertainty.
It does nothing for determining the source of our systematic errors.

-----Original Message-----
From: owner-tass@listserv.wwa.com [mailto:owner-tass@listserv.wwa.com] On
Behalf Of Stupendous Man
Sent: Thursday, March 03, 2005 5:28 PM
To: tass@listserv.wwa.com
Cc: andrew.bennett@ns.sympatico.ca; mwrsps@rit.edu
Subject: Re: Calculation of mag errors


  I have figured out the reason that the database values for
uncertainty in each measurement do not agree with Andrew's
calculations.  The difference is .... sky-subtraction.

  Andrew (and I, in a followup) went through the calculations
of photon noise due to the star+background, and compared it to
the signal in the star.  We measured the sky background value
from a Mark IV image, and measured star signal from the same
Mark IV image.   We found the uncertainty in the instrumental
magnitude for a V=12 star should be close to 0.06 or 0.07 mag.

  I took advantage of Tom's data storage to look in detail
at one particular V-band image which matches one particular
V-band measurement in the database.  The database shows
V=11.927 with uncertainty 0.020 mag.  The processed image
I grabbed from Tom's disks has had

             - dark subtracted (good)
             - flatfield divided out (good)
             - sky background fitted and subtracted (uh-oh)
               
  Tom found that a number of the Mark IV images had large-scale
variations, even after flatfielding, so I put into the pipeline
an option to fit a low-order polynomial (order 1 = linear or
2 = quadratic) to the background sky, then subtract that model
from the image, before searching for stars.  I believe
that Tom does use the sky-subtraction in his normal
processing.

  Now, this means that the mean sky level in the processed
Mark IV images is close to zero.  That means that the 
program which extracts instrumental magnitudes from the image
calculates a very small background noise level -- since it
believes that the mean sky level in an annulus surrounding
the star will indicate the noise level.  The trouble here
is that we've basically subtracted a big fixed number from
all sky values, so the extraction routine ends up with
a background noise value much lower than is proper.

  I went through the calculations for this one particular
star in this one particular sky-subtracted image.  If I 
use the sky-subtracted pixel values, I end up with an
uncertainty of about 0.018 mag in my calculations by 
hand (using square apertures -- don't ask, but it's close
enough for these purposes).  If I instead put back something
like the original sky value into the pixels, and repeat
the calculations, I end up with an uncertainty of 0.052 mag.

  The value of 0.052 mag is pretty close to the typical
uncertainty for a V=12 star which Andrew derived for
one particular image on his own.  I think this means we
have found the discrepancy.

  Consequences?

     - the values for instrumental magnitude uncertainty
           in the Mark IV database are big underestimates;
           we knew that already, since they don't take into
           account systematic errors in flatfielding,
           but now we see that the databased values don't
           even provide accurate measurements of uncertainty
           in the relative magnitude across a single frame.

  Hmmm.  Before this week, I would have told someone to ignore
the uncertainty values for _most_ purposes.  Now, I would tell
someone to ignore them for _all_ purposes.

  Fixes?

     - I can imagine several ways to propagate the information
           about actual sky levels into the calculation of
           uncertainty in instrumental magnitude.  It would
           mean modifying the TCL scripting code which runs
           the pipeline and _possibly_ the C code of the
           "phot" program within the XVista package.

     - we can make available to users of the database some
           better measure of the actual uncertainty in each
           measurement.  There are several possibilities:

             a. based on ensemble solution to all stars within
                a small local area (i.e. based on Mark IV data)

             b. based on "theoretical" calculations of the
                sky brightness, star signal, etc., of the
                Mark IV systems

             c. a simple table of values, based on some  
                real data and/or theoretical calculations

           I imagine these would all be _random_ uncertainties.
           One could also try to calculate
 
             d. the uncertainty in the actual V or I magnitude,
                combining random uncertainties with systematic ones

  A very easy, simple thing to do is to run a few examples of
ensembles of real Mark IV data, look at a few graphs, and make
up a simple table somewhat like this:

         if mag is      uncertainty is
           V = 7          0.02
           V = 8          0.03
           V = 9          0.04
                   etc.

Place the calculations and the table into a Tech Note.  Refer to the
Tech Note somewhere on the Mark IV database web pages. 

  _If_ we desire, we could modify the Mark IV database scripts
(I say, "we", but Michael S. would have to do it, of course)
to use the simple table to look up a value on the fly 
and place that value into the "uncertainty" column of the
output.  In other words, we _could_, if desired, leave the
Mark IV database as it is, not changing any values in
the stored records, but add a bit of code to the 
access scripts so that they use a table value for the
quoted uncertainty instead of the value stored in the database.

  See, that is relatively quick, painless, and easy to do.

  Making changes to the pipeline so that future extracted
magnitude values have proper _instrumental_ uncertainties
would be nice, I suppose.  I suspect that we will always
pay little attention to these uncertainties....

  Comments?

                                    Michael Richmond