Re: Fwd: [Aavso-photometry] pipeline

[Thomas Droege <tdroege2@earthlink.net>]

Let me try to put it in a slightly different way.  If one attempts to take
only perfect data, then one will take very little data.  Chances are that
unless one understands what is important,  bad data will still be taken and
that good data will be discarded.  

The solution is to take as much data as possible with as much information
as possible as to how the data was taken.  Then as time goes on and the
data is studied, then it might be possible to sort out the things that are
important for taking good data.  Then it will be possible to select the
good data.

At the moment, I have very little indication as to what will make good
data.  I am particularly suspicious of the I data since I think things are
going on in the sky that causes big problems with the I data.  I don't yet
know how to characterize this.

I agree with Michael on sanity checks.  One interesting plot is to take a
plot of sigma-vs-mag of all the data and then draw a trend line and select
all the measurements that are say 2 sigma above the trend.  Just use a
straight line on a long scale.  Now plot these points.  One gets a bunch of
horizontal lines spaced at 4 degrees. (by using the data taken on even RA
positions one can get vertical lines.) There is obviously a difference
between the top and the bottom of the frame.  Why? 

>   Double-check, triple-check, and check again.  I think this is 
> especially important when dealing with data from an automated 
> instrument, or, in fact, any data which one has not gathered
> oneself.  
>

Ahhh!  But what to check?  

I think taking the data oneself does not help much.  If one knew the right
things of concern one would take care of them.  But there are many things
of concern.  I repeat my point about field size.  We need some lurkers with
Schmidt camera experience to show up and say "Why aren't you making the
Framstad-Barnhoff correction for water vapor when taking this wide field
data?"  Or some such thing that someone has found to be a problem with
similar data.   

Tom Droege


> [Original Message]
> From: Stupendous Man <richmond@stupendous.cis.rit.edu>
> To: <tass@listserv.wwa.com>
> Cc: <mwrsps@rit.edu>
> Date: 1/2/2004 12:26:32 PM
> Subject: Re: Fwd: [Aavso-photometry] pipeline
>
>
>   Tom wrote:
>
> > One thing that I have learned is that if you try to be perfect, nothing
> > will come out of the end of the pipeline. ...
>
>   I'm working on reducing some of the accumulated Mark IV data right
> now, and one thing _I've_ learned is that it's (almost) impossible to have
> too many cross-checks.  One of my goals is to figure out which
> of the data was taken under the best sky conditions, so that I can
> concentrate on it (for some purposes -- for other purposes, one
> might want to look through all the measurements).  I _thought_ that
> I'd found a good way to characterize each night, or portion of a
> night, as "good" or "not good" (see section 2 of TN 97).
> Earlier today, however, when making a graph of sigma-vs-mag from
> a section of "good" data, I discovered that a bunch of measurements
> were way, way off the rest, by 0.3 or 0.4 mag!  Good thing I 
> made that little sanity check.  After a little digging, I found
> that all the bad measurements came from the I-band camera on
> one particular night (the latter half of JD ...2817, June 26, 2003).
> A little more checking showed that my "night-quality" tool 
> actually _had_ shown this night to be abnormal, but I had
> not noticed and accepted it as "good" anyway.
>
>   Double-check, triple-check, and check again.  I think this is 
> especially important when dealing with data from an automated 
> instrument, or, in fact, any data which one has not gathered
> oneself.  
>
>   I will be going through the data Tom collected in the summer and fall
> of 2003, looking for systematic errors in photometry as a function
> of position in the focal plane.  My first efforts on TOM1 for June, 2003,
> show the same sort of pattern I noticed in earlier data (see TN 97,
again).
> I'll be checking TOM2 and TOM3 for such effects as well.  I'll
> try to make some empirical correction for the errors, then re-reduce the
> data, and see if the scatter at bright magnitudes diminishes.
>
>   It will take a while -- my poor old computer requires about six hours
> to chug through the main bit of analysis for a month worth of data
> from one unit -- but I'll try to give little updates as I go.  
>
>                                            Michael
>