Bennett's progress (?) March 2001

[Andrew Bennett <andrew.bennett@ns.sympatico.ca>]

I'm writing this a bit early as we are now
pruning the grapes and nothing much to do with
TASS is likely to happen for the next few months.

Tom kindly (?) sent me some more data for the
Dat Set 20 area so, as my PSF processing routines
were "all working" I ran it through. This took
forever, or at least 8 days. I found some new and
fascinating bugs.

I was developing the ensemble processing routines
in the evenings and running the star-fitting the
other 20 hours a day. Each day, less got done. All 
this in a DOS window under Windows 98. Compiling
Ensemble took 1.5 seconds ... 6 seconds ... 25 
seconds ... 65 seconds. With the disk light on all
the time. Eventually, I rebooted DOS. No improvement. 
So I closed everything down, turned the power off and 
started Windows from scratch: first compile under one 
second. It seems Windows has a memory leak in its 
memory management (that I didn't know I was using)!

One of Tom's new day's data brought the whole processing
to a halt - the declination in the header is nearly 2
degrees off and it seems my version of Mike Richmond's
matching routines quit at 1.5 degrees. More tuning. I
got it to work in the end but the RA turns out not to
overlap the other 5 days' data (RA was right in the header 
but I've given up believing headers ...) 8 days to get 2 
extra days worth of usable data for the ensemble processing.

Each added day's data reduced the number of sources
for which an ensemble could be found on every image
so I have abandoned the original Ensemble processing.

The new version attempts to find an ensemble for each
source on each image but with no attempt to use the same 
ensemble each time (since this doesn't work.) This is 
nowhere near as good. There is an extra error term from 
the differences between the different ensembles.

The 5 days images range over 2 degrees in dec and nearly
4 degrees in RA so sources are measured in wildly 
different positions on different images. The corrections
for this are quite large, exceeding 0.1 mags, so if not
corrected there are lots of long-period variable! I 
attempted to calibrate this using the Tycho2 magnitudes 
(modified for V-band and extrapolated to I-band per 
Arne's prescription) but this doesn't work very well.
Many of themagnitudes are a long way off - presumably
some real long period variables plus all the usual
nastinesses of resolved/unresolved pairs etc.
Averaging over 600+ sources doesn't get you even close
to 0.01 mags if there are sources over 1 mag off,
different from one image to another.

Anyway, to cut a long story short, I have cooked up
a method not yet fit for public scrutiny that takes
out a good deal of this gradient stuff so that a
majority of sources don't have 0.1 magnitude steps
between different days's data. But there are still
completely unexplained bad data points with errors
of 0.1 or even 0.4 mags (formal error 0.01- mags):

1) Near the edges of images ... possibly problems
with picking an ensemble.

2) Flagged as bad fits ... possible cosmic rays but there
are too many and too many +ve magnitude errors.

3) Not near edges. Not bad fits. Image appears typical.
I've absolutely no idea.

Grape vines permitting, I will pull out the raw data
for some of these, together with the images on either
side and stare at it. Till I go cross-eyed.

Signing off till October ...

Andrew Bennett, Avondale Vineyard