Reply to Brian Skiff #1

[Tom Droege <tdroege2@earthlink.net>]

First a disclaimer.  Brian has written a long and useful note about my tass 
operation.  I very much appreciate his comments.  I do not agree with all 
of them.  This is a point by point response that is intended to open up a 
dialog.   I hope that we will all learn something from it.  I will only 
excerpt the sections as I respond to them.  Those interested in the 
complete text can find it in the 16 September 2002 e-mail to the list.

 >> From what I've seen/read, the MkIV data-reduction seems to be very
 >>shaky, and I myself am not ready to use any of it yet.

Brian, you are not being asked to use the data.  Tass is a 
collaboration.  What goes on here on the list is the equivalent of what 
goes on in an academic institution.  Some of it is "data reduction" 
meetings.  For those unfamiliar with the process I will try to describe 
it.  At one level a number of people are working on the data.  They talk 
about it in the halls.  Everyone is trying to understand the 
data.  Everyone talks about the *bad* things they see.  They will probably 
have data reduction meetings.  Individuals will get up and display a few 
view graphs to describe their work.  This is the function of the tass TNs.

As an example of how this goes,  I worked on a high energy physics 
experiment where the data coming out of the computational pipeline was 
making no sense at all.  We all knew it because we were all talking about 
it in the halls and in the lunchroom.   Just as we talk about problems with 
the tass data here.  I started looking at the data and using the 
measurements to trace out the particle tracks on graph paper.  I found that 
the tracks were not coming from the target.  So I grabbed my graph paper 
and went in to see the group leader and found him doing something 
similar.  We then went in to see the guys working on the software, and sure 
enough they had not considered this possibility.  Soon there was a few tons 
of lead bricks in a strategic spot.  That is the kind of thing that is 
presently going on in tass.  We are working on the data.   All defects and 
warts are exposed.  They need exposure to get fixed or eliminated as a real 
problem.

Tom:
 >>I have commented on this before, but working in public tends to get you a
 >>reputation for sloppy work.

Brian:
 >I didn't intend to mean "sloppy", but instead "incomplete" or maybe
 >"unfinished"---not ready for anyone to use, even those who took the data.
 >The evident non-uniformity of flat-fielding and other calibration, even the
 >continually-changing state of the instrumentation are the sorts of things
 >I'm referring to.

 >> Anymore, I'm not sure that "preliminary" data of the sort that's
 >>coming from the MkIV cameras has all that much value.

How about some error limits?  At what error level would the Batavia tass 
data have value?  When we talk about measurements we also have to talk 
about error.  I have been looking at this.  I would say that it is not 
evident that the flat-fielding is non-uniform.  We have discussed the 
possibility that there is a problem in the flat-field, and I have been 
looking for a problem.  But I have not yet found one.  In fact, I have 
found the opposite.  But I have not yet given up.

Uhhh!  What do you mean by the "continually-changing state of the 
instrumentation"?  The instrumentation of TOM1 has not changed over the run 
that I have been processing.  It has been sitting in exactly the same 
place, ran one program for 9 months and is now running a second program to 
switch from a short to longer period search.  It has been running at the 
same TEC operating temperature through summer and winter.  The only event 
of consequence during this time was the finding of a bug in the 
software/hardware.  This had no effect on the data taken, only whether or 
not we could take data.  As I have said, one of the problems of working 
this way as people stop by and pick up mind seta about what is going on 
from year old problems.  If they do not follow every day, then they do not 
see where the problem has been fixed or found to not be a problem.

As for the value of the "preliminary" data coming out of the engineering 
run.  It has allowed one individual to go out and make detailed 
measurements on several stars.  True, these results are "infesting" the 
IBVS.   One would think that they would want such an infestation.

Another result of the distribution of the "preliminary" data is the 
development of several software codes by Patrick Wils, Rich Knowles, and 
Doug Welch among others.  This was the announced purpose of posting these 
lists.  It has been successful.

Tom:
 >No Mark IV data has been published.

Brian:
Wasn't there MkIV data in one of the IBVS flyers? Or was that the
contentious Figure 1 from the recent reject paper? In any case, I wasn't
referring so much to what little has been published. It seems folks are
mentioning quite regularly having looked at "tonight's" list of magnitudes for
individual stars, and saying "oh, here's a variable". But nearly as often
the variability turns out to be spurious due to some reduction snafu or
there was ice on the chip, or simply because it was cloudy, or whatever.
I'd tend to slide the whole night of such data off into the trash.

Yes, I was wrong here.  There is MkIV data in the one published IBVS 
flyer.  The paper says "Using data acquired by The Amateur Sky Survey 
(TASS), variability was clearly demonstrated (Fig. 1)."  I think this is 
perfectly appropriate use of the tass data.  I stand by the statement for 
being appropriate at this stage of tass.  It clearly demonstrated 
variability.  It enabled an observer to go out and with great confidence 
measure a star that had previously not been known to be variable and to 
measure it and to classify it.

Brian, I think you criticism above is completely unfair.  Yes, there are 
spurious detections in the data.  We have discussed what might cause 
them.  For example, passing a bad spot can tend to produce a dip that looks 
like an eclipse.  Once this has been discussed on the list, it should fool 
no one.  This is discussed thoroughly in one of Andrew Bennett's TNs.  The 
problem is not that there are spurious detections, it is rather that we do 
not yet have software that digs all the good candidates out of the 
list.  Once a human looks at a plot of the data it is pretty easy to decide 
that it is worth further study.  There are hundreds of thousands of plots 
to look at unless we have software that does this for us.  That is one of 
the purposes for distribution of this data.

The present software gives you lists of things worth looking at.  But it 
misses many good candidates.  If you set the limits lower, then you have 
too many graphs to look at.  Still, if you step through each plot, then one 
finds obvious variables that the software is not coming close to 
detecting.  So more work needs to be done.

It is true that you cannot mindlessly pass the raw data through a program 
and have 100% certain variable stars come out the end of the 
pipe.  Actually, you can set limits that do this now, but you do not get 
all the good variables.  But you do get a lot of new variables.  One also 
gets good data on old known variables.  The programs do not discriminate.

The test is Mr. Koppelman 's (and others who care to do this) batting 
average when using the tass data to decide on a star for measurement.  So 
far he is batting 1000.  I would say the tass data would be useful if it 
allowed a 500 batting average.   As a survey it is serving it's 
purpose.  You can argue that this is not a useful purpose or that one needs 
to collect more data or with greater sensitivity or with greater color 
accuracy.  That is another matter and I would particularly like to discuss 
these points.

Brian, you can very useful if you enter into this discussion and point us 
to the most effective use of the apparatus.

Enough for part 1.

Tom Droege