Re: TASS data and what to do with it

[hjohnson@pluto.njcc.com (Herbert R Johnson)]

*>> At 02:53 PM 10/26/00 -0400, Aaron Price wrote (in part):
*>> 
*>> If so, I will be glad to write a program to convert your data
*>> to AAVSO format. I have alot fo experience programming database
*>> scripts for the AAVSO format requirements. :) Maybe that is 
*>> something I can contibute while I get my CCD skills up to par.

On Thu, 26 Oct 2000 15:15:20 -0700,
*>"Albertson, Chris" <CAlbertson@primeadvantage.com> wrote:
*>
*>Aaron and all,
*>
*>Tom's right, the reason data is not sent to the AAVSO is simply
*>because noone has sent it.  Let me add to Tom's comments.  
*>
*>IMO, the way to do this is to post a proposal to the TASS mail
*>list saying "Here is what I plan to do."  If you have a good
*>idea people will tell you, If your plan could be improved I'm
*>sure you will hear about that too.  It's good to post ideas
*>early and often, the good ones will get support the bad ones will
*>get pointed out to you.  
*>
*>Maybe more importantly, if your ideas about how the "data mining"
*>script should work represent the consensus of interested TASS list
*>members then you have a stronger claim to call what you submit to AAVSO
*>"TASS' observations".  Working only by yourself what you would send to AAVSO
*>could best be called "Aaron's observations of the TASS database".

*>The problem is more then simply reformatting existing data. 
*>You will need to calibrate the data.  This is a moderately hard
*>problem.  The full database is stored in a Postgresql DBMS.
*>(see www.postresql.org)  I wrote some of the software that puts data
*>into the database but Michael Richmond took that software, fixed it up
*>and built the current database. I can help explain the structure of the
*>database tables if need be and Michael (I assume) can give you an account
*>with which you can access the data.  

There seem to be a number of offers to get involved with TASS data
reduction lately, that seem to be in the form of "I can write some software
to convert data, I'd be glad to help". The presumption that seems to
be held is that what TASS has (or will have) from its cameras is some
kind of data stream that is in the "wrong format" for some later use,
so all that is needed is some "programming" to convert it, like you'd
convert a spreadsheet into an HTML document or some such. This is
simply not the case, and I'm glad to see a few TASS members call this out.

Even if the problem is represented as working
on a set of what TASS calls "star lists" - lists of objects derived
from TASS camera images; or "the TASS database" - Richmond's
reduction of several Mark III site's observations (and incidently
not the ONLY database); the issue is STILL
not one of "format conversion". It is an issue of data reduction by
explicit, defendable criteria; based on principles of optics and
astronomical observation. And, not incidently, some criteria for
a PARTICULAR reduction of observational data, like "stars of a
given magnitude range with certain observational features".

I think most TASS members are reluctant to comment upon these issues at any
length, simply because as a fact of human nature most people don't care
to disappoint anyone who offers to help, and no one cares to be seen as
a "spoiler". My attitude is a little different: I think that it is not
fair to underestimate a task at hand just to avoid potential hurt feelings.
Another way to consider this issue is that by not focusing
in on the problems and difficulties "up front", we lose an opportunity
to address them directly, and to get assistance to do so.

That said, the facts are that we've had YEARS of discussion on many
points about how we should reduce the image data or how the starlists
should be processed. The TASS Web page and other TASS member's WEb pages
refer or archive some of this discussion. And, the TASS Web page has
an archive of our maillist that is somewhat searchable.

But someone new to TASS, or scientific astronomical observation, may find it
difficult to get a handle on TASS, or what I'm referring to above.
And the various TASS Web sites simply have too much information on hand.
However, a few years ago, I wrote TASS Tech Note #57 on "Mark III
databases,... by site". It's on the TASS Web page in the Tech Notes section.

This somewhat
missnamed TN is a survey of what work WAS done with the Mark III cameras.
It provides many references to sites and Web documents for the entire
(I believe) Mark III process from camera sites, to software, to how
camera image data was reduced to star lists and the Mark III "tenxcat"
database - as well as referencing efforts by other TASS members who
ALSO produced results in other forums. What *I* wrote in that TN is not
important, in fact the note is primarily quotes from TASS members and
Web pointers, all of which was reviewed by the people referenced.

In a similar way, Michael Richmond's TN #56 on the Tenxcat database
provides a description of the reduction criteria HE used to combine
observations from three Mark III sites. Other Tech NOtes are of
interest for other parts and parallel data reductions of TASS efforts,
but these two TN's will offer some context to people new to TASS.

My point in repeately mentioning my Tech Note #57, is that it provides a
capsule reference for what TASS members have done with the Mark III
camera and its data. I've seen nothing that suggests that SIMILAR work
will not be done with Mark IV data: consequently, anyone interested in
Mark IV data should (in my opinion) review this note. Also, so far all
the potential Mark IV camera owners include almost all the Mark III
camera owners; and so far the proposed Mark IV image reduction software
methods (there is not one standard method) are using variations of
the same programs used in Mark III. In Ohio, we say "sauce for the
goose is sauce for the gander".

My own approach to projects and to working with others, is to pay
less attention to what is SAID, and more attention to what is DONE.
Less to apparent policies, and more to WHO has WHAT STUFF, and what
one needs to do to get access to that stuff. At the end of the day,
several people will have Mark IV cameras. A number of people will
offer software to reduce that image data to starlists. A few people
will  collect that data into larger collections of data and
reduce it. And a number of people will pull data from ANY of those
sources for their own analysis. As I see it, the questions for a potential
TASS member are: what part of this is of interest to me? What do I need
to know to DO that bit of work? Who do I need to contact to get
directly informed and active?

Of course, this is just my view, just like other TASS members have
expressed their views as Chris has above. But to disagree with Chris,
I think some potentially good ideas have not recieved much "consensus"
or "support" from TASS members; and some good work HAS been done by
individuals
with TASS data. I see nothing wrong with "Aaron's reduction of
TASS observations", the issue is not WHO, but the results - do they
stand up to review? are they sound? can Aaron defend his scientific
assumptions? and so on. FOr ongoing efforts like regular data reduction,
a consideration is simply: can you do the work regularly and consistently?
(Not that anyone in TASS is going to "enforce" these "rules", no one
is in charge - except as a practical manner whoever you are working with.)

As for AAVSO, if a TASS member has reported
work with them, they may have posted something in the TASS maillist;
check the maillist archives on the TASS Web page; or perhaps a TASS
member will contact you accordingly.

Herb Johnson
                        
Herbert R. Johnson              http://pluto.njcc.com/~hjohnson
hjohnson@pluto.njcc.com         voice 609-771-1503, New Jersey USA
             amateur astronomer and astro-tour guide
           S-100 computer parts, manuals as "Dr. S-100"
    reseller of 68K Macs & accessories for your computing pleasure