Re: bad columns not masked properly?

[Tom Droege <tdroege2@earthlink.net>]

OK, now I understand.  The analysis found 62 bad pixels.  The 1948,1953 
450,470 is the smallest box that contains all the bad pixels.  Only the 
program was not very smart about what a bad pixel is.  So we have to go in 
by hand and fix some problems.  The document probably described this 
correctly.  It is just a question of having the truth beat into one's skull.

I would say don't work on the manual, you have written a fine 
manual.  There is enough there, I think, that anyone that has the ability 
to work on the code can work on it.  I figure until I have studied the 
details of the code enough I should not be modifying it.

Much more important, I think, is to work on the code to make it better.  I 
stand ready to process stuff through the pipeline given tasks to do.  I 
will try to do some things on my own, and then I will ask dumb 
questions.  A few answers like below will be enough to keep me working.

Now if we had 20 people actually working on analysis, then possibly more 
write up would be productive.   As it is, I think Michael, that your time 
is better spent trying things to improve the code.

Some things I will try:

1) Run TOM2 and see if it has the same problems.  The I data could be the 
result of a bad lens alignment or some such.
2) Various cloud rejection schemes.
3) Run TOM2 to tie equator standards to the TOM1 region.
4) Study the bad pixels and figure out how to get the best results from TOM1.
5) There are plenty of funny things in the data.  One has to look at the 
raw fits files and tie the funny things into what was happening in the 
image.  There are lots of things to investigate here.  WS finds things 
where V and I are varying together that are obviously not star 
variations.  One just has to study data to understand what is going on.

There is lots to study.  DS20 has enough data that one with a mind to do it 
can study and work out problems.  As I keep saying, I stand ready to 
provide data sets to anyone that can not get enough out of DS20.   Note 
that you can't really do much with DS23 but detect variables.  Problems 
need the raw data, I think.

Since this is my retirement project, I have no particular goal other than 
to have fun.  Having fun for me means making the data better and better and 
building an archive.  I don't have to publish as I would if I were trying 
to make my way in academic circles.  Still, publishing the data is part of 
being useful, and an engineer wants to be useful.

The last two weeks have been almost a total loss.  I have had the flu bug 
that it going around.  Four absolutely miserable days followed by 10 days 
of slow improvement.  But very slow.  I am still weak and my brain is 
barely working.

Tom Droege

At 05:27 PM 7/6/02 -0500, you wrote:
>Michael and all,
>
>Well, I did read the documentation.  I did try to figure out how to patch 
>out the bad column.  It is pretty confusing to someone who has not written 
>the code.  ;^)   Still, there is documentation, and I could have got it 
>right had I really studied it.   The problem with something like this that 
>there are endless pits to fall into.  One really has to read all the code 
>and understand what is going on.  That is a big job.
>
>OK, below I notice you did not do anything to the 62 - the total number of 
>pixels patched out.  Does the program care if this is not right?  I assume not.
>
>Also, I give up:
>
>|So the second line in the mask file describes a bad region
>|which covers columns 1948 to 1953, and rows 450 to 470.
>
>How can you get 62 bad pixels out of this?    Note that 62 is only 
>divisible by 31.  So the bad area can only be 1 x 62 or 2 x 31.  Some 
>creative geometry is required, I think!  ;^)
>
>Well I had just run some data through attempting to patch out the bad 
>data.  But I did it wrong so will have to try again.  I think that this 
>bad column will not be the cause of the crummy I data.  But I will try it 
>to see if fixing the column does anything.
>
>Tom Droege
>
>At 02:47 PM 7/6/02 -0400, you wrote:
>
>> > 2)  The mask does not realize that the defect extends down the whole 
>> column.
>>
>>   Yup.  The program which created the mask didn't get it right.
>>In a case like this, the user should edit the mask file manually,
>>so that its second line looks something like this:
>>
>> > 1899    62      0      2028     1948    1953
>>
>>   _Now_ the mask will cover the entire set of columns (assuming that
>>the trimmed images have 2029 rows).
>>
>>   The pipeline isn't going to get this part right automatically.
>>The user should check images manually every now and then, and modify
>>the mask files as needed.
>