Hi,
Again, trying to keep myself up-to-date with this extremely interesting thread. Don't be surprised, though, if my answers form an unordered set, as I'm collecting them throughout the thread seen "as a whole". Here we go.
(and, if you are listening Juan, I would like to be able to disable the stacked image as well for these iterative tests)
Good point. I'll implement an additional check box in the next version.
What I still need to have clarified (by Juan) is what the ADU value at a given pixel location, in each of the 'clip images' actually represents.
Is it a purely 'boolean' value - set to '1' when a pixel needed to be 'clipped' in at least one of the source images? Or does it behave as a 'counter', incrementing for every image that requires that pixel to be clipped?
A pixel in a rejection map image has a real value directly proportional to the number of rejected (=clipped) source pixels in the integrated set at the corresponding map coordinates.
Rejection maps, as you probably have guessed at this point, are normalized to the [0,1] range. If a rejection map pixel is 1.0, that means that all source pixels have been rejected at the map pixel coordinates. If a rejection map pixel is 0.0, then no source pixel has been rejected at its coordinates. Put as a simple equation:
r = N
r / N
where r is a rejection map pixel, N
r is the number of rejected pixels at r coordinates, and N is the number of integrated images.
Note that a rejection map can be used as a mask. For example, you can apply a low-pass filtering process (e.g., with wavelets) to the integrated image masked with the high rejection map (or with the maximum of the low and high rejection maps) to compensate for the lack of signal resulting from pixel rejection during integration. This is an effective noise reduction technique where you can take advantage of the —extremely unusual— fact that you
know accurately where the noise is, and its relative amplitude.
I would still have liked the ability to set the 'target percentage' and then see PI iterate the process as needed - returning the Sigma value needed to achieve that level of requested clipping.
Correct me if I haven't understood what you want to do, but I detect some statistical misconception here. The fact is that you really
don't want to know what the sigma value has to be in order to reject a given fraction of pixels in the integrated stack. You want to find an optimal sigma value that achieves a good compromise between efficient rejection and minimal SNR degradation. Usually the process to find the correct sigma is iterative: you implement a sort of
binary search where the correct value is within the limits of a narrower boundary at each iteration.
If you want to know
that sigma value in advance because you already know how many pixels you want to get rejected, then you don't need sigma clipping at all, but something without any statistical basis, such as min/max clipping.
In cases where you don't have enough images to substantiate a sigma clipping procedure (say less than 8 or 10 images), you can use either percentile clipping or averaged sigma clipping. Both algorithms utilize robust statistics and provide excellent results with reduced data sets.
ImageIntegration implements my idea of looking for a minimum noise in the resulting image. Noise is evaluated through an algorithm designed by Jean-Luc Starck, where the basic idea is to measure noise where there are no significant wavelet coefficients.
It cannot be overemphasized that the original —and extremely brilliant— idea of noise estimation/minimization is due to Vicent Peris.
For those who want to know the original sources, the iterative wavelet-based noise evaluation algorithm that I have implemented has been described here:
Jean-Luc Starck and Fionn Murtagh,
Automatic Noise Estimation from the Multiresolution Support. Publications of the Royal Astronomical Society of the Pacific, vol. 110, February 1998, pp. 193-199.
Okay, let's return to work. I'll try to answer more questions later on this thread.
