Location and Scale estimators

[ngc1535]

I have read the Image Integration document and understand the location estimate to basically be the peak of the histogram (the mean about which everything fluctuates) with a scale estimate that is related to the dispersion of the fluctuations- using the various methods available in PI. The ratio of scale estimates from a reference frame compared to an nth frame is how you get the weights.  Hopefully this is close.

So... my question is how to common maladies or techniques affect the calculation of the scale?

1. The big one is variation of sky brightness. Does the addition of shot noise due to sky photons (say from high thin clouds) affect the scale estimate in such a way that it appears *less* disperse than it otherwise should? (Do the sky photons make the distribution more peaked even though it isn't a signal we are interested in?) Could this be a reason for weighting variations (different than expectations)>?

2. Another one is the use of dark frame scaling. I do  not see how this would adversely affect the computation of the scale (even if the dispersion is greater in general..it would be greater for all images?

3. The above are total guesses... that I why I am asking! Are there any "other" (or any at all) sources of scale estimation effects that might be commonly encountered?

4. Does the Sn/Qn method minimize any of the above? (Is the only reason it isn't the default is because of computation time?)

I hope these make sense!
-adam

[mschuster]

Adam, I had an issue with high thin clouds and scaling. High thin clouds on some exposures increased sky background and also reduced sky transparency, both of which led to lower H II region SNR (ie, more noise and less signal). However the clouds diffracted light from a bright star into the surrounding area of the frame, which lead to a larger scale value (eg more contrast between the diffracted area and other darker areas of the frame) and as a result incorrect scaling. See the frames below, the first with high thin clouds and the star diffraction problem and the second without. Workaround? Don't scale. Another potential option (but I think not yet supported by PI) would be to use an ROI that eliminates the diffracted area for measurement purposes.
-Mike

High thin clouds causing bright star diffraction:


No high thin clouds:

[ngc1535]

Yes...I think I understand.
However, by not using the scale estimator for noise based weighting- I will not be able to weight the images (based on noise).
The images with poor seeing or high thin clouds which I would have wanted to use (but given less weight) I would either need to throw away entirely (awwww) or allow them equal weight (awwwww).

That is the conundrum I have in my head. It may be that this isn't a real issue and I need to think about it differently.

-adam

Adam, I had an issue with high thin clouds and scaling. High thin clouds on some exposures increased sky background and also reduced sky transparency, both of which led to lower H II region SNR (ie, more noise and less signal). However the clouds diffracted light from a bright star into the surrounding area of the frame, which lead to a larger scale value (eg more contrast between the diffracted area and other darker areas of the frame) and as a result incorrect scaling. See the frames below, the first with high thin clouds and the star diffraction problem and the second without. Workaround? Don't scale. Another potential option (but I think not yet supported by PI) would be to use an ROI that eliminates the diffracted area for measurement purposes.
-Mike

High thin clouds causing bright star diffraction:


No high thin clouds:

[Juan Conejero]

Another potential option (but I think not yet supported by PI) would be to use an ROI that eliminates the diffracted area for measurement purposes.

This is in the to-do list with moderate priority. I have also been thinking on new local estimators. Basically, the idea is to evaluate statistics on regions defined by a regular lattice and fit a surface approximation device (such as surface splines or B-splines), which is later used to interpolate location and scale pixel-by-pixel. Similar to local normalization but integrated in the ImageIntegration process.

[Ignacio]

FWIW, I have been using StarSupport as a proxy for frame quality. I find it interesting as it combines SNR, transparency, and FWHM aspects of the subexposure.

In the past I have encountered weighting issues due to thin clouds, optics dew, and changing gradients from imaging at different elevations.

Ignacio

[mschuster]

Ignacio, I agree, StarSupport can be helpful, but unfortunately I found that it does not work well on my setup. In excellent seeing, with excellent focus, FWHM is below 1 pixel on my binned, short focal length setup, and hence PI's star detector ends up rejecting many small stars as hot pixels. So in fact a smaller StarSupport often indicates a better frame. But not always, as it could also mean bad transparency. So I can't rely on it.

Mike

[Ignacio]

Yeah, if you undersample it won't work. My typical fwhm is around 1.5 pixels or higher, so don't have that problem.

Ignacio

[mschuster]

Good. Maybe a hot pixel count would be another quality proxy for my setup? (assuming the true hot pixel count doesn't change much across frames)

[Ignacio]

Good. Maybe a hot pixel count would be another quality proxy for my setup? (assuming the true hot pixel count doesn't change much across frames)

It might. In any case, I do the quality ranking/weighting with subframe selector after full calibration (including cosmetic correction), so hot pixels should be gone by then.

I am quite happy with this approach, giving very consistent results. What I am still debating is the weighting scale: linear? square root? etc. I tend to use linear when frames are quite even, and square root when there are large differences. But no technical argument behind this.

Ignacio

[ngc1535]

FWIW, I have been using StarSupport as a proxy for frame quality. I find it interesting as it combines SNR, transparency, and FWHM aspects of the subexposure.

In the past I have encountered weighting issues due to thin clouds, optics dew, and changing gradients from imaging at different elevations.

Ignacio

Ignacio,

I am not familiar with this- can you give a few more details on what you are doing? (What process?)
-adam

[Ignacio]

Ignacio,

I am not familiar with this- can you give a few more details on what you are doing? (What process?)
-adam
[/quote]

It is actually quite simple, Adam. I use the script SubframeSelector to add weights to the fits headers, and I choose StarSupport/max(StarSupport for the whole series). So you end up with weights <=1, which I then use in ImageIntegration. When there is much dispersion in quality, I take the square root of that fraction as weight.

Ignacio