Question on weighting during integration

[Ignacio]

I have been doing quite a bit of comet imaging lately, in short sessions at daybreak or dusk, when there is a strong background level variation and coloration.

I have noticed that SNR weighting does not behave as I would expect, assigning higher weights to subs with higher background levels (probably cannot differentiate between object signal and background level).

What would be the best way to set weighting during image integration in these cases?

thanks,
Ignacio

[Carlos Milovic]

Hi Ignacio

I usually disable weighting, or use the noise evaluation.

[Ignacio]

Thanks, Carlos. By noise evaluation, you mean an estimate of noise using the script, and then entered by hand in the FITS header?

Ignacio

[Carlos Milovic]

No, just using II's noise evaluation.
Then I compare the results of the integration on a small preview (with no weights at all).
Usually, there is not a noticeable difference between both methods.

[Ignacio]

Thanks, Carlos. I am coming back to this issue, since I want to figure out a way to adequately weight subs during intergration, when there are strong background level variations. In short, I would like to ponder darker background takes higher than others, since not weighting at all is clearly suboptimal.

Any suggestions? In particular, using frame adaptation during registration, would it help?

Ignacio

[Carlos Milovic]

In my opinion (so, there is a great chance that I am wrong), in this case the rejection algorithm is much more important than the weights, specially if you have large sets. You may try to force a better SNR estimation by subtracting the background sky, and see what happens.

[Ignacio]

Carlos, I don't have a problem with outlier rejection, and I don't think trending background levels can be treated as such (with the exception, of course, of sudden changes in few subs due to passing thin clouds, for instance).

The concept of subtracting a pedestal so that the background level of the given sub matches the reference sub, is what I would expect for a normalization method during integration (prior to SNR weighting). The "zero offset" normalization in the outlier rejection section, seems to do that exactly. But there seems to be no equivalent in the integration section. And as I said, the SNR normalization doesn't seem to work in these cases, as subs with higher background levels get weighted higher, and by amounts as much as 30% or more.

One could do this manually, subtracting a pedestal to each sub so as to match the reference (best) sub (will try it). But given that this is a very typical situation in astrophotography, I would expect to see such feature built into the normalization options.

best
Ignacio

[mschuster]

Ignacio, integration has both scale and scale+offset options. For integration weighting I am using other techniques, but I don't know if they would work well on your data. Here is what i am doing. My subs have almost no gradients, so I just need to measure noise in what is basically a constant background. One technique uses device model parameter estimates (device gain, device read noise and device dark current). Measure median background on the calibrated sub and apply device parameters to get background noise (read noise plus dark current noise plus shot noise, spatial noise (ie fixed pattern noise) is not a factor because it was removed by the calibration). The other technique tries to measure background noise directly from the sub, the challenge is to robustly ignore stars and unresolved stars and galaxies and what not. In both cases, the noise gets scaled by the inverse of the integration scaling factor (mean absolute deviation from the median), inverted and squared to form a weight.
Mike

[Ignacio]

Thanks for your reply, Mike. Your approach sounds interesting as a way to weight based on SNR estimation, although it does not address this issue.

Integration weighting does not include an offset normalization that I am aware of. It includes SNR estimation and other ways to estimate exposure. Pixel rejection does include offset normalization, but that is only to discard outliers. This is my understanding based on the documentation, and on how the algorithm works with this particular data set.

Maybe Juan can clarify this.

Ignacio

[Juan Conejero]

If I understand you well, the output normalization options should do just what you are asking for. It's the Normalization parameter in the Image Integration section. The tooltip for this parameter explains what it does, and you have more details in the documentation for the ImageIntegration tool, (jump to Image Normalization > Output Normalization):

http://pixinsight.com/doc/tools/ImageIntegration/ImageIntegration.html

[Ignacio]

Thank you, Juan, I was just reading that, and noticed that normalization "additive + scaling" in integration does the exact same transformation as "offset and scaling" in pixel rejection. So, you are right, that is what I was looking for.

Then, the question remains why am I getting high weights for subs with high background levels, when I would expect the opposite. Had the same issue doing other low elevation targets before (comets at dusk/sunset).

Any clues?

Ignacio

[Juan Conejero]

Hi Ignacio,

As far as I know the noise evaluation routines implemented in our tools (ImageCalibration, Debayer and ImageIntegration) are very reliable. If they assign a higher weight to one image that's because its noise estimates are smaller. However, it's always hard to say anything useful without having a look at the data...

[Ignacio]

Is there a way I could upload a couple of calibrated, registered frames (the reference frame and an ill-weighted one) so that you can take a look?

Thanks,
Ignacio

[Ignacio]

And one more thing, looking at the output in the process console during imageintegration, the noise estimates displayed there are inconsistent with the weights. That is, higher noise frames are weighted higher. My initial suspicion was that somehow high background levels were "confused" with signal levels in estimating SNR, hence the weight discrepancy. Does this make any sense?

Ignacio

[Ignacio]

Juan: I have uploaded to the server two frames: #31, which is the one I use as reference (taken in the middle of the session with the object higher up), and #00, from the beginning of the session with the object low and a high background level. They are in directory /ignacio/weight_problem_sample_frames.

Both were treated exactly the same way (calibrated, debayered, and registered). When I stack them using additive normalization and noise evaluation weighting, using frame #31 as reference, I unexpectedly get a weight of 1.2 on the red channel on frame #00. Other frames with high background levels show the same behavior.

If you have a chance, please take a look.
Thanks,
Ignacio