Question on weighting during integration

[mschuster]

Hi Ignacio,

I looked at the data you posted. I understand the cause of your problem.

Integration weights are determined by an estimated "contrast to noise ratio".

Both the estimated contrast and estimated noise for frame #00 are larger than for frame #31, but the increase in contrast is larger than the increase in noise. This is because the gradient in #00 is larger than #31. As a result the weight of #00 is larger than the weight of #31 because the increase in contrast dominates in the ratio.

I believe PI estimates contrast by measuring the mean absolute deviation about the median. IMO this is a very good way to estimate contrast, but it is sensitive to differing gradients in the subframes being integrated. A subframe with a large gradient has large mean absolute deviation and so it is considered a relatively "good" sub. Obviously this may not be the best interpretation when gradients vary significantly across subframes.

What to do? (1) Just accept PI's interpretation and use the weights as is. (2) Consider doing an unweighted integration.

Another possible solution: Use the DBE tool to remove the gradient in each channel of each subframe (use calibrated but unregistered subframes to avoid the low-pass interpolation filtering). Measure the contrast and noise in the results to determine integration weights for the registered subframes.

Mike

[Ignacio]

Thanks a lot, Mike, for taking the time. So my first impression was "right", in the sense that signal is overestimated in the SNR evaluation, by misinterpreting gradients as part of the actual (useful) signal.

Clearly, flattening each frame will solve this...but is a lot of work!

Are there other ways of coming up with a signal "proxy" that is more immune to linear gradients?

best
Ignacio

[Carlos Milovic]

Hey Ignacio, you may try using ABE. You may tell it to use a first degree function, so only linear gradients will be removes (and they should the ones adding the more important contribution to mean dispersion). Applying ABE through an ImageContainer should be simple.

[Ignacio]

Thanks, Carlos. That's exactly what I had in mind.

In fact, I never use DBE, I always use a linear model in ABE (with good flats, of course), and sometimes a second ABE iteration with much tighter bounds and a higher order model.

Yet, I wonder if there is a better way to estimate the signal side of SNR evaluation.

Ignacio

[Ignacio]

Ok, so I flattened the field (with a linear model) of each subexposure, before registering, and that solve the problem. The weights behaved nicely during integration, and the master light image had about 4% less noise before processing. In the final image, the background looks smoother and less lumpy compared to my previous processing.

I will share the result in Gallery (Markarian's Chain).

Thanks to all who contributed.
best
Ignacio

[Carlos Milovic]

Good to hear that

[mschuster]

+1. Good to hear of an easy fix. Your gallery image is really nice.
Mike

[cs_pixinsight]

Now, it would be terrific to have this added as an optional processing step in the BatchPreprocessing script for anyone imaging from light polluted sites.  Having to stop the script after calibration to remove the gradients would require doing everything manually from that point forward.

Thoughts?

Craig

[Ignacio]

Thanks again, Mike.

Craig: I doubt this could be automatizes, as you need to play with the ABE parameters (Deviation and Unbalance) on a calibrated image to get it right. I don't use batchpreprocessing anyway.

best
Ignacio

[cs_pixinsight]

Ignacio, you didn't have to play with the settings for each calibrated image did you?  If not, then perhaps BatchPreproc could pause while we tune the settings and once we are done click continue?  Or maybe this can be done in the same fashion as CosmeticCorrecting curently is; we just set up an process icon with the settings we wish to use.

I'm probably dreaming and I'll have to go back to doing this all manually.

 

Craig

[Ignacio]

Hi Craig,

No, I didn't have to go thru the whole series, just picked the two extreme cases (minimum and maximum gradient), and adjusted the settings of ABE that would work on those. Thing is, that I am afraid you need to work on calibrated frames, but maybe not. In that case, you could feed the parameters to the batch process.

Ignacio