New script for denoising linear monochannel images

[mschuster]

Thank you Nick. Two things:

1) The DBE image is for some unknown reason overstretched. Here is an example of how to remove the overstretch: Open DBE image, open STF process, make sure the track view 'check' icon is enabled and the DBE image is the active window, click on STF's wrench icon, you will see on the top row the numbers 0.005630, 0.002633, 1.0. Change the first two to 0.0014, 0.015, leave the last 1.0. These changes will lighten the image and reduce the stretch. This is only a quick example, you can tweak the numbers and/or move the STF sliders as you wish, of course.

2) The denoising is too aggressive. I am guessing that this is because ImageIntegration used widely varying frame normalization parameters, and the MureDenoise is not able to track these differences sufficiently well. Please try setting the script's Variance Scale parameter to say 0.7 and denoising once again.

Basically, if you zoom way into red_MURE, with the appropriate STF, you will see a slight checkerboard pattern in adjacent pixels. Checkboarding indicates over denoising. Note, problems like this can happen if the script's detector parameters are set incorrectly (ie, Gain and Gaussian noise). If you have not already done so you can run the script MureDenoiseDetectorSettings as a double check for these values.

Mike

[nm303]

Thanks Mike. I will give it a go again tonight.

It seems odd the screen stretch over stretches. I've never even noticed teh STF wrench icon so something new to play with :-)
Yes I noticed the checkerboard pattern too but wasn't sure if that was just the lesser of two evils (noise being the other evil).
I have used the Mure detector settings script and the numbers seem to be moreorless identical whichever flats and baises I use.

BTW I'm not really clear about what the Flat selection in the MureScript brings to the party.

Many thanks. I'll let you know how I get on. The nice thing is even with my incorrect settings the result is good. :-)

[nm303]

Thanks Mike,

I can't actually find the  "track view 'check' icon". What does it look like? I've tried hovering over everything in the process but no tool-tip seems to say "track view". That said, the image does change when I slide stuff, type in the numbers in the wrench etc so I guess it is on.

The changing the variance seemed to improve matters. Then PI froze (I've had an issue like a lot of people, something to do with the graphics card and Windows) - I digress. More playing around needed for me :-)

Thanks for all your help.

[Geoff]

It looks like a tick (UK) or a check mark (US). The sort of thing a teacher does when you get the right answer.

[nm303]

Ah yes. Found it. Many thanks :-)

[nm303]

A quick question about bias frames chosen for the mure denoisedetectorsettings, which may account for slightly weird results.

I use a "super master bias" for my calibrations (which I do in WBPP). The bias frames I've been loading into the detector settings script are uncalibrated ones and obviously not master bias or super master bias.

I've tried loading both the master bias and the super master bias  into the detector settings alongside my raw untouched bias and (as I might expect even with my limited understanding of these things) massively changes the generated values.

So my question is, what is the procedure/considerations for using master super bias calibrated files?

Many thanks

[mschuster]

Yes, the detector setting script needs raw, untouched frames, both flats and biases (or darks if you use darks rather than biases). If you instead use masters in this script, you will likely get incorrect values that if used for denoising will likely compromise denoising accuracy.

You are free to use masters and/or super masters for calibration purposes without restriction. Caibration always adds the residual noise in the masters to the result, so less noisy masters are always better.

[mschuster]

The changing the variance seemed to improve matters.

Ok, good.

Looking at the metadata recorded in red.xisf, it appears some frames in the integration were significantly modified by ImageIntegration's normalization process. For example, the 18th frame was scaled by about 2.2 and offset by about 0.4 relative the the reference frame. This sort of result indicates that ImageIntegration believes that the frame was exposed less than half as long as the reference frame, and also had a significantly different median sky background level. The script tends to have major difficulty tracking the impact on noise statistics of these large normalization differences, and hence the need to tweak the variance scale parameter.

This is a problem with the script. Ideally it would handle these types of normalization differences well without intervention.

[nm303]

Hi Mike,

Thanks for the clarification on the use of raw bias and flats. The script does seem very clear in its request for these but thought it worth double-checking.

I guess I can just run the script a few times with different variance scale settings and see which looks best. Will the script work appropriately with previews and/or massive crops, for such testing purposes?

[mschuster]

Yes, previews and crops work OK. Except if too small, when you will see an error message.

[nm303]