My tutorial (french) and Calibration & optimize dark question

[Philippe B.]

Hello

I am writing my own (french) tutorials on PreProcessing images under PI which is wonderfull for that.

A) Monochrome images

B) Multispectral images (LRGB, narrowband)


To continue the discussion http://pixinsight.com/forum/index.php?topic=2269.0, in my preprocessing, if I select OPTIMIZE (Master dark) in calibration, all the hot pixels seems to desappear. Then, the final image shows all hot pixels.
If I don't check it, the process is better and most of the hot pixels are corrected.

Do you have some idea ? Maybe I made something wrong ?   

Best regards
Philippe

[Philippe B.]

Hello

No answer, maybe explanation is not goog 

The question is about the "CALIBRATE" option in calibration menu and the dark OPTIMIZE

So :

when I follow the tutorial :
I integrate the MASTER BIAS by integration menu
I integrate the MASTER DARK by integration menu (without remove the Master Bias)
I integrate the MASTER DARK FLAT by integration menu (without remove the Master Bias)
I process the FLAT files by calibration menu (calibrate Master Bias and Master Dark flat), I check "calibrate" for all
I integrate the MASTER FLAT by integration menu
I process all Light image in calibration menu (calibrate Master Bias, Master Dark, Master flat) I check "calibrate" for all
I align and integrate final image

But...
When do I have to check or uncheck CALIBRATE option ?

The real thing should not be somthing like this :
integrate master bias
integrate master dark
integrate master flat (no bias and dark correction)
Then, calibrate images with all the master and check "CALIBRATE"

Or
integrate master bias
calibrate each dark with master bias and check calibrate
integrate master dark (also dark of the flat)
calibrate each flat with master bias and master dark (maybe ONLY master dark because master bias has already been removed) : check "calibrate"
integrate master flat
calibrate each light image with masters (bias, dark, flat) but no check calibrate (or check) ?

As you see, there is different way where we can make a mistake !!  So what is the real method when you developped this function JUAN ?

Also, the "Optimize" remove hot pixels.

Many thanks in advance.
For sure I don't use any more DSS, IRIS, or other software because I REALLY GET BETTER RESULT with PI preprocessing functions

[Juan Conejero]

Hi Philippe,

Sorry for overlooking your first post. Nice tutorials, thank you for contributing them!

Thank you also for your nice words; It's nice to know our calibration routines are working well for you. Our set of calibration tools require more user intervention and knowledge that other implementations, but we are confident that they are more flexible and accurate.

You can do it both ways:

- You can follow Vicent's tutorial and calibrate and integrate all bias, dark and flat frames to generate calibrated master bias, dark and flat frames. This is the most accurate option, as in this way, for example, you are calibrating each individual flat frame separately, before integrating all of them as a single master flat. In this case, you must disable the Calibrate options for the master frames in ImageCalibration when you calibrate your lights.

- Or you can just integrate all bias, dark and flat frames to generate uncalibrated master bias, dark and flat frames. In this case, you must enable the Calibrate options for the master frames in ImageCalibration.

The rationale is: Disable Calibrate for master calibration frames that have already been calibrated. Otherwise, you'll be calibrating them twice, with incoherent results. Enable Calibrate only when you are using uncalibrated master frames. In this case, both the master calibration frames and the science frames will be calibrated in the same execution of ImageCalibration, but with less accurate results.

The next version of the ImageCalibration tool will have the Calibrate options disabled by default, as the best and most frequently used option is working with calibrated master frames.

Hope this helps clarify.

[Philippe B.]

Thanks Juan
I think I understand now !!!

I will make other test and correct my tutorial (it is a long work, now it will be the processing part)

I will check if OPTIMISE is working by this way without removing the hot pixels !
If not, do you think it is a normal result ?

Many thanks
cheers

[Juan Conejero]

Your second question:

if I select OPTIMIZE (Master dark) in calibration, all the hot pixels seems to desappear. Then, the final image shows all hot pixels.
If I don't check it, the process is better and most of the hot pixels are corrected.

It is true that dark frame optimization can lead to undercorrection of some bright hot pixels. However, the goal of dark frame optimization is not removing hot pixels better, but to minimize the noise induced by dark frame subtraction in the calibrated images. So even if some hot pixels survive with dark frame optimization enabled, the result will be less noisy. Hot pixels should be easy to remove with Carlos' DefectMap tool or Nikolay's CosmeticCorrection script, but less noise is always preferable in my opinion.

Why dark frame optimization can degrade hot pixel removal? Mainly due to nonlinear sensor behavior, and this means that each CCD is a different world. Our current dark frame optimization routine applies a linear scaling function to the dark frame. In an incoming version of ImageCalibration, we have implemented a multipoint dark frame optimization algorithm (basically a spline-based optimization curve) that hopefully will provide much better results. Multipoint dark frame optimization will compute separate scaling factors for different sections of the dynamic range. In this way hot pixels will be optimized in a different way from the rest of dark frame pixels.

[Philippe B.]

Many thanks, Juan, it answers to all my questions 

[NKV]

In an incoming version of ImageCalibration, we have implemented a multipoint dark frame optimization algorithm (basically a spline-based optimization curve) that hopefully will provide much better results. Multipoint dark frame optimization will compute separate scaling factors for different sections of the dynamic range. In this way hot pixels will be optimized in a different way from the rest of dark frame pixels.

Juan, any news? 3 years passed...
I ask you because:
With enabled dark optimization, I see that Amp Glow not correctly removed during calibration and RBI pattern too.