Problem with calibration

[ornotp]

Hi,
I'm quite new with PI and I wonder if I do the calibration correct because the frame looks almost the same before and after I calibrate it with dark and bias frames... Only flat frame makes a difference. I already watched a lot of PI tutorials but I couldn't get better result...
So I have the light, dark and bias frames plus master flat. First I integrate the bias frames. Then I calibrate the dark frames with the bias master frame. After that I integrate the calibrated dark frames. After that I have master dark, master bias and master flat.
I attach the settings I used for the calibration frames and the result frames itself.
The photo is a intensive crop and beside I made it for the first time with a barlow so it's far away from to be good but I mean the noise level here. I use Canon 1100D Mod and the ISO was 800.
Could anyone give me a hint? Or maybe it should look like this? The frames are autostretched so the noise is more visible but I though there should be any difference...
Best regards
Przemek

[bulrichl]

Hi Przemek,

don't calibrate the dark frames. I know this is recommended in some tutorials, but it's simply nonsense. Your settings for the integration of bias and dark frames are OK. So please integrate your NOT calibrated dark frames. The result is your MasterDark.

Depending on whether you want to try dark frame optimization (scaling of the darks), these are the correct settings for the calibration of the light frames:

1) With dark frame optimization:
Section 'Master Bias' enabled, MasterBias selected, option 'Calibrate' disabled

Section 'MasterDark' enabled, MasterDark selected, both options 'Calibrate' and 'Optimize' enabled

Section 'Master Flat' enabled, MasterFlat selected, option 'Calibrate' disabled


2) Without dark frame optimization:
Section 'Master Bias' disabled

Section 'MasterDark' enabled, MasterDark selected, both options 'Calibrate' and 'Optimize' disabled

Section 'Master Flat' enabled, MasterFlat selected, option 'Calibrate' disabled


Hope this helps.

Bernd

[ornotp]

Hi Bernd,

thanks for your answer. I did as you write but I still see no difference at all... On the right is the original frame and on the left the two ways of calibration you described. Of course first I simply integrated the dark frames and I used it for the calibration. I don't get it why I still become the same images?

Przemek

[bulrichl]

OK, one step back. I wonder why your MasterDark and MasterBias are color images.

When you open a CR2 file, is the displayed image b&w or colored? If colored, you must change your current settings in Format Explorer/RAW (open Format Explorer, double click on "RAW"). In this case, set to "Pure Raw" and confirm with OK.

In the preprocessing workflow, the integration of bias, flat-dark and dark frames, the calibration of the flat frames, the integration of the calibrated flat frames, the calibration of the light frames and (optionally) Cosmetic Correction shall be performed on Color Filter Array (CFA) data. Only after light frame calibration or Cosmetic Correction respectively, the data have to be debayered (= converted to RGB images), then aligned and finally integrated.

If RAW Format Preference was already set to "Pure Raw", your dark frames do not seem to match the light frames. Take a look at the light and dark frames: were they really captured with the same ISO setting, exposure time and temperature? Did you capture light frames and calibration frames in the same file format (here: CR2 files)?

Bernd

[ornotp]

I see I'm a hard case... I changed the option and integrated bias, dark and created new flat. The dark and bias frames were taken at the same session as the light frames. Now all pictures are black and white but the calibrated frame looks still the same for me. When I calibrate it with dark frame optimization then I receive this warning: No correlation between the master dark and master flat frames (channel 0). Maybe that's the problem?

Przemek

[bulrichl]

Please upload the following files to a filehoster and send me the link to that folder:

2 bias, 2 dark, 2 light, and 2 flat frames (all in CR2 format),
the MasterBias, MasterDark, and MasterFlat (all in XISF format) which were generated after changing the RAW Format Preference to "Pure Raw".

I will take a look then.

Bernd

[ornotp]

Here you have it: https://drive.google.com/drive/folders/1Sy4ZEi9heg-P5sHCb8NuRpaLHCC3x6Pt?usp=sharing
Thanks a lot
Przemek

[bulrichl]

Hi Przemek,

regarding the error message "No correlation between the master dark and master flat frames (channel 0).": According to my experience with DSLR and astro cameras utilizing a CMOS sensor it is not advantageous to use dark frame optimization when calibrating flat frames. Better is to use either a MasterBias or a MasterFlatDark (generated from flat-darks that were captured with the same ISO and exposure time as the flat frames).


The difference between no calibration and calibration can be seen in the comparison of the appended two screen sections that show a debayered uncalibrated light frame (left side) and the same light frame after calibration with MasterDark and MasterFlat and debayering (right side):

1) "comparison_1.JPG", zoom 1:1 (section of image):
The hot pixels are gone. This is due to the MasterDark.

2) "comparison_2.JPG", zoom 5:1 (whole image):
The vignetting is gone. This is due to the MasterFlat.

These differences might appear subtle, but when you integrate many light frames after correct calibration, debayering and alignment, the difference will be surprisingly large. You'll need some hours of total integration time in order to recognize this clearly.

When you perfomed the calibration as I recommended, 0.4267 % of the pixels (after all 52286 pixels) were clipped after calibration (these numbers apply to IMG_4605, see appended screen section "clipping.JPG").

There are two main reasons that can cause clipping:
- weak exposure of the light frames,
- much noise in the MasterDark.

You can increase the exposure time of the subframes. However, this will also have the effect of saturating the cores of bright stars. So this is always a trade-off.

The noise in the MasterDark can be lowered to a certain extent by capturing more dark frames. Your MasterDark is prepared from 59 dark frames. This number seems to be quite reasonable to me.

If you don't want to either increase the exposure time of the subframes or capture some more dark frames you should apply an output pedestal during the calibration of the light frames (see appended screen section: "output_pedestal.JPG"). Applying an output pedestal of 100 DN decreases the number of clipped pixels to < 2000 (< 0.016 %) in the case of IMG_4605.

My conclusion is: calibration does what it is supposed to do. Applying an output pedestal will slightly improve the calibration results and thereby also the integration result.

Bernd

[bulrichl]

... and annex_2

[ornotp]

Hi Bernd,

thanks for checking it So it seems that my calibration files are now correct, right? I have calibrated all light frames with settings from your screenshot and I got no warnings or errors there. But when I try to register them then I get this: *** 0 star pair matches found - need at least six matched stars. I ask already the last question - how to turn the frames to color again?

Przemek

[bulrichl]

thanks for checking it So it seems that my calibration files are now correct, right? I have calibrated all light frames with settings from your screenshot and I got no warnings or errors there. But when I try to register them then I get this: *** 0 star pair matches found - need at least six matched stars. I ask already the last question - how to turn the frames to color again?

Take a look at my reply #3 above:

"In the preprocessing workflow, the integration of bias, flat-dark and dark frames, the calibration of the flat frames, the integration of the calibrated flat frames, the calibration of the light frames and (optionally) Cosmetic Correction shall be performed on Color Filter Array (CFA) data. Only after light frame calibration or Cosmetic Correction respectively, the data have to be debayered (= converted to RGB images), then aligned and finally integrated."

CFA data cannot be registered. First you have to debayer the calibrated light frames, this will generate RGB images. THEN register the debayered images and finally integrate the registered images.

Bernd

[ornotp]

Hi Bernd,

sorry, I never did it this way and didn't understand what you mean... Thanks for the explanation. I'll try today to debayer the frames and then again to align them.

Przemek

[ornotp]

Hi Bernd,

thank you so much for your help, everything works perfect now  Now I will have to go through all my previous photos and process them again

Przemek

[bulrichl]

Super, I'm glad that it worked for you.

Bernd