Author Topic: impact of "negative" dark in the BPP script?  (Read 678 times)

Offline whwang

  • Member
  • *
  • Posts: 62
impact of "negative" dark in the BPP script?
« on: 2020 January 08 21:00:19 »
Hi,

I want to know if this can be an issue in BPP.

Many DSLR has "optical dark" pixels along the borders of their sensors.  Those pixels do not receive light, but silently accumulate dark current during long exposures.  The cameras take the mean or median (or whatever) from those pixels and subtract it from the real image, as a very crude dark subtraction.  This does not remove hot pixels or any fixed pattern in the dark, but at least remove the low level haze across the entire image.

A big problem of this is that those pixels are closer to the heat-generating parts of the camera, so they are hotter than normal pixels.  This leads to over-subtraction of dark.  Because of this, some camera have dark images with tons of zeros.  In such a case, the data are just gone, and cannot be recovered.  Some other camera manufactures are smarter (and nicer to astrophotographers), and apply a constant offset to all image data before the data are written into unsigned integer files.  For example, a camera can apply a +1024 offset to the data.  In this case, the bias file may have values of 1024 +/- a few ADU.  A dark file from such a camera can have values of, for example, 990 +/- many ADU.  It's 990 but not 1024 because the camera applies the dark subtraction using the optical dark pixels in long exposures, and this produces an over-subtraction.

In the above example, if a master bias (about 1024 ADU) is subtracted from the master dark (about 990 ADU), the data will be on average -34.  However, there should also be a -34 over-subtraction in the lights, since the same dark subtraction based on optical dark pixels should also be applied to the lights.  So this -34 offset in the bias-subtracted dark and in the bias-subtracted light will cancel each other out after the dark subtraction for the light.

The question is, does the BPP internal calculation allow such negative values?  If yes, then there should be no problem at all and everything should be fine.  Does any one know?

(Because of the subtraction of optical dark pixels, dark scaling in BPP should not work well if my understand is correct.  But this is not what I am worrying about.)

Offline mschuster

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1087
Re: impact of "negative" dark in the BPP script?
« Reply #1 on: 2020 January 08 21:54:20 »
It is easy to answer this question by running a test.

Create three frames bias, dark, and light using pixelmath "1024/65535", "990/65535", and say "2048/65535". That is, the same pixel value everywhere in each of the frames.

Then setup a ImageCalibration instance with the master frames and calibrate options set as you wish. Try various combinations and look at the resulting value in the light_c frame.
« Last Edit: 2020 January 08 22:49:57 by mschuster »

Offline bulrichl

  • PixInsight Guru
  • ****
  • Posts: 524
Re: impact of "negative" dark in the BPP script?
« Reply #2 on: 2020 January 09 02:47:38 »
The way ImageCalibration (which is used in BPP) works is (cited from "Image Calibration and negative values (KAF 8300)", https://pixinsight.com/forum/index.php?topic=6822.0 ):

Quote
Quote
it seems that the ImageCalibration truncates the intermediate result 'dark - bias'. Is this right ?


Not actually. Truncation to the [0,1] range is carried out as the very last step of the calibration task for each frame, i.e. after overscan, bias, dark, flat and pedestal correction.

Therefore, it is not a good idea to pre-calibrate either dark frames or the integration of the dark frames. In order to control the calibration of the MasterDark with the MasterBias, the way to go is: use the options in the 'Master Dark' section. That means:

1) No dark frame optimization
No need to use a MasterBias for light frame calibration. This means: there is no issue at all because the MasterDark will not be calibrated with the MasterBias. Disable the 'Master Bias' section. Enable the 'Master Dark' section, select the MasterDark, disable both options 'Calibrate' and 'Optimize'. Enable the 'Master Flat' section, select the (already calibrated) MasterFlat and disable option 'Calibrate'.


2) With dark frame optimization
Enable the 'Master Bias' section, select the MasterBias and disable the option 'Calibrate'. Enable the 'Master Dark' section, select the MasterDark, enable both options 'Calibrate' and 'Optimize'. Enable the 'Master Flat' section, select the (already calibrated) MasterFlat and disable option 'Calibrate'. As the truncation is carried out as the very last step of the calibration task, negative values of the intermediate result (MasterDark - MasterBias) are used in the calculation. Performing the calibration of the light frames in this way, the result will be correct.


Also see my guide https://pixinsight.com/forum/index.php?topic=11968 , especially sections 4 and 5.

Bernd