Author Topic: BasicCCDParameters: flaw in calculation of dark current?  (Read 2432 times)

Offline alessio.beltrame

  • Newcomer
  • Posts: 2
BasicCCDParameters: flaw in calculation of dark current?
« on: 2017 November 09 00:52:41 »
I recently used BasicCCDParameters (v0.3.1) to evaluate my CMOS monochrome camera (QHY163M) and I found suspicious results. I took a look at the source code and I found a portion of the code that is not sound to me. Let's start by saying that the sensor on my camera does black point correction, so the single dark frame method (first part of the <if> instruction - Line 421) does not apply.

In the <else> clause (lines 431-438) there is the following code:

var heat1_noise = Math.sqrt( dark1StdDev[c] * dark1StdDev[c] - bias1StdDev[c] * bias1StdDev[c] );
var heat2_noise = Math.sqrt( dark2StdDev[c] * dark2StdDev[c] - bias1StdDev[c] * bias1StdDev[c] );
// "n" units of additional read noise increase sigma(heat noise) by sqrt(n),
// so too measure the darkCurrent per second:
var sbq = heat2_noise / heat1_noise;
darkCurrent[c] = gain[c] * ( sbq * sbq ) / ( self.dark2ExposureTime - self.dark1ExposureTime );

Let's pretend that heat1_noise and heat2_noise actually represent the dark noise only (this is true as a first approximation only). The variable sbq is calculated as the ratio of dark noise in the two frames, so it's not "n units of additional read noise", whatever that remark means.  Instead of sbq^2, the formula should use a value that corresponds to how much the dark signal (= Dark current * exposure time) has increased in dark2 with respect to dark1.

It is also easy to prove that, for negligible values of read noise, the formula for dark current only depends on gain and exposure time, which is clearly wrong.

Finally, let's do a dimensional analysis of the dark current equation. We expect to get a result in electrons per second, however sbq is a unitless number, so sbq^2 is also unitless; sbq^2 is multiplied by gain (e-/ADU) and divided by the dark2ExposureTime - dark1ExposureTime (seconds), so the result is measured in e-/(ADU*s) instead of e-/s.

That's why I'm currently using different methods for the measurement of dark current, which provide completely different results. BTW, I can't find the formula used for the dark current in the referenced HAIP book, let alone a formal demonstration. The other calculations of the script - FWC, gain, dynamic range, etc. - seem to be correct to me.

Can somebody please confirm that this portion of the script is actually flawed or explain to me where my line of reasoning is wrong?

Thanks in advance
Alessio

Offline sharkmelley

  • PTeam Member
  • PixInsight Addict
  • ***
  • Posts: 241
    • Mark Shelley Astrophotography
Re: BasicCCDParameters: flaw in calculation of dark current?
« Reply #1 on: 2017 November 09 01:13:40 »
It looks flawed to me.

I wrote my own:
https://pixinsight.com/forum/index.php?topic=8683.0
Pick up version 6 at the end of that thread.

Mark
Takahashi Epsilon 180ED
H-alpha modified Sony A7S
http://www.markshelley.co.uk/Astronomy/

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: BasicCCDParameters: flaw in calculation of dark current?
« Reply #2 on: 2017 November 09 01:43:52 »
Hi Alessio and Mark, and Georg,

The BasicCCDParameters script was released under GPL license. Can we include it in the official GitHub repository and update it as you are describing here?

Mark, can we release your script as well?
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline alessio.beltrame

  • Newcomer
  • Posts: 2
Re: BasicCCDParameters: flaw in calculation of dark current?
« Reply #3 on: 2017 November 09 05:50:11 »
Thanks Mark,
I looked at your script and it's based on the same method I use in my manual calculation (with the exception that I use multiple darks at once to provide a single result, while your script lists the result for each couple of frames). I tried to run the script and it works well, even if my CMOS camera is actually monochrome (it's not a DSLR): the output results for the 4 channels are almost identical, as they should be. Last notice: I use Sequence Generator Pro to capture images and the FITS header uses EXPOSURE as the keyword for exposure time. It might be a good idea to check to check for that keyword in addition to EXPTIME.

I also appreciated the fact that your script does check for errors and for pixel clipping.

Very good work

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: BasicCCDParameters: flaw in calculation of dark current?
« Reply #4 on: 2017 November 10 01:52:43 »
Hi Alessio and Mark, and Georg,

The BasicCCDParameters script was released under GPL license. Can we include it in the official GitHub repository and update it as you are describing here?

Mark, can we release your script as well?
I have no objection. I would also be fine with giving it a different license that allows anyone to use or modify it.
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline sharkmelley

  • PTeam Member
  • PixInsight Addict
  • ***
  • Posts: 241
    • Mark Shelley Astrophotography
Re: BasicCCDParameters: flaw in calculation of dark current?
« Reply #5 on: 2017 November 10 02:42:55 »
Quote from: georg.viehoever
Quote from: Juan Conejero
Hi Alessio and Mark, and Georg,

The BasicCCDParameters script was released under GPL license. Can we include it in the official GitHub repository and update it as you are describing here?

Mark, can we release your script as well?
I have no objection. I would also be fine with giving it a different license that allows anyone to use or modify it.
Georg

I'm also happy for mine to be released under a similar license that allows anyone to use or modify.

Mark
Takahashi Epsilon 180ED
H-alpha modified Sony A7S
http://www.markshelley.co.uk/Astronomy/

Offline niccoc1603

  • Newcomer
  • Posts: 42
Re: BasicCCDParameters: flaw in calculation of dark current?
« Reply #6 on: 2019 April 29 21:52:39 »
Hi is the script fixed now?