Script for calculating DSLR sensor parameters

[sharkmelley]

I've written a script for calculating DSLR sensor parameters i.e. gain, read noise and dark current profile over a sequence of dark frames.  It will calculate sky fog noise as well if (featureless) light frames are supplied to it.

I hope I've managed to attach a zip file containing the script.  The script should be placed in your PixInsight\src\scripts folder.  The code is probably not brilliantly written (I'm no JavaScript expert) but it does appear to work.

The output is written to the Process Console. Here is a result from a Sony A7S camera at a constant ambient  temperature of 20C:

=============================
DSLR Sensor Parameters v0.0.4
=============================
Camera: Sony ILCE-7S
ISO speed: 2500
Image Size: 4256 x 2848
Bias frames exposure time: 1/8000 sec
Flat frames exposure time: 1/4000 sec
Dark frames exposure time: 297 sec
Light frames exposure time: 302 sec
Gain per channel(e/ADU):   [ 1.626, 1.607, 1.608, 1.615 ]
ISO for unit gain:         [ 4066, 4018, 4019, 4039 ]
Read Noise(e):             [ 1.33, 1.33, 1.34, 1.33 ]
Dark Current (e/pixel/sec) [ 0.1143, 0.1102, 0.1099, 0.1132 ]
Dark Current (e/pixel/sec) [ 0.1216, 0.1168, 0.1175, 0.1218 ]
Dark Current (e/pixel/sec) [ 0.1281, 0.1239, 0.1238, 0.128 ]
...<snip> ...
Dark Current (e/pixel/sec) [ 0.1675, 0.1606, 0.1618, 0.1666 ]
Dark Current (e/pixel/sec) [ 0.1684, 0.1617, 0.1628, 0.1674 ]
Dark Current (e/pixel/sec) [ 0.1711, 0.164, 0.1639, 0.1683 ]

Noise estimates for 297sec exposure
Read Noise(e):             [ 1.33, 1.33, 1.34, 1.33 ]
Thermal Noise(e):          [ 7.13, 6.98, 6.98, 7.07 ]
Skyfog Noise(e):           [ 46.59, 50.87, 50.88, 37.52 ]

Mean values in ADU
Bias Frame 1: [128, 127.9, 127.9, 128 ]
Bias Frame 2: [128, 127.9, 127.9, 128 ]
Flat Frame 1: [850.6, 881.7, 882.1, 532 ]
Flat Frame 2: [831, 860.6, 861, 520.6 ]
Dark Frame 1: [ 126.9, 126.7, 126.9, 126.6 ]
Dark Frame 32: [ 126.6, 126.6, 126.7, 126.5 ]
Light Frame 1: [1367, 1641.5, 1642.1, 903.6 ]
Light Frame 2: [1479.4, 1787.2, 1787.6, 958.4 ]

DSLR Sensor Parameters v0.0.4 Completed.


The results are given for each of the 4 channels:  R, G1, G2, B

It's fairly straightforward to use (I hope!)
Pairs of raw bias frames and (unsaturated) raw flat frames must be supplied to calculate gain and read noise (in electrons)
To calculate dark current, a sequence of dark frames must be supplied (at least 2)
For skyfog, 2 light frames must be supplied (with the same exposure time as the darks)
All frames must have the same ISO.

The only issue at present is that if dark frames and bias frames are black clipped to zero (which is typical of Nikon behaviour) then the results will not be reliable.  Warnings are produced if black clipping or flat frame saturation is encountered.

The calculations use only the central 25% of the image in both width and height in order to avoid vignetting problems or amp glow effects near the edges.

Any suggestions are welcome!

Regards,

Mark

[Alejandro Tombolini]

Hi Mark, thank you for the script. I must be doing something wrong, I get the following error:

Code: [Select]

Processing script file: /opt/PixInsight/src/scripts/DSLRSensorParameters.js
*** Error [222]: /opt/PixInsight/src/scripts/DSLRSensorParameters.js, line 1252: TypeError: globalData is undefined
Saludos, Alejandro

[sharkmelley]

Hi Alejandro,

Thanks for your feedback. No doubt the problem is my own bad program code but it works fine on my Windows 7 PC.

Maybe someone would be kind enough to tell me what is wrong with this code snippet:

Code: [Select]

MainDialog.prototype = new Dialog;

var globalData;
globalData.biasFiles = new Array;      //This is line 1252
globalData.flatFiles = new Array;
globalData.darkFiles = new Array;
globalData.lightFiles = new Array;
globalData.fileSelectDialog = new MainDialog ();

main();

I can then fix it tonight.

Regards,
Mark

[Andres.Pozo]

You have declared the variable globalData but you have not initialized it.

You should do something like this:

Code: [Select]

//....
var globalData = {}; // Initialize globalData to an empty object
globalData.biasFiles = new Array;
globalData.flatFiles = new Array;
globalData.darkFiles = new Array;
globalData.lightFiles = new Array;
globalData.fileSelectDialog = new MainDialog ();
//....

or even better:

Code: [Select]

//....
var globalData = {
   biasFiles: new Array;
   flatFiles: new Array;
   darkFiles: new Array;
   lightFiles: new Array;
   fileSelectDialog: new MainDialog ();
};
//....


[sharkmelley]

Thanks Andres!

I've made the fix, so hopefully it will work now.  It's difficult for me to test because it already worked with no problems under Windows 7.

The fixed version is attached.

Mark

[Alejandro Tombolini]

Thank you Mark, it works perfect now.

Saludos, Alejandro.

[sharkmelley]

I found a bug in the skyfog calculation - forgetting to subtract the bias from the light.  Now fixed in version 0.0.6 attached.

Mark

[niccoc1603]

Hi, thank you for the script, it works great

Would it be possible to add the possibility to select 12bit/14bit/16bit scale like in BasicCCDParameters

My CMOS camera is 12 bit and gain e/ADu here is returned in 16bit scale

Another question. Any ideas why Dark current values are so different for RGGB channels?

[sharkmelley]

Hi, thank you for the script, it works great

Would it be possible to add the possibility to select 12bit/14bit/16bit scale like in BasicCCDParameters

My CMOS camera is 12 bit and gain e/ADu here is returned in 16bit scale

Another question. Any ideas why Dark current values are so different for RGGB channels?

It's an interesting point about the 12/14/16 bit scale - I haven't previously seen the script used on a camera that is not a DSLR.  The digital values reported are the ones found when opening the raw file, so it could be that a dedicated astro-cam is working slightly differently since it is not using a DSLR raw format.  What file format is being used?

The variation in dark current is simply a question of statistics.  Those dark current levels are extremely low compared with an uncooled DSLR.  Slight statistical deviations in the calculated bias level or read noise level can lead to much greater variations in dark current.  You need to use much longer dark exposures (e.g. 400sec) to obtain more reliable results for dark current.

It's also odd that the Dark Frame 1 has a mean level of around 1065-1066 and Dark Frame 5 has a different mean level of around 1005-1006 whilst the bias frames have levels of 1081 -1086.  I don't have an explanation for that.  Anomalies like that can skew the statistics quite seriously.

Mark

[niccoc1603]

The file format is FITS (standard for astro cam) . It has a Keyword for Bits, but they are already scaled to 16bits



In my case the camera (QHY183C) has a 12bit ADC, so a backward conversion is needed to calculate gain in 12bit scale instead of 16bit scale (and the 0.06 e/ADU is correct in 16bit scale)

It's also odd that the Dark Frame 1 has a mean level of around 1065-1066 and Dark Frame 5 has a different mean level of around 1005-1006 whilst the bias frames have levels of 1081 -1086.  I don't have an explanation for that.  Anomalies like that can skew the statistics quite seriously.

There is an explanation for that https://www.qhyccd.com/bbs/index.php?topic=6159.0
It's too technical for me to really understand

[niccoc1603]

And this is from my D5300

[sharkmelley]

It's also odd that the Dark Frame 1 has a mean level of around 1065-1066 and Dark Frame 5 has a different mean level of around 1005-1006 whilst the bias frames have levels of 1081 -1086.  I don't have an explanation for that.  Anomalies like that can skew the statistics quite seriously.

There is an explanation for that https://www.qhyccd.com/bbs/index.php?topic=6159.0
It's too technical for me to really understand

Thanks for that - it's interesting. I'll need to take time to read and digest it.  It certainly looks as if it complicates the problem of calculating the sensor parameters for cameras affected in this way.

Mark