Understanding raw settings impact on image processing

m_abukhalid

Well-known member
Jan 20, 2018
63
2
Hello everyone,

I updated my raw settings recently based on an online recommendation to check the highlighted options below
raw_settings_pix.png

When reviewing some flats I was scratching my head as to a big disconnect between the values reported by pixsinight's image statistics process and what I was seeing in SGP Pro. It then dawned on me that the change in my raw file settings might be the culprit. Sure enough, using a 4 second flat as an example If I open the file with the settings active I get the values on the left (grossly underexposed flat), while if I disable settings prior to opening the file I get the values on the right (overexposed).
raw_settings_pix_result.png

I tried this for a 2 second exposure which gives me a max of 10k ADU and appears to be the limit where the response is linear from my camera sensor but with the appropriate raw settings on it reads as 1800 max ADU.

I initially thought those options would just ignore pixels that are either 0 or 1 but that is what the Unclipped option does in image statistics and it does not have the same effect. Can someone help me make sense out of this? Trying to make sure I have the right raw settings and the right exposure lengths as I have little success flat calibrating images properly.
 

bulrichl

PTeam Member
Nov 2, 2016
859
65
La Palma, Canary Islands
Both results are wrong for different reasons:

When the marked options are NOT enabled (i.e. your results on the right side), you will get wrong values because corrections of unknown degree have been applied in the low and in the high range and the result is rescaled. In this case, the data are unusable since they are not raw anymore.

The values on the left side are wrong as well because you set the wrong data range in Statistics. When you open a file in a proprietary raw format, 16-bit [0,65535] has to be set, and then the shown values have to be compared against the maximum possible value. In case of a ADC with 14 bits, the maximum possible value is 2^14 - 1 = 16383. If you set 16-bit [0,65535], the median will be 4 * 2340.9 = 9363.6 (= about 57 % of 16383), thus the flat frame is not underexposed at all.

Bernd
 

m_abukhalid

Well-known member
Jan 20, 2018
63
2
Thanks Bernd. What you are saying makes sense. If the raw file is unedited then even if it is now 16bit the values should remain 14 bit. Does that mean that the values portrayed in SGP Pro (and used by the app in the flat calibration wizard) are not a reliable means of gauging true ADU values. Heres what I see in SGP vs 16bit with the correct raw settings in pixinsight:
sgpvspix.png

To the meat of the question. What I'm hearing from you is those options SHOULD be set. Do the raw settings only apply when I manually open raw files in pixinsight or do they apply to pixinsight processes when calibration/integration scripts are reading file lists? Ie. if I have the wrong raw settings could they interfere with the batch processes?

Thanks again for the help.
 

bulrichl

PTeam Member
Nov 2, 2016
859
65
La Palma, Canary Islands
I have checked the statistics that SGP and PixInsight calculate on one and the same Canon CR2 file. These are the results obtained from a light frame:

SGP_20171218_0005.JPGPI_20171218_0005.JPG

SGP scales the intensities in the CR2 file according to:

I' = I * 4 - 1

If you set SGP to save FITS files, scaled intensities are also stored in the FITS files created by SGP. So the values indicated by SGP are reliable, but they are different from the values indicated by PixInsight. So when judging a flat file in CR2 format in SGP, you will aim for a mean intensity value of about 32,000 ADU whereas when checking the mean intensity of the same file in PixInsight, the goal is about 8,000 ADU. I assume that this scaling is applied by SGP for all proprietary raw formats, but I don't know that for sure.

Which proprietary raw file format are you using?

Are you sure that the same file was compared (SGP / PI)?


The settings in RAW Format Preferences apply generally. However, you can override particular settings by using input hints in processes that will open files in a proprietary raw format. Please open the Format Explorer/RAW and double-click on 'implementation'. The supported format hints are listed and described.

Bernd
 
  • Like
Reactions: m_abukhalid

pfile

PTeam Member
Nov 23, 2009
5,710
117
also note that in 1.8.8-6 at least, if you reset the ImageCalibration or ImageIntegration processes the following input hints are populated by default:

fits-keywords normalize raw cfa signed-is-physical

so for a DSLR file, it would be opened as a raw cfa file no matter what the format explorer is set to, unless you delete those keywords.

rob
 
  • Like
Reactions: m_abukhalid

m_abukhalid

Well-known member
Jan 20, 2018
63
2
Thanks for the help! Glad to hear the settings dont interfere with the calibration process.

Which proprietary raw file format are you using?

Are you sure that the same file was compared (SGP / PI)?
Bernd
I am using a Nikon D5500 and yes I am made sure to compare the same files... I am having issues with SGP Pro though when it comes to flats which I posted to the SGP Pro forum and hopefully someone will respond. The calibration wizard fails depending on the target. I manually swept through the ADU values from 0.25sec exposure to the point where I hit the saturation point of 64k for max ADU and noticed that beyond 0.75sec (11k mean ADU in SGP Pro) the response of ADU mean/max to exposure time turns into a staircase so I have been limiting my flat exposures to 11k ADU. However last night as an experiment I tried taking 51 flats at 1sec and 2 sec exposure times and the flat correction of the 2 second flats (19k mean ADU in SGP Pro) was better so maybe I should aim for 30k afterall. I was trying to limit my exposure times to where the response appeared linear.
 

bulrichl

PTeam Member
Nov 2, 2016
859
65
La Palma, Canary Islands
I manually swept through the ADU values from 0.25sec exposure to the point where I hit the saturation point of 64k for max ADU and noticed that beyond 0.75sec (11k mean ADU in SGP Pro) the response of ADU mean/max to exposure time turns into a staircase so I have been limiting my flat exposures to 11k ADU.
Yes, it sounds as if you limited the exposure time too much.

It is not reasonable to evaluate the maximum intensity value for the determination of the correct exposure time for the flat frames. Very likely the maximum intensity value is reached only for few hot pixels - this is insignificant.

Bernd
 
  • Like
Reactions: m_abukhalid

Tim Ellison

Member
Sep 3, 2020
6
0
Interesting to read your conversation. I've been thinking about similar things (re post "Puzzling Intensity Values in Raw Images" from 5th September - I'm not sure how to link posts, or whether that is a bad thing to do - apologies for my lack of knowledge!). You've all added to my understanding. Thanks!
Tim