Saturated areas

[andyc67]

While processing images from the recent eclipse I noticed something interesting:
The frame in question is a single 1/2s frame at ISO 200 f7.1 of the corona. There is a large area that is saturated as expected.
What I don't quite understand are the following stats of a sample of the saturated areas.

CR2 file no calibration:

count (%)   100.00000
count (px)  107500
mean        65535.0
median      65535.0
avgDev      0.0
MAD         0.0
minimum     65535.0
maximum     65535.0
No surprise there

CR2 file flat corrected only:

count (%)   100.00000
count (px)  107500
mean        56112.135
median      57724.291
avgDev      5694.467
MAD         7432.285
minimum     42047.808
maximum     65535.000



CR2 file flat corrected and bias subtracted:

count (%)   100.00000
count (px)  107500
mean        52290.342
median      54383.518
avgDev      6557.315
MAD         6051.540
minimum     36215.576
maximum     65377.991

So it looks like some of the pixels that have no information because they are saturated, after calibration have an actual value that is below saturation. That is not real information of course.
Can this potentially throw off HDR composition?

[pfile]

have been talking with Ram about this, i don't think so, but i don't know for sure. the source code for HDRComposition shows a rescaling of each image so saturated areas should end up as 1.0 internally to the HDRComposition process.

the cause of this is of course that almost all DSLRs have 14-bit sensors and PI reads this data in such that saturated pixels have the value 0.25. so in fact i wonder if your original CR2 frames were opened as "pure raw" in PI, because they all show saturation values near 65535, and i'd expect closer to 8191.

i do however think there is something funny going on with HDRComposition. first of all things don't seem to work right at all on the 3-plane images. and even when i run with HDComposition on each channel separately, i always end up with one bad "band" which is the result of the merge between the first and 2nd images. i need to upload some of this data for Juan to take a look at because something does seem off.

rob

[andyc67]

Yes Ram told me and that's what prompted me to start this thread. I have PI set to open the files as pure RAW but I think that PI converts those to 16 bit greyscale.
Also the readout tool is no longer working on the current version of PI. At least it isn't on my iMac with the mac mouse so I can not change the readout mode. I am using the statistics process. I think it defaults to 16 bit readout, I will have to check and se if I can change it to 14 bit.
I actually abandoned HRD composition for this task. I am calibrating the files with PI then using photoshop for the blending of the various exposures. The issue is that the areas that should be saturated are no longer saturated after I run them through PI.

[pfile]

it converts them to 16-bit greyscale but it does not scale the data, so the top two bits are always 0... a fully saturated image will then have values of 16383 everywhere (sorry i said 8191 before, that was a mistake).

so i guess if you are set up for pure raw, the 65535 numbers in the saturated areas are suspicious to me. not sure why the data would have been scaled to 65535 unless for some reason the Debayer process does that.

after bias subtraction and flat scaling of course the images will invariably have different values for the saturated data... but still i don't think it matters because HDRC is rescaling each image to fill the range 0.0-1.0.

maybe Juan will see this and comment - there are definitely problems here and i'm not sure it is due to the input data (seems that all of us are having similar problems with eclipse images and HDRComposition)

rob

[andyc67]

That is possible however the files that I was working with were not debayered. I don't think it matters though.