14 Bit Image, 16 Bit Processing

[Farzad_k]

Hello,

In search of answers to why my hour-long images of the Veil nebula and the Pinwheel galaxy are so faint someone has mentioned that a reason could be the 14 bit images coming out of my Nikon D7200 DSLR are being processed as 16 bit images. I have never heard of this before. Where can I get more information on this? I had assumed that PI can readout the technical aspects of the hardware in the image headers and deal with them internally.

Thanks


Farzad

[Nocturnal]

Hi,

In summary, whoever said that did not know what they were talking about. As long as your raw files contain all 14 bits it doesn't matter to PI (or other software for that matter) what the bit depth of the ADC was. It just sees numbers and works with those.

Confirm with others who use your camera that you are using an appropriate ISO setting. You need unity gain or higher. Then first stack uncalibrated to verify the data is there. Once this is taken care of, calibrate first with darks and then with flats and dark-flats. After each step confirm you are getting a good result. If not then you know which step is blowing away your data.

Or post a link to an unprocessed (raw) stack here so someone can take a look at it. 'Faint' means nothing if all the data is there. My ASI1600 images have a very narrow peak on the left after stacking. Once stretched the data is there.

[pfile]

are you using SGP and saving as FITS?

rob

[Farzad_k]

Hi,

In summary, whoever said that did not know what they were talking about. As long as your raw files contain all 14 bits it doesn't matter to PI (or other software for that matter) what the bit depth of the ADC was. It just sees numbers and works with those.

Confirm with others who use your camera that you are using an appropriate ISO setting. You need unity gain or higher. Then first stack uncalibrated to verify the data is there. Once this is taken care of, calibrate first with darks and then with flats and dark-flats. After each step confirm you are getting a good result. If not then you know which step is blowing away your data.

Or post a link to an unprocessed (raw) stack here so someone can take a look at it. 'Faint' means nothing if all the data is there. My ASI1600 images have a very narrow peak on the left after stacking. Once stretched the data is there.

I suspect a lot of people out there don't know what they are talking about. Which is why I am here. It seems upsured that I can't rely on the stats that are being shown to me by SGP. And in fact I have also heard from them and they say that the FITS is always 16 bit and for purposes of the stats the bits are already converted.

Even though I don't yet understand all this at least I am glad I don't need to sit there and make my own calculations to see if the highlights are being blown or not.

In terms of the practice of others, it is all over the map. The best thing for me to do is to just try a bright target with different settings and see what I get. I don't understand the statement "You need unity gain or higher." I would appreciate it if you could elaborate. Also I am not very good with acronyms although I am getting better at it.

Thanks for your input.

Farzad

[Farzad_k]

are you using SGP and saving as FITS?

rob

Rob,

I am using SGP and I am saving as FITS. And i now understand that FITS is 16 bits and the stats that are shown are based on the data being converted to 16 bits - or something like that.

Farzad

[pfile]

this is correct - the SGP guys a long time ago decided that when converting a CR2 file to fits, they would call DCRAW with the option that rescales the data from the 14-bit native space of the camera to the full width of a 16-bit integer. at some point i asked them not to do this, and they said they'd think about it and then nothing happened. i think for their userbase, if they just changed this midstream, there would be a lot of confusion. the net result is that if you save a CR2 with SGP, then it's still in the native 14-bit format, and if you save a fits, the data has been rescaled. if you open that CR2 in pixinsight, PI calls DCRAW in such a way that the file is 1) composed of 16-bit integers and 2) the 14 bit data occupies the lower 14-bits of these 16 bit integers with the top two bits set to 0. this creates an incompatibility with, say, calibration frames captured as CR2 and lights captured as FITS coming from SGP.

FITS itself can handle all kinds of data types - 8 bit integer, 16 bit integer, 32b floating point, 64b floating point, etc.

as for unity gain, that's a term that means 1 photoelectron = 1 ADU. generally speaking the ADU values that the camera stores follow the formula ADU = gain * electron count. in a DSLR, the ISO is essentially an amplifier gain, and in every DSLR there's going to be an ISO for which the gain is near 1.0. probably never exactly 1 because of how the ISOs are quantized... on most canon cameras this is somewhere between ISO 800 and ISO 1600.

rob

[Farzad_k]

Rob, thanks, and what is CR2?

[pfile]

sorry - CR2 is NEF for canon. it's canon's Canon Raw 2 format...

most people use canon cameras for astro so i'm used to writing CR2 when i mean "the camera's native raw file format"

rob

[Farzad_k]

No problem.

So you are saying my goal should be to shoot at an ISO that produces a final "gain" that is equal to one or higher. Do you know a source of reading on that for Nikon?

thanks

[pfile]

i think roger clark at clarkvision.com has a bunch of camera statistics listed.

rob

[Nocturnal]

I am quite sure it doesn't matter if a 14b image is saved as-is or shifted right (multiplied essentially) to make it look like a 16b number. As long as all your files are treated the same way the end result of your stacked image is identical. My ASI1600 has a 12b ADC and FITS files have 16b values where all are shifted right by 4 bits. I think they do this to prevent confusion when values from the camera won't go over 4095. Clearly this multiplication is a NO-OP. The essential thing is that all detail from the ADC is preserved. You don't want to shift left and loose your LSBs.

[pfile]

my understanding is that DCRAW is scaling the data, not merely shifting it right.

i have to read dcraw.c again to really find out for sure.

rob

[Nocturnal]

If you have 14b integers and scale them to 16b you simply shift them all right 2 bits. There is after all no detail hidden behind the LSB.

[pfile]

please read dcraw.c. that's not all it's doing, it's literally scaling the data to fill the 16 bit space from the darkest to brightest values in the file.

Code: [Select]

dcraw -v -4 -d -T _MG_3475.CR2
Loading Canon EOS 50D image from _MG_3475.CR2 ...
Scaling with darkness 995, saturation 15763, and
multipliers 2.329392 1.000000 1.186999 1.000000
Building histograms...
Writing data to _MG_3475.tiff ...

to my knowledge, SGP is calling dcraw with "-d" instead of "-D" as pixinsight does.

rob

[Nocturnal]

They are taking the range of numbers they find and scaling that to 16b? That would be so indescribably horrible I did not even consider that. It would make stacking of those images meaningless. So a bias frame with max value 200 now has max value 2^16 -1? Completely useless for astro photography.

Luckily the ZWO ASCOM driver does not not that. It simply shifts values 4 bits to the right. Goodness we'd have pitchforks in the streets if they scaled as you are describing :-)

Not to mention float multiplication of ints and then shoving them back into ints is not a great idea either. Shudder.