Bit depth of Images

[Jack Harvey]

This discussion is going on in a different forum.  I thought it might generate some interesting dialog on this forum:
Re: [SBIG] Re: Bit depth of images

 Any good processing program will work with the
data in floating point format for all its internal processing. After you've
done some sort of stacking- including averaging- you don't have integer data
anymore. If you convert to integer, you introduce quantization errors. For
the highest quality data, it's best to leave it floating point.

Also, just the basic calibration process- darks and flats- can push the
files over 16-bit. If so, your stack will be similarly deeper.

I can't think of any reason not to save your processed data as anything but
floating point FITS. All the raw data is probably best saved as 16-bit
integer FITS.[/i][/i]

[Nocturnal]

Doesn't sound like there's much to discuss. The author is correct I use 32b float TIFF but that's only because DSS generates that by default. Files are also a good bit smaller than 32b float FITS.

[Jack Harvey]

Sander, I agree as it is what I do, but just thought I would check to see if my logic is off<G>.

[Nocturnal]

Jack,

I think you should never discard a 32b master image once fully processed in PI. For publishing I convert a copy to 16b TIFF (although 8b png would work too) so I can add text in PaintShopPro and create different sized versions in jpg etc. I never fiddle with color in PSP.

[Alexander]

Hi this some thing i have wondered about for some time my camera is a Meade DSI which give out 16bit images, I use nebulosity which has different options 1 save as compressed Fits
                                     2 save in 32 bit floating point
                                     3 scale to 15 -bit( 0-32767)

I was told that leaving all these boxes unticked the images will be saved in 16 bit.which i do for my LIGHTS DARKS FLATS.
Hope im on the right track with this any advice would be helpfull.

Alexander

[Nocturnal]

Nebulosity does both capturing and processing so it's important to distinguish what type of saving we're talking about.

- make sure captured images are stored as 16b FITS
- make sure processed images are stored as 32b float

I think that means you leave all boxed unchecked. I'm pretty sure I do wit my QHY8. I don't use Neb for any processing though. Ask the stark-labs group for further help.

[mmirot]

The is one fellow on there that thinks importing at 16 bit fits or tiff image into PS is just as good as 32 bit.

He does to think it does not matter because the output such as print or the computer screen will never see it.

Max

[Niall Saunders]

OK,

Here is my take on the situation.

If you are using a 16-bit CCD imager, then you should save ALL of your image data in 16-bit Unsigned Integer FITS format. That is effectively what the electronic circuits in your imager will be sending to your PC, so to convert this data to any other format seems foolhardy.

Some of us will be using imagers like Meade's DSI, using Meade's "Envisage" software (or 'daftware' as I like to think of it) - this software 'can' let you save in 'FITSINT' mode (a 16-bit Integer mode) but the file format is corrupt, because the siftware writers made a mistake - they do not seem to have relied on CFITSIO - and Meade have steadfastly refused to correct the bug since it was first pointed out to them - by me - YEARS ago). So, be warned, do NOT use "FITSINT", which means that you will HAVE to use "FITS" or "FITS3P", both of which save data in 32-bit Float mode.

If anyone else is using (unwittingly) 32-bit Float mode, then you will soon find out that you will NOT be able to use the Image Calibration routines that will be released with PI v1.6.0. This is because PixInsight has NO IDEA that your 32-bit data does NOT represent data in a range from -2^31, through 0, to +(2^31)-1. There is NO DEFINED STANDARD to allow a 32-bit Float FITS image file to inform subsequent software that the data contained in the image actually only should have a maximum range of 0.0 to 65535.0 - and because of this lack of 'standard definition', Juan (quite rightly) refuses to allow PixInsight to make 'assumptions' that this will normally be the case.

So, do NOT 'capture' your image data in 32-bit Float mode - stick to 16-bit Unsigned Integer (iff possible).

If you absolutely can't do this - like me, you are sticking with Envisage, because you have no need to move to Nebulosity, and because Nebulosity does not (temporarily) 'stack on the fly' like Envisage does - then you have to 'pre-scale' or 're-scale' all your images before starting the calibration process. In other words, you have to get all the 32-bit Float data transformed into 16-bit Unsigned Integer format. (With 16-bit Unsigned Integer, the maximum data range of 0 to 65535 is IMPLIED BY DEFINITION - and so PixInsight knows EXACTLY how to interpret the data). It sounds complicated, but it isn't really that complicated - an MOST OF YOU will NOT have to worry about this. For those of you who feel that they WILL have to face up to this step, let me know and I will explain in gretaer detail (I can put together a brief video explaining the steps in more detail, but until Juan sorts out v1.6.0 and the server issues, I can no longer access my online web storage area, so all of my past videos are unavailable, and I cannot upload any new stuff).

Whilst you are in 16-bit Unsigned Integer mode, PixInsight will quite happily read in your raw subs, and will create MasterFlatDarks, MasterFlats, MasterDarks and MasterLights from that 16-bit data - as needed - and will do ALL the 'internal processing' in 32-bit Floating point mode, and will also automatically 'iternally rescale' all the data to a range of [0.0, 1.0]. Furthermore, any data that you ask to be saved to disk will remain in the [0.0, 1.0] range - providing you ask for it to be saved in a floating-point format. And, here, the recommendation is that (now) you should save in 32-bit Float Fits mode - and should also save all further images stages, or associated images, for the processing sequence in the same 32-bit Float mode - right up until your final image.

If, for some reason, you need to get your image 'out of' PixInsight and into some other processing package - and if that package cannot read a 'standard' FITS 32-bit Float image (like PS, for example), then you can use the SampleFormatConversion process to 'resample' the image to, say, 16-bit Unsigned Integer, and then save THAT image as a FITS or TIFF file.

However, be aware that there WILL be a small 'quantization' loss of detail in doing so. Not that this should be that significant a problem - after all, the whole reason for doing this, remember, is to be able to further degrade the image in some software package that probably can only work with 16-bit (or even only 8-bit) integer data in the first place. Just don't do this and expect to 'maintain' image quality if you are intending doing a lot of processing outside PixInsight, or if you will subsequently be bringing the image back into PixInsight for further subsequent processing.

The only exception that I could foresee, is if you were exporting image data to a 'painting' package, to allow the creation of 'artistic masks' that will then be brought back into PixInsight - to be used AS MASKS.

Once you have started to manipulate your original 16-bit Integer images in a 32-bit Float world, then you can only ever LOSE data integrity by changing back to 16- or even 8-bits, That is why it is so importatnt to make those changes ONLY WHEN NECESSARY, and most often ONLY at the 'end' of your processing stage - typically at the JPEG or 'print' stages.

Even right now, with PixInsight v01.05.09.0547, I can FULLY CALIBRATE AND PROCESS my raw CCD data without EVER returning from 32-bit mode, and without ever leaving the realm of PixInsight. The ONLY time that I change from 32-bit mode is if I finally choose to create a JPG image. In fact, other than the software used to capture the data, and that used to guide my scope, the only other package that I currently use is AIP4WIN - and even then that is only because I have been too lazy to write a script that will tabulate the whole-of-image 'statistics' for a series of subs in a given folder (oh, and I suppose that I can also claim to use MS Excel to 'graph' the tabulated statistics, so that I can be sure that no gross anomalies will creep in to the earliest stages of calibration - and I also use the freeware "Sequentialize" program from Polybyte Software to rename all the images in my 'sorted' subfolders). I no longer need Nebulosity or DSS to calibrate, deBayer, align or stack my subs, and I can no longer justify the time, or cash, investment to explore the likes of CCDStack as an alternative - PixInsight does EVERYTHING I need it to do, RIGHT NOW - and that is BEFORE the new calibration routines are made available in v1.6.0

HTH

Cheers,

[Nocturnal]

The is one fellow on there that thinks importing at 16 bit fits or tiff image into PS is just as good as 32 bit.

He does to think it does not matter because the output such as print or the computer screen will never see it.

Max

People like that really don't understand what happens when you process your image. Minute brightness differences can be recovered using wavelets for example. Compressing a 32b float to 16b int range removes some of that detail and it'll be gone forever.

[RBA]

People like that really don't understand what happens when you process your image. Minute brightness differences can be recovered using wavelets for example. Compressing a 32b float to 16b int range removes some of that detail and it'll be gone forever.

Might as well convert the image to 8bits from the beginning *since* in the end what he'll share is a 8bit JPEG... 
There's a lot of nonsense in that thread. I was going to chime in answering that guy who claims that using masks is cheating, but then realized I've got better things to do with my time 

[sreilly]

Well guys, I started that SBIG thread because after all these years, I needed to know. But what did not come out in that thread is saving the raw data as 16 bit unsigned as Niall pointed out. At least I didn't see this if it was posted. I use MaxImDl/CCD for image acquisition and there are a number of options for saving the data. By default, whatever you set the settings for it will save the new raw data and processed data as the same, I think.

Looks like I'll need to post on MaxIm's group and see how the raw data is saved if any other than 16 bit fits. I'll corner Doug at NEAF and get a bit more in-depth information on this as well.