PixInsight can't read FITS files

[bodo]

I have fits files created with FireCapture. If I drop or open them in PixInsight, they are not shown correctly.
The following problems occur:
If I have saved it in 16bit format, I see a gray picture with no data, even through scaling - seems empty
If I have saved them in 32bit format, I see a grayish picture, but scaling brings up some data, but 'black' is at 0.5

Both formats are read correctly by dropping them into Fitswork.

Please Help

PixInsight Version 1.8.5 / FireCapture Version 2.6.08 / Fitswork Version 4.47

[pfile]

hi i think the file is too large to be attached in this forum - 161kb for 2 fits files seems much too small, even compressed. can you upload them to google drive or dropbox?

rob

[bulrichl]

Hi Bodo,

for the 16-bit format, this could be due to a wrong BZERO value in the FITS header of the FITS file (32767 instead of 32768) see https://pixinsight.com/forum/index.php?topic=4212.0 , reply #7.

If so, the best way is to patch this value (with a Hex editor) in the FITS header (and advise the programmer of FireCapture to alter the value).

Bernd

[bodo]

Hi pfile, I deliberately just posted a small file - 32 MB would be too big for a 'real' image from my camera 
they are just showing the hp symbol of my printer ....

SIMPLE  =                    T / Java FITS: Sat Sep 15 05:53:16 CEST 2018       
BITPIX  =                   32                                                 
NAXIS   =                    2 / Dimensionality                                 
NAXIS1  =                  288                                                 
NAXIS2  =                  212                                                 
PCOUNT  =                    0 / No extra parameters                           
GCOUNT  =                    1 / One group                                     
EXTEND  =                    T / Extension permitted                           
OBSERVER= '        '           /                                               
TELESCOP= '        '           /                                               
ORIGIN  = '        '           /                                               
OBJECT  = 'Test1   '           /                                               
DATE-OBS= '2018-09-15T05:53:16.753' /                                           
CAMERA  = 'ZWO ASI1600MM Pro'  /                                               
CCDTEMP = '30.5    '           /                                               
EXPTIME = '0.100000'           /                                               
END                                                                             

@bulrichl
If you look at the header, there are no entries for bzero or anything that can go wrong ...
I just tested MaximDL, if I drop them there, it shows correctly ...

I've attached the corresponding jpg, saved by fitswork.

Regards
Bodo

[bulrichl]

Hi Bodo,

I was talking about 16-bit integer format (only then BZERO makes sense), and you show a header of a 32-bit floating point format (BITPIX = 32).

Please save a file in FireCapture in 16-bit integer format and have a look at the FITS header then.

Bernd

[pfile]

ok well when i did 7za e on this archive i got two 0 length fits files, hence my supposition that something was wrong with the upload.

so unless someone else can unarchive the file, i think the archive might be corrupt.

rob

[msmythers]

rob

I was able to open the archive. I re-zipped the files in this archive. Give it a try.


Mike

[bodo]

I downloaded and tested it here - everything fine  ...

Bernd, please notice the little difference for the BITPIX entry:
BITPIX = 32  -> is 32 bit integer format
BITPIX = -32 -> is 32 bit float format
(as I saw from fitswork)

I don't know how they distinguish between signed and unsigned values ...

Regards
Bodo

[pfile]

well i opened the zip file that mike posted.

for the 32-bit file, all i can say is that there's no standard for how values should be represented in floating point fits files. pixinsight uses [0.0-1.0] and expects fits files to have this format. in your 32b file the values appear to be centered around 0.5. since this is not out of range for PI, it happily brings in the file with no remapping. if you subtract 0.5 from the image using pixelmath, you get something that looks about right, though already stretched. at any rate the 32b file behavior is not surprising since writing a floating point fits with one application and reading it with another is essentially an undefined operation.

the 16b file sort of looks OK to me. it does not have a lot of contrast but this may simply be because the file is in a linear state. i've never used fitswork so i have no idea what it does - does it apply a stretch by default?

unsigned 16b integer files should be pretty darn compatible across platforms...

rob

[bulrichl]

I experienced the same as Rob with the zip file that Mike posted: the 16-bit file displays the information shown in the JPG file (albeit with lower contrast), but the 32-bit file doesn't. Bodo wrote that the latter is a 32-bit integer (not floating point) format. I have never heard of that. Is it possible to configure Firecapture to write 32-bit floating point instead of 32-bit integer format?

Is this a real problem or just academic? Single frames can never exceed 16 bit with present-day sensors. I don't know Firecapture though. If this program is able to stack frames, a 32-bit format would make sense of course.

Unfortunately, I cannot analyze the 32-bit integer format because i am on the move for the next 8 days.

Bernd

[bodo]

Hello Rob,
Fitswork does basically the same as the histogram function in PixInsight with Auto Zero Shadows / Highlights.
A scaling appears in FireCapture when switching 16 / 32 bit integers (a factor of four), but linear.
So both pictures contain the same data - except data width and a scaling factor of four. (I've attached the histograms)

I played around a bit and here are the things that are irritating me:
if i use the default for FITS:
The 16bit version comes up, but if I look at it with the histogram function, it has an offset of 0.5. Where does it come from?
If you scale min / max, it looks ok on the screen.
The 32bit version looks just gray, the histogram / STF does not display anything. Why?

if i use the 'Singed integer images store physical pixel data' for FITS:
The 16bit version comes up, and the histogram looks correct for the expected values, but if you try to set the range, the preview window is gray.
The 32bit version looks black, the histogram does not display anything. Why? STF brings up a Gray/Black two color picture.

So it seems only the first case is working in some way ...
but I would like to have the right scaling (0 to ..., not 0.5 to ...)

For 32bit:
One Idea: is PI assuming a 32bit integer picture as a 32 bit float one?

@Bernd:
both formats (32bit unsigned integer / 32bit float) are listed in the format explorer for the FITS format.
and yes, FireCapture can do DarkFrame / FlatField and stacking.

From my side I can live with 16 bit at the moment (since I want to do the corrections / stacking in PI) and it saves disk space 
just the offset in the data is irritating me.


Regards
Bodo

[pfile]

ok for the 32b images i made the assumption that they are f32, but in fact you are right, they are i32. PI is reading it OK - the console says it's a 32b integer file. so i don't think it's misinterpreting the file as floating point.

but i think there is a misinterpretation going on - PI handles only unsigned integers in i16 and i32 fits files.

so, the 0.5 might be a consequence of firecapture writing the most significant bit of the data to 1. interpreted as a 2s complement signed integer, these would be negative numbers. i don't know why firecapture would do that, but when PI reads this file it will assume the samples are unsigned integers... so for instance the 16b numbers 1000_0000_0000_0000 -> 1111_1111_1111_1111 represent -32768 to -1 to firecapture but PI sees them as 32768->65535 which are normalized to 0.5 -> 1.0 in PI.

i think it's likely that the same is going on with the 32b images - they are negative signed integers which PI interprets as unsigned. again, why firecapture would write negative numbers is a mystery, but it seems to me the only explanation for the 0.5 offsets.

you can't be the only person using firecapture and PI so my assumption is there must be some setting in firecapture that will cause it to output unsigned integers and you should be fine from that point.

rob