IEEE 754

[harist]

While integrating flats I noticed that saving the "Integration" as 32bit IEEE 754 fits, the values' range is expanded from 0 to 1 (with the initially integrated image having values 0.23 to 0.47). This does not happen if I save the image as 32 bit unsigned integer and does not seem to create any problems with unbinned flats. But when I integrated 2x2 binned flats the IEEE 754 fits looked different from the produced integration (could be the autostretch function producing different results due to different ranges of values) but more importantly with catastrophic calibration effects. Temp_flat is the original image and temp_flat1 is the IEEE 754 saved image after auto stretching both.
What's the catch here, am I missing something obvious?

Thanks,
Tasos

[Juan Conejero]

Hi Tasos,

So if I understand this, the master flat frame (the output of ImageIntegration for your individual flat frames) is correctly generated. However, if you save it with PixInsight in 32-bit floating point format (which is the default output format of ImageIntegration) and load it again from disk, the image is clipped at high values, but this doesn't happen if you save in 32-bit unsigned integer format.

While integrating flats I noticed that saving the "Integration" as 32bit IEEE 754 fits, the values' range is expanded from 0 to 1 (with the initially integrated image having values 0.23 to 0.47).

If this is happening, then it is indeed very strange. Nothing similar has been reported before. Could you upload both images (in float and integer format) somewhere?

[harist]

Hi Juan,

So if I understand this, the master flat frame (the output of ImageIntegration for your individual flat frames) is correctly generated

Yes, this is correct. I'll upload the images this evening and send you an email.

Thanks,
Tasos

[Juan Conejero]

Hi Tasos,

Thanks for uploading the images. I can't reproduce your problem on any platform. Both images (32-bit floating point and 32-bit unsigned integer versions) are identical except for the natural numerical precision difference. See this screenshot:

http://pixinsight.org/images/forum/20110629/ieee754.jpg

Statistics are identical for both images, which look perfectly valid as master flat frames. Check if you've been confused by unexpected STF settings or something similar. Another possibility could be custom FITS floating point range settings. Open the FITS Format Preferences dialog from the Format Explorer window and check that you have 0 and 1 for the low and upper bounds, respectively, in the default floating point input range. If in doubt, click the Reset button then OK.

[harist]

Hi, Juan

I checked the fits floating point settings and it was min:-9223.372036854780 and max:0.0 !!! No idea where these came from and I wonder how much of my processing has been affected so far! Anyway changing these to 0 and 1 fixed the problem.

Thanks,
Tasos