(SOLVED) what is wrong with this FIT?

[roryt]

Hello,

Not sure if this is a PI bug, but I have no other clue so far.
I have a stack of bias that PI shows with unreasonable histogram/statistics and the calibration from PI fails.
From the other hand both nebulosity (the creator) and MaximDL can read and show them correctly.
The same with the master bias, created with nebulosity. Please see the attached screenshots.
The FIT used can be downloaded from http://www.roryt.gr/astronomy/bias.zip .
Of course no stretch has been applied.

Any help will be much appreciated, currently I'm stacked to use Neb to do the pre-processing and I hate to do so.
Since 1.6 I use PI for everything and I'm happy with it.

 

[Nocturnal]

Hi,

you should create new masters for your calibration frames in PI. Compatibility between stacking software isn't very good. You can't use a DSS master file in PI either. Nor can you do it the other way around.

 

[roryt]

Sorry Sander but you missed the point. I may used the master bias for my screenshots and for the download, but the exact same problem exist with the single raw bias frames. They are just not been read correctly in PI, something that never happened before. The same time the other programs can read them (just confirmed with ImageJ).

 

[Nocturnal]

Ah. In that case I don't know, sorry.

 

[ceterumnet]

Have you tried changing the

"Signed integer images store physical pixel data"

In the fits format settings?

I've attached a screenshot showing the item.

Sometimes I have to check that when I bring things in from Neb or other software...

 

[ceterumnet]

Nevermind...I downloaded your file and tried that.  It didn't fix the issue.  Weird.

 

[ceterumnet]

I have played with this a bit more...needless to say - I am puzzled...

I played around with some pixelmath just to isolate ranges of pixels, and it is definitely not behaving the way I would expect...

I look forward to hearing a solution on this one...

 

[roryt]

Strange, is it? I'm straggling with it the last days and I'm stacked.

Mind that with my current PI settings I have fully processed a lot of images captured in the past with Nebulosity and my previous camera.
This is the first attempts though to process images with the new one, and the only different I can think between them is that the drivers of the new one produce images with overscan areas. But again, it should not matter, should it? You average your masters and then you correct for overscan during lights' calibration, or not ?
BTW, using the masters made from Nebulosity I can continue the rest (lights calibration etc) from PI, so it is not a matter of incompatibility of masters, just PI, for some strange reason, can not read correctly the bias.

 

[ceterumnet]

The strangest thing to me is that it says there is a max value of 65535 - which should be a white pixel(s).  I can't find any fully saturated pixels... 

 

[roryt]

The strangest thing to me is that it says there is a max value of 65535 - which should be a white pixel(s).  I can't find any fully saturated pixels...  

PI shows a hot pixel at the position that it says (1359,752), which is again strange since this is a master bias from raw biases at -25C with Sony 285 chip
and the other programs do not report hot pixels

BTW, using the masters made from Nebulosity I can continue the rest (lights calibration etc) from PI, so it is not a matter of incompatibility of masters, just PI, for some strange reason, can not read correctly the bias.

I stand corrected, if I use the CALIBRATED lights from nebulosity then I can do align/stack with PI.
But that means that I can not use the overscan correction option from PI.

 

[Juan Conejero]

Hi Ioannis,

The FIT used can be downloaded from http://www.roryt.gr/astronomy/bias.zip

This FITS file stores a 32-bit floating point image. *This* is the problem. Due to the lack of standardization of the FITS format (others call it 'flexible') that we have commented countless times, PixInsight has no way to know the numeric range to which the stored data has been referred (that is, where is black and where is white). PixInsight always generates floating point images in the [0,1] range, but other applications apply their own (usually undocumented) rules. In practice, this prevents sharing floating point data between applications.

The only reasonable solution is generating raw data as it *should* be generated: unsigned 16-bit integers. This is the native format of CCD cameras and there are no problems at all with it.

Having said that, it seems you can load this image correctly if you do the following:

- Open the Format Explorer window.

- Double click on the FITS format item, left column.

- On the FITS Format Preferences window: (1) click the Reset button, (2) enter 65535 as the Upper Range value of Default Floating Point Input Range, and (3) click OK.

Now you can load this bias frame. Apparently this FITS file stores 16-bit integers as 32-bit floating point numbers. As you surely know this does not mean that you have more precision; it just means that you are wasting twice the required disk and memory space.

Finally, don't forget to reset FITS Format Preferences when you are done, or any floating point FITS file generated with PixInsight won't load correctly.

 

[roryt]

Great Juan, thank you !

I'm not on my field laptop right now but I'll try asap. I was suspecting that the problem was something similar, after upgrading Nebulosity to he newer versions I had a lot of similar small problems with the configuration options. I recall there is an option in Neb for 32bit floating point, either the upgrade or me by mistake should have enabled it.

Your work around above is very welcomed , that means that I will not trash the last 2 days (I could get again bias/dark and -maybe- flats indoors, but I was anxious about my lights)

Once again thank you, I'll test and revert.

 

[roryt]

Yap ! Spot on, thank you.

I actually used Maxim to batch convert all to 16bit integer (I hope it means unsigned because it does not have other options) and now everything is correct

 

[Juan Conejero]

Glad to know it's working. Note that you can carry out the batch conversion very easily with ImageContainer in PixInsight.

 

[roryt]

Yes I know, but that would needed to change PI's ranges for FIT first and then put them back to default, right?
Actually I used Maxim already, it was only three nights , and now nebulosity's settings are back to normal 16 bit, so....