Thanks Fred I have plenty to get on with now ;-)
WBPP Processing fits Files of OIII SII results in Greyish can't reset to default Dark but Ha is Okay?
- Thread starter Hagar
- Start date
Hi Fred,
Okay I have it up and running ;-)
I have been researching and playing around with your suggestions and written a script file in Python Jupyter notebooks, matplotlib and using astropy.io.fit. and have a visual Histogram this which still was not working but after reading a Post in Forum back in 2007 from Juan Conejero it clicked, I can now process these Fits files correctly and adjust before importing to Pixinsight with Pix Floating point range set 0.1.
Now I can adjust the Fits Header BZERO BSCALE and BITPIX values and scaling data using int16 but my numbers are a little differentbzero = -15.00
bscale = 35535.0
bitpix = 16 (do you think I should use 32bit here? )
to reflect your numbers above in Fits Floating Point Ranges ?
Question.. How did you know what numbers to use in the Pixinsight Fits Floating point ranges to fix this issue I have?
as those numbers you put in are perfect.
Obviously its early days for me I still have do some fine tuning here but would not mind you thougths
I appreciate your help on this.
Thanks Peter
Last edited:
No. Normally when the sample format integer is used for intensity values in FITS format, these are signed integer values. For 16 bit, the range is -32768 to +32767. The keywords BZERO and BSCALE are used in order that unsigned integer values (for 16 bit: in the range of 0 to 65535) can berepresented in FITS files. When unsigned integer values are to be represented, the value of keyword BZERO must be set to 2^(BITPIX - 1), so for BITPIX = 16, this value has to be 32768, and BSCALE must be set to 1.0. So you should not change values of the keywords BITPIX, BZERO or BSCALE, and this is not at all possible in PixInsight.
This all has nothing to do with the present problem, because the file in question contains intensity values in the sample format 32-bit floating point. PixInsight internally represents values in the range of 0.0 to 1.0, and when writing a file, it will contain values in this range. However, for FITS files containing floating point numbers, no range is defined (this is the issue!). Therefore, when you want to import a FITS file in PixInsight, the input range of floating point values has to be set in the FITS Format Preferences (see post #17).
When you set in FITS Format Preferences, section 'Floating Point Out Of Range Policy', 'How to proceed:' to 'Always ask', PixInsight wil indicate the full range present in the FITS file, in case of your file:
Minimum: -52620.3203125Maximum: +64173.91015625In this file, some negative values are present. If you set an input range of 0.0 to 65535.0 and 'Action' to 'Truncate all out-of-range values', some values are clipped. You can call the Statistics tool (option 'Unclipped' shall be disabled) and will see count (%) = 99.41889. This means: (100.0 - 99.41889) % = 0.58111 % of all pixels are clipped with this input range setting.
When you set an input range of -15.0 to 65535.0, count (%) = 99.93772 results, i.e. 0.06228 % of all pixels are clipped.
Bernd
Last edited:
Hi Bernd,
Thank you for the detailed reply, I will do some more research and homework on this.
regards Peter
fredvanner
Well-known member
I think it is worth examining this data a bit more closely.
The FITS header indicates that it is from an SBIG STXL11002 CCD camera, which matches the image dimensions.
This camera has a 16 bit A/D, so the "natural" data format would be 16 bit unsigned integer (range 0..65535).
It appears to be a single 1800 sec exposure (not an integrated image), calibrated with a matching reference dark and a reference flat.
There are still a few hot pixels visible, so a run of CC is recommended.
The data has been pre-processed. A number of keywords included are exclusive to CCDStack [The keywords starting with "DSP" seem to refer to the CCDStack local display settings, and should not influence the data]. CCDStack, like PI, has a standard working image data format of 32 bit float, but unlike PI it does not normalise to [0..1].
The data range of this image (-52620.3203125 .. +64173.91015625) suggests that either (A): the original integer range has been mapped onto a floating range (probably initially -65535..+65535; this would be similar to the FITS signed integer 32768 offset combined with a multiplication by 2) or (B): some post-processing has (a) shifted the 16 bit image data downwards, creating negative values, and (b) generated some extreme negative values.
To test these posibilities, I have histogrammed the data, testing the distribution well into the low tails. Here is a multi-scale histogram plot:
By increasing the bin size to change the plot scale (bin widths in square brackets), it can be seen the the central peak tails of smoothly for both positive and negative values. This suggests that option A is a reasonable hypothesis - which suggests that the whole floating point range of the image is probably valid data (i.e. the "background peak" was close to 50% signal in the original 16 bit data); this is plausible given the F number (~F/6.8) and exposure time. However, very few pixels fall in the negative tail. Truncating at -15 was a "trial-and-error" result, but analysis of a cumulative histogram indicates that this rejects less than 8000 pixels in the image, which confirms it as a reasonable choice. The upper limit was selected as being likely to be the max of the original data range; however, it would be reasonable to set a max value as the maximum data value - +64173.91015625, or 64174 for simplicity.
The FITS header indicates that it is from an SBIG STXL11002 CCD camera, which matches the image dimensions.
This camera has a 16 bit A/D, so the "natural" data format would be 16 bit unsigned integer (range 0..65535).
It appears to be a single 1800 sec exposure (not an integrated image), calibrated with a matching reference dark and a reference flat.
There are still a few hot pixels visible, so a run of CC is recommended.
The data has been pre-processed. A number of keywords included are exclusive to CCDStack [The keywords starting with "DSP" seem to refer to the CCDStack local display settings, and should not influence the data]. CCDStack, like PI, has a standard working image data format of 32 bit float, but unlike PI it does not normalise to [0..1].
The data range of this image (-52620.3203125 .. +64173.91015625) suggests that either (A): the original integer range has been mapped onto a floating range (probably initially -65535..+65535; this would be similar to the FITS signed integer 32768 offset combined with a multiplication by 2) or (B): some post-processing has (a) shifted the 16 bit image data downwards, creating negative values, and (b) generated some extreme negative values.
To test these posibilities, I have histogrammed the data, testing the distribution well into the low tails. Here is a multi-scale histogram plot:
By increasing the bin size to change the plot scale (bin widths in square brackets), it can be seen the the central peak tails of smoothly for both positive and negative values. This suggests that option A is a reasonable hypothesis - which suggests that the whole floating point range of the image is probably valid data (i.e. the "background peak" was close to 50% signal in the original 16 bit data); this is plausible given the F number (~F/6.8) and exposure time. However, very few pixels fall in the negative tail. Truncating at -15 was a "trial-and-error" result, but analysis of a cumulative histogram indicates that this rejects less than 8000 pixels in the image, which confirms it as a reasonable choice. The upper limit was selected as being likely to be the max of the original data range; however, it would be reasonable to set a max value as the maximum data value - +64173.91015625, or 64174 for simplicity.
fredvanner
Well-known member
The header notes that the image was captured withn the SBIG AOX (adaptive optics) corrector. The equipment configuration has a calculated Dawes limit of ~0.3", and a measured image scale of ~0.63"/pix. Running FWHMEccentricity with a Moffat4 model, the mean FWHM is measured at ~3 pix = ~1.9". This would be fantastic seeing for me here in the UK, but may be a little disappointing for an excellent dark sky site in NSW with AOX. Of course, there may be issues tracking a 17" scope for 20 mins to < 2" ... 
Last edited:
fredvanner
Well-known member
Looking a little further, the "Heaven's Mirror" observatory is indeed near Yass in New South Wales, but the particular telescope that captured this image has a note "this telescope hosted in Chile".
The "site location" data in the header confuses things further:
SITELAT = '-34 51 50.96' / SBIGFITSEXT Latitude of the imaging location
SITELONG= '-148 58 35.00' / SBIGFITSEXT Longitude of the imaging location
Conventionally, a negative longitude is West of 0, which would place this in the middle of the South Pacific. Measuring East, however, it does correspond to the NSW location of Heaven's Mirror Observatory.
If this is a remote hosted telescope I can see that it would be easy to forget to set up the processing application with the appropriate (remote site) coordinates. Perhaps you should contact Martin Pugh and ask him.
Last edited:
fredvanner
Well-known member
Come to think of it, it would also be easy to forget to set the time / time zone for the remote location. Most of this probably doesn't matter for deep space images (where geocentric coordinates are good enough), but could cause problems for astrometry on solar system objects.
Hi Fred,
Thanks for you time on this information, I am slowly absorbing this and understanding but need more time to do some testing.