Author Topic: New user question  (Read 11275 times)

Offline Martin

  • Newcomer
  • Posts: 19
Re: New user question
« Reply #15 on: 2010 February 21 16:19:02 »
Thanks Niall for the additional explanation.

So, when a 16bit unsigned FIT created in Maxim does not open correctly in CCDSTACK (saturated areas are black e.g star cores), then is this likely to be how Maxim writes the files or how CCDSTACK reads it?

thanks
Martin


Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: New user question
« Reply #16 on: 2010 February 21 16:24:05 »
The only thing odd now, is that 16 bit unsigned FITS files from Maxim, do not open in CCDSTACK (a BZERO problem somewhere) - but that's CCDSTACK/Maxim issue.

And, there we see the nail being firmly hit on the head, twice !!

Several software packages obviously have failed to understand how to implement 16-bit Unsigned Integer data using the FITS specification - and yet it is NOT difficult to do (even without CFITSIO) if you sit down and take your time (it took me one evening, three large coffees, and about three sheets of paper to note all the differences down). I knew that Envisage had made this mistake (quite amusing if not so tragically sad, Meade's OWN software could not even read images that THEY created!!!) - and despite having pointed this out to them, the error STILL exists - after several YEARS now. It seems now that the 'Lord High and Mighty' of acquisition programs may also be capable of the same error (I can't confirm this as, for as long as I can possibly do so, I will NOT invest the kind of money they are asking for their software, even more so if they have 'bugs' like this).

And your final statement,
Quote
So as you can see - the FITS standard is loosely interpretated by the different softwares.
is the most damming. The FITS standard IS 'flexible', bith by name and by nature. But, therein lies its 'power'. It is only when programmers do not 'understand' this flexibility that problems arise.

Fortunately, for us, Juan DOES appreciate the 'flexibility' issues, which is why he cannot allow PixInsight to 'bend' to accommodate them. PixInsight adheres, strictly, to the FITS standard - and ONLY uses the data available in the 'compulsory' FITS header keywords, and only uses that data in the manner originally intended.

If everybody else 'worked to rule' then we wouldn't have these problems.

Cheers,
(sound of soap-box being pushed back into cupboard, and 'click' of megaphone being switched off. Rant over :footinmouth:)
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: New user question
« Reply #17 on: 2010 February 21 16:44:35 »
Thanks Niall for the additional explanation.
So, when a 16bit unsigned FIT created in Maxim does not open correctly in CCDSTACK (saturated areas are black e.g star cores), then is this likely to be how Maxim writes the files or how CCDSTACK reads it?

Hi Martin,

That would be dependent on how the header and data sections of the Fits file have been created in Maxim.

Now, as I refer to my 'FITS Bible' (i.e. the notes I gathered on my long learning session, back on 21/10/07, Chapter One, Verse One):

There are 5 possible FITS Data Unit types
8-bit Unsigned Integers, where BITPIX=8
16-Bit Signed Integers, where BITPIX=16
32-Bit Signed Integers, where BITPIX=32
32-Bit Single Precision Floating-Point Real Numbers, where BITPIX=-32
64-Bit Double Precision Floating-Point Real Numbers, where BITPIX=-64

There is also a proposal to include the following (but I do not believe that it has, as yet, been ratified)
64-Bit Signed Integers, where BITPIX (would be)=64

As you can see , there is NO 'standard' for 16-Bit Unsigned Integers - which is unfortunate, because this is EXACTLY what current CCD imagers need.

However, the 'standard' can be 'expanded' to allow for 16-Bit Unsigned and 32-Bit Unsigned Integers as follows:-

Subtract 2^(BITPIX-1) from every data value, before saving
Then save the data, using BITPIX=16 (or BITPIX=32, for 32-bit data),
but ALSO incorporate two extra FITS Header keywords; BSCALE=1 and BZERO=(2^(BITPIX-1))
Now, when the image is to be opened, if the BSCALE=1 parameter is present,
Add 2^(BITPIX-1) back to every stored data value as it is passed to the parent program

For 16-bit data, 2^(BITPIX-1) = 32768 and for 32-bit data 2^(BITPIX-1) = 2 147 483 648

Usually, what happens is that this crucial stage is either NOT implemented, or is implemented incorrectly (such as failure to include BOTH of the extra keywords, or failure to 'subtract before saving'). Alternatively, images might be saved correctly, but applications attempting to read the format have no idea how to process the incoming data.

In any case, however the failure occurs, it is just shabby programming, made worse by the failure to acknowledge the fact, and then worse still by the failure to do anything about the problem.

Hope this background helps.

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline Jack Harvey

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 975
    • PegasusAstronomy.com & Starshadows.com
Re: New user question
« Reply #18 on: 2010 February 21 16:48:16 »
Howdy Niall

a raw frame downloaded from Maxim says in the fits:  BitPix  16/8 unsigned int, 16 & 32 int, -32 & 64 real
Jack Harvey, PTeam Member
Team Leader, SSRO/PROMPT Imaging Team, CTIO

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: New user question
« Reply #19 on: 2010 February 21 16:53:15 »
Sorry Jack,

I could not clearly interpret your layout of the BITPIX summary.

Can you compare it with my 'understanding', and let me know where the differences are?

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline mmirot

  • PixInsight Padawan
  • ****
  • Posts: 881
Re: New user question
« Reply #20 on: 2010 February 21 17:00:43 »
This why Juan need to release the calibration module so we can work on the stack from the start of the process. Hopefully soon.

Max

Offline Jack Harvey

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 975
    • PegasusAstronomy.com & Starshadows.com
Re: New user question
« Reply #21 on: 2010 February 21 17:06:14 »
In the "Fits Header" in Maxim the BIT Pix entry is the one we are interested in.  That is how it is printed out and I suspect Doug George the Maxim guru would have to give you what you ask for - sorry:-((

And I too are eagerly awaiting the ability to easily calibrate in PixInsight.

And finally (for me) Welcome aboard Martin!

« Last Edit: 2010 February 21 17:24:51 by Jack Harvey »
Jack Harvey, PTeam Member
Team Leader, SSRO/PROMPT Imaging Team, CTIO

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: New user question
« Reply #22 on: 2010 February 21 23:54:45 »
Hi Jack,
Code: [Select]
the ability to easily calibrate in PixInsight
Well, the 'ease' will be dependent on the calibration data being correctly interpreted by PI - so this 'FITS' discussion is probably going to affect more and more people.

I would love to see the actual header from a 'typical' Maxim sub-frame. Feel free to email me one.

You could also Google for <FITS verifier online> - there is an online utility that should test your FITS file for correct use of 'the standard'.

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline Jack Harvey

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 975
    • PegasusAstronomy.com & Starshadows.com
Re: New user question
« Reply #23 on: 2010 February 22 06:54:23 »
And of course many of us create our calibration frames differently, so this will be a discussion also<G>.  The Fits Header from Maxim is sent to your email.
Jack Harvey, PTeam Member
Team Leader, SSRO/PROMPT Imaging Team, CTIO

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: New user question
« Reply #24 on: 2010 February 22 08:17:25 »
Thanks for the Maxim header Jack, I saw it pop up on my iPhone, but I need to look at it in more detail once I am back home.

Cheers,
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1458
    • http://www.harrysastroshed.com
Re: New user question
« Reply #25 on: 2010 February 22 11:13:19 »


Good thats that all sorted then  8)

Harry
Harry Page

Offline Simon Hicks

  • PixInsight Old Hand
  • ****
  • Posts: 333
Re: New user question
« Reply #26 on: 2010 February 22 11:46:45 »
I'm so glad I only work with CR2 and Tiff.   :P   :D