PI doesn't read the correct focal length from the FITS header

A

alange

Member
This is a new issue, I didn't use to see earlier.

PI seems to read a wrong value for scope focal length, which IS correctly written in the FITS header.

The FITS header contains the value 442 for the fits keyword FOCALLEN.

But PI seems to read it as 110.49 mm e.g. 4 times too small... (see Process console for PI interpretation).
I've also checked with both the FITSheader and PhotometricColorCalibration processes, they both agree on the same, but wrong value of 110.49 mm when reading the FITS header.
PixInsight focal-length read wrongly from FITS header.jpg


However doing plate-solving in the PhotometricColorCalibration process, the correct focal length is identified.
PixInsight focal-length identified correctly using PCC plate-solving.jpg


Does anybody have an explanation for this?
 
Last edited:
This cannot be reproduced under normal working conditions. It seems the astrometric solution in the original FITS image is not valid. It is providing a focal length of 110.49 mm when the image is opened, which is taking precedence over the first FITS keyword that you are showing.

This error does not exist in our implementation, or every astrometric solution would be wrong and our entire astrometric engine would be failing constantly. This does not happen, so the only logical way for this to happen is some incorrect metadata in the original image; a bug here is extremely unlikely.

Can you please upload the original FITS file, so we can try to understand what happens?
 
Hi Juan

I believe you are 100% correct, the bug must be somewhere in the FITS header content of my images...
I'm sorry for creating a BUG report on this, as it's most likely NOT PixInsights fault.

I would however still very much appreciate help in understanding the cause of this issue.

I've located an older frame, taken on the exact same equipment, but with another software version of the automation software (ASI Air Pro).

I've attached a link to a google drive folder in which the follwing can be found:

PI focal length issue - Google Drive

drive.google.com drive.google.com

1. New subframe with the problem: Light_SH2-275_600.0s_Bin1_gain90_20210304-204206_-19.8C_0002.fit
2. Older subframe without the problem: Light_IC1805_600s_frame0001_bin1_2020-04-10-222837_-19.8C_gain90.fit
 
In the FITS header of the new light frame "Light_SH2-275...", the value of FOCALLEN (110.49) is too low by factor 4, whereas the EGAIN value (1.00057) correctly corresponds to the GAIN value of 90 that was set in the acquisition software.

In the FITS header of the older light frame "Light_IC1805...", the value of FOCALLEN (443) is correct, whereas the EGAIN value (2.82) is wrong, again a value of 1.00057 is the correct value that corresponds to the GAIN value of 90.

It looks like in the acquisition software one bug (EGAIN value) was fixed and another one (FOCALLEN) was created. Maybe this new bug is fixed in the current version of the ASIAIR app, otherwise please contact ZWO.

Bernd
FITS_headers.JPG
 
I have exactly the same issue that wasn't always there. I also use ZWO ASIAIR Pro. But nebulosity reads the correct focal length of 420mm and PIS 105mm in the FITS Header. ASIAir has 420mm.
 
Bazza104 said:
I have exactly the same issue that wasn't always there. I also use ZWO ASIAIR Pro. But nebulosity reads the correct focal length of 420mm and PIS 105mm in the FITS Header. ASIAir has 420mm.
Click to expand...
Sorry, but this is nonsense. PixInsight reads the value of the focal length that was written by the acquisition software.

There are two possible sources of a wrong focal length value:
1. you inputted the wrong value into the acquisition software or
2. the correctly inputted value was altered by the acquisition software.

If PixInsight shows a focal length that differs from the value that you inputted for your scope, contact the programmer of the acquisition software.

Bernd
 
I believe I have worked out what happened here; it's a bit complicated!:oops:
We are going to look at " Light_SH2-275_600.0s_Bin1_gain90_20210304-204206_-19.8C_0002.fit", which I will just call "SH2".
If you look at the FITS header of SH2 in a FITS viewer (such as FITS Liberator), you will see a quite different header to the one displayed if you load SH2 into PI:
Code: 
    SIMPLE  =                    T / file does conform to FITS standard
    BITPIX  =                   16 / number of bits per data pixel
    NAXIS   =                    2 / number of data axes
    NAXIS1  =                 4944 / length of data axis 1
    NAXIS2  =                 3284 / length of data axis 2
    EXTEND  =                    T / FITS dataset may contain extensions
    COMMENT   FITS (Flexible Image Transport System) format is defined in 'Astronomy
    COMMENT   and Astrophysics', volume 376, page 359; bibcode: 2001A&A...376..359H
    BZERO   =                32768 / offset data range to that of unsigned short
    BSCALE  =                    1 / default scaling factor
    CREATOR = 'ZWO ASIAIR'         / Capture software
    OFFSET  =                   65 / camera offset
    XORGSUBF=                    0 / Subframe X position in binned pixels
    YORGSUBF=                    0 / Subframe Y position in binned pixels
    FOCALLEN=                  442 / Focal length of telescope in mm
    EGAIN   =     1.00057375431061 / Electronic gain in e-/ADU
    XBINNING=                    1 / Camera X Bin
    YBINNING=                    1 / Camera Y Bin
    CCDXBIN =                    1 / Camera X Bin
    CCDYBIN =                    1 / Camera Y Bin
    XPIXSZ  =     4.78000020980835 / pixel size in microns (with binning)
    YPIXSZ  =     4.78000020980835 / pixel size in microns (with binning)
    IMAGETYP= 'Light   '           / Type of image
    EXPOSURE=                 600. / Exposure time in seconds
    EXPTIME =                 600. / Exposure time in seconds
    CCD-TEMP=    -19.7999992370605 / sensor temperature in C
    RA      =              98.2026 / Object Right Ascension in degrees
    DEC     =              4.89836 / Object Declination in degrees
    DATE-OBS= '2021-03-04T19:42:06.601' / Image created time
    INSTRUME= 'ZWO ASI071MC Pro'   / Camera model
    BAYERPAT= 'RGGB    '           / Bayer pattern
    GAIN    =                   90 / Gain Value
    TELESCOP= 'EQMod Mount'        / Telescope name
    CTYPE1  = 'RA---TAN-SIP'       / TAN (gnomic) projection + SIP distortions
    CTYPE2  = 'DEC--TAN-SIP'       / TAN (gnomic) projection + SIP distortions
    CRVAL1  =        97.8924335401 / RA  of reference point
    CRVAL2  =        5.65279346665 / DEC of reference point
    CRPIX1  =        323.626228333 / X reference pixel
    CRPIX2  =        423.537017822 / Y reference pixel
    CD1_1   =    0.000307529599758 / Transformation matrix
    CD1_2   =     0.00245990429904 / no comment
    CD2_1   =    -0.00245833912057 / no comment
    CD2_2   =    0.000313085844039 / no comment
    A_ORDER =                   2. / Polynomial order, axis 1
    B_ORDER =                   2. / Polynomial order, axis 2
    AP_ORDER=                   2. / Inv polynomial order, axis 1
    BP_ORDER=                   2. / Inv polynomial order, axis 2
    A_0_0   =                   0. / no comment
    A_0_1   =                   0. / no comment
    A_0_2   =   -4.67633313285E-06 / no comment
    A_1_0   =                   0. / no comment
    A_1_1   =    6.41107476689E-06 / no comment
    A_2_0   =    -2.9891817272E-07 / no comment
    B_0_0   =                   0. / no comment
    B_0_1   =                   0. / no comment
    B_0_2   =    1.36915406576E-07 / no comment
    B_1_0   =                   0. / no comment
    B_1_1   =   -5.14498079084E-07 / no comment
    B_2_0   =   -3.23651389249E-07 / no comment
    AP_0_0  =    -0.00083198820658 / no comment
    AP_0_1  =   -3.25764060716E-06 / no comment
    AP_0_2  =     4.6922110427E-06 / no comment
    AP_1_0  =    2.95443551766E-06 / no comment
    AP_1_1  =   -6.41595078061E-06 / no comment
    AP_2_0  =    2.97304247704E-07 / no comment
    BP_0_0  =   -7.25986741272E-06 / no comment
    BP_0_1  =    6.76939933507E-08 / no comment
    BP_0_2  =   -1.37154745116E-07 / no comment
    BP_1_0  =   -6.21080938766E-08 / no comment
    BP_1_1  =    5.12261404578E-07 / no comment
    BP_2_0  =    3.24016955865E-07 / no comment
    IMAGEW  =                1236. / Image width,  in pixels.
    IMAGEH  =                 820. / Image height, in pixels.
In particular, note that "FOCALLEN=442", correctly.
However, the header already contains an astrometric solution and it is wrong! The error is clear in the last two keywords: IMAGEW=1236, IMAGEH=820. Clearly this solution was produced from a 4x4 binned version of the image (probably some UI setting error when the solution was performed). This solution uses SIP parameters (the A..., B... parameters) to describe image distortion; I don't know what app was used, but Astrometry.net can generate SIP solutions. PI does not support this (the PI method of quantifying image distortion is much more accurate). To make the solution compatible with PI, PI has "hacked" the header, on input, to extract the linear (first order) part of the solution; it has then calculated the (missing) axis linear scaling factors CDELT1, CDELT2 (which can be worked out from the scale matrix). Then, using these and the pixel size (or some other derived parameter) it has "reverse engineered" the focal length - but since the solution is for a 4x4 binned image, the result is out by a factor of 4. The image RA and DEC are also "corrected" on the basis of the scaled solution, giving wrong values.
What you should really do is delete the (wrong) astrometric solution from the original FITS header, and solve the resulting image in PI.
 
Oh, I'm perplexed, and I stand corrected.

Indeed when I view the FITS file "Light_SH2-275_600.0s_Bin1_gain90_20210304-204206_-19.8C_0002.fit" in a Hex editor, the focal length is:
Code: 
FOCALLEN=                  442 / Focal length of telescope in mm

whereas PixInsight's FITSHeader process displays:
Code: 
FOCALLEN        110.49429        Focal length (mm)

If I understand it correctly, the acquisition software called the plate solver using a 4x4 binned image but did not pass the binning information.

Despite of a wrong plate solve, I guess there is also a bug in PixInsight. I can not imagine any reason for altering an existing FITS keyword FOCALLEN which was written by the capturing software. If the data don't fit together (focal length and pixel size), and the discrepancy is that large, PixInsight should output an error or a warning message, and demand from the user to perfom a new plate solve with PixInsight's plate solving script.

Bernd
 
bulrichl said:
PixInsight should output an error or a warning message
Click to expand...
I think PI thought it understood the header, and was just correcting it (i.e. adjusting FOCALLEN to be compatible with the astrometric solution). However, it missed (and probably didn't even look for) the "IMAGEW, IMAGEH" keywords that made the error relatively clear (if you looked).
Note: IMAGEW, IMAGEH are not in any FITS header keyword dictionary that I've seen; the correct keywords are NAXIS1 and NAXIS2.
I fully understand the PI design thinking: if you are going to import an astrometric solution, you want to make sure that it is at least self-consistent.
bulrichl said:
the acquisition software called the plate solver using a 4x4 binned image
Click to expand...
That's what it looks like - but stored the result in the unbinned file header. I'd call that a bug (the user interface shouldn't let that happen), but it may just be a feature of an over-flexible interface. Whatever the reason, it is a problem of the external acquisition / solver application; PI was just an innocent victim of this error. I don't regard this as a PI bug - the PI adjustment would have worked if the the header had been correct: garbage in - garbage out.
 
Last edited:
Despite of a wrong plate solve, I guess there is also a bug in PixInsight.
Click to expand...

Sorry, but I disagree completely here. Our implementation always gives precedence to the astrometric solution if one is present in the FITS header. Otherwise, what are we supposed to do? Ignore the astrometric solution? Why? If the astrometric solution is wrong or is not coherent with the image, or maybe the other keywords are wrong and the solution is correct, that isn't our problem. The FITS format does not specify a prescribed or standardized procedure to be followed in these cases, so we are free to implement what we consider as the most logical option: trust the astrometric solution.

This is not a bug in PixInsight. This is a bug in the application or process that has generated the original FITS header. This is just how FITS is: an obsolete, inconsistent format where everything is possible, hence a source of endless interoperability problems. This is just a 'nice' example.
 
We agree on the fact that there is a bug in the acquisition software.

However, PixInsights "FITSHeader" (!) process must not display value of 110.49 when the FITS header actually contains a value of 442. I definitely consider this as a bug as well.

Bernd
 
bulrichl said:
However, PixInsights "FITSHeader" (!) process must not display value of 110.49 when the FITS header actually contains a value of 442. I definitely consider this as a bug as well.
Click to expand...
On this I agree with Juan. If the header contains an astrometric solution, it should surely override other (typically approximate, manually input) parameters. I am always happy when an ImageSolver result updates my (approximate) focal length with an accurately-calculated value. If PI is going to leave the solution in the header, it is only reasonable to assume it is correct (what else can you do), and to make the other header keywords consistent with it. However, I would agree that a warning (not error) message if a header value is changed by more some percentage could be helpfull...
... but I often use my Edge8 (nominal f=2032) at focal lengths up to 20% different (because mirror focusing for various different physical lengths of optical train changes the focal length).
 
bulrichl said:
However, PixInsights "FITSHeader" (!) process must not display value of 110.49 when the FITS header actually contains a value of 442. I definitely consider this as a bug as well.
Click to expand...

Our implementation trusts the astrometric solution and replaces the existing FOCALLEN keyword value with the "correct" value that has been derived from the astrometric solution. This is not a bug IMO, but just the most logical decision, for the reasons Fred has described.

On the other hand, "IMAGEW" and "IMAGEH" are invented nonstandard keywords. Nobody should expect us (or anybody) to be aware of the infinite possibilities posed by arbitrary keywords invented by third parties. Again, this is FITS in its purest state.
 
Juan Conejero said:
Our implementation trusts the astrometric solution and replaces the existing FOCALLEN keyword value with the "correct" value that has been derived from the astrometric solution. This is not a bug IMO, but just the most logical decision, for the reasons Fred has described.

On the other hand, "IMAGEW" and "IMAGEH" are invented nonstandard keywords. Nobody should expect us (or anybody) to be aware of the infinite possibilities posed by arbitrary keywords invented by third parties. Again, this is FITS in its purest state.
Click to expand...
I don't advocate evaluating further nonstandard FITS keywords either. However the focal length must not be tacitly altered by a factor of 4. This is not the 20 % deviation that Fred mentioned.

Once again: the FITS file (containing a value of 442 mm for the FITS keyword FOCALLEN) is opened in PixInsight. Without doing anything to the image, calling the "FITSHeader" process displays a value of 110.49 mm without a warning. This shall be considered OK?

No. If the astrometric solution in the FITS file implies that the focal length (or the pixel size) is wrong by factor 4, a warning should be outputted (e.g. Wrong astrometric solution?), and a demand to apply PixInsight's plate solving script with checked parameters (focal length, pixel size, image scale) in order to fix that. This would be the correct behavior in my view.

When I apply PixInsight's plate solving script to this image (search for "Rosette nebula", set the correct the focal length of 442 mm, and start the script), the result is a correct astrometric solution with a focal length of 442.28 mm:

Image Plate Solver script version 5.5.0
===============================================================================
Referentiation matrix (world[ra,dec] = matrix * image[x,y]):
+7.67087647e-05 +6.14639745e-04 -1.19877776e+00
-6.14269243e-04 +7.69008518e-05 +1.39230597e+00
WCS transformation ....... Linear
Projection ............... Gnomonic
Projection origin ........ [2472.149054 1641.843477] px -> [RA: 6 31 48.150 Dec: +4 55 27.61]
Resolution ............... 2.229 arcsec/px
Rotation ................. -97.143 deg
Observation start time ... 2021-03-04 19:42:07 UTC
Observation end time ..... 2021-03-04 19:52:07 UTC
Focal distance ........... 442.28 mm
Pixel size ............... 4.78 um
Field of view ............ 3d 3' 41.4" x 2d 2' 0.9"
Image center ............. RA: 6 31 48.171 Dec: +4 55 27.99
Image bounds:
top-left .............. RA: 6 26 58.815 Dec: +6 18 53.94
top-right ............. RA: 6 28 31.235 Dec: +3 16 47.32
bottom-left ........... RA: 6 35 06.083 Dec: +6 34 05.01
bottom-right .......... RA: 6 36 36.318 Dec: +3 31 54.26

So all means to handle this situation well are already present in PixInsight. It would only require a comparison of the actual FOCALLEN value in the file with the focal length that PixInsight determined from the astrometric solution when the image is opened.

Bernd
 
pfile said:
a warning seems like a good idea
Click to expand...
I agree, but let's just remember this is probably a highly atypical case. I would anticipate that nine times out of ten, if the FOCALLEN keyword disagrees dramatically with an embedded astrometric solution, it will be the (usually manually entered) FOCALLEN key word that is wrong. I've lost count of the number of times I've switched from my Edge 8 (f=2032) to my Esprit 80 (f=400), and forgetten to tell APT, which faithfully keeps writing my specified f=2032 into the FITS headers.
 
fredvanner said:
I agree, but let's just remember this is probably a highly atypical case. I would anticipate that nine times out of ten, if the FOCALLEN keyword disagrees dramatically with an embedded astrometric solution, it will be the (usually manually entered) FOCALLEN key word that is wrong. I've lost count of the number of times I've switched from my Edge 8 (f=2032) to my Esprit 80 (f=400), and forgetten to tell APT, which faithfully keeps writing my specified f=2032 into the FITS headers.
Click to expand...
And what would then be the disadvantage of my proposal?

Bernd
 
If a user (or his chosen application) puts junk in his header, I don't think it is PI's job to sort out the mess (there is a virtually bottomless pit of other possible header errors that it would then seem "reasonable" to try and fix). If we see an inconsistency, by all means note it, but then I think we just make a "reasonable assumption" - which in this atypical case would happen to be wrong; but the fault is in the file, not in PI.
 
This file fault is obscure, and likely to be very rare - if not unique. I have not yet worked out any likely way for it to arise. Somehow, the astrometric solution for a 4x4 binned reduction of the image has been written to the header of the unreduced image. That should not be possible; no robust application interface would allow it. I think it would be a seriously misguided use of valuable effort for the PI team to waste one minute on correcting this obscure error of some other application (never mind developing a solution requiring ad hoc image solving of input files on the basis of broken headers). A warning message on identification of an inconsistent header would, however, be sensible.
 
I did not deny that the fault is in the file, and perhaps my wording (bug in PI) was not appropriate.

My suggestion was to output a warning. Please tell me: what is wrong with it? I prefer a hint which enables the user to recognize that something isn't straight. In my view this is useful, at least for deviations being that large.

Bernd
 
You must log in or register to reply here.
Back
Top