PI doesn't read the correct focal length from the FITS header

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
My suggestion was to output a warning. Please tell me: what is wrong with it?
Nothing.
I think we all agree that a warning message is a good idea:
However, I would agree that a warning (not error) message if a header value is changed by more some percentage could be helpfull
a warning seems like a good idea
If we see an inconsistency, by all means note it
A warning message on identification of an inconsistent header would, however, be sensible.
You don't seem to be facing any argument on this point!
 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,380
467
57
Valencia, Spain
pixinsight.com
OK, OK, gentlemen, let's have peace! :) I'll see how a warning message can be shown on the console when this happens (excessive difference between the original focal length metadata item and the value derived from the astrometric solution).
 
  • Like
Reactions: bulrichl

TEngwall

Member
May 4, 2021
7
0
I'm a new user with Pixinsight and have discovered that I have the same problem of an incorrect focal length for my scope in the Fits header for the Lights (1/4 of the true FL). I gather from the discussion that this has something to do with the ASIAir Pro that I use to control the aquisition of the images. My question is should I edit the FL in the Fits header? If not, what should I do?
 

bulrichl

PTeam Member
Nov 2, 2016
1,265
166
La Palma, Canary Islands
My question is should I edit the FL in the Fits header? If not, what should I do?
No, the focal length in the FITS header of the FITS files is correct, it is the plate solving result that is wrong in the FITS files.

A reasonable solution is: After ImageIntegration, apply PixInsight's Plate Solve Script to the integration result setting the correct focal length value. Thus you'll get a correct plate solving result which can be used for PhotometricColorCalibration.

And please report the plate solve bug to the creators of ASIAir Pro in order that it will be fixed.

Bernd
 

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
should I edit the FL in the Fits header? If not, what should I do?
The problem is not the FOCALLEN keyword in the header (this has the correct value in the file header; it gets changed to the wrong value when it is read into PI).
The problem is that the FITS file header includes an incorrect astrometric solution, which implies a different focal length. Most capture software does not write an astrometric solution to the header, so the question is:
  1. Why does the ASiair write a solution to the header?
  2. Why is it wrong?
  3. What can you do about it?
I think the clue is in the following note in the ASiair manual:
③ The default setting in GoTo will automatically center the image using a short
alignment exposure and plate solve
I suspect that the resulting solution is written (with good but misguided intentions) into the header of subsequent image captures "to document the exact coordinates of the image".
However ... when taking "short alignment exposures" it is common to use maximum binning (typically 4x4). It is just such a 4x4 binned solution that is written to the header. Since this is not a solution for the later unbinned images this is an error.
Editing FITS headers is more complex than it looks. This is because FITS headers are block organised fixed length record structures, not simple text files (so you can't, for example, just delete a few keywords). The easiest solution (as Bernd say above) is to re-solve the image(s) in PI with manually entered image parameters.
Unless there are ASiair set-up options that are not in the user manual I looked at, I can't see any way of stopping the ASiair from writing this data. You really need to report this as a fault to ZWO.
 
  • Like
Reactions: Juan Conejero

TEngwall

Member
May 4, 2021
7
0
No, the focal length in the FITS header of the FITS files is correct, it is the plate solving result that is wrong in the FITS files.

A reasonable solution is: After ImageIntegration, apply PixInsight's Plate Solve Script to the integration result setting the correct focal length value. Thus you'll get a correct plate solving result which can be used for PhotometricColorCalibration.

And please report the plate solve bug to the creators of ASIAir Pro in order that it will be fixed.

Bernd
Thanks for your response. I understand what to do now. I will report this problem to ZWO. However, since I'm not sure that I can adequately aticulate the issue to ZWO, I would apreciate it if you could give me the specific wording and I will send it on to them. Thanks again.
 

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
Firstly I would collect at least three separate examples of the fault. Pick three separate targets and use the ASiair Goto with automatic centering to acquire each target and then capture at least one unbinned frame. Check that these image files have the same fault (i.e. the focal length is adjusted to an incorrect value when loaded into PixInsight).
If you can't reproduce the problem, then you need to explore the possibility that this was a "one-off" error.
If the problem is reproduced, then you need to send the example files to ZWO support with a comment such as:
"The astrometric solutions in the FITS headers of these files captured using the ASiair Astrophotography Controller do not match the images in the files. In particular, the image size is different. These errors cause incompatibility with other software that processes FITS header data correctly. This incompatibility is made worse by the use of non-standard keywords, such as 'IMAGEW' and 'IMAGEH' instead of the standard 'NAXIS1', 'NAXIS2'. All FITS header content should comply with the FITS standard, available at http://fits.gsfc.nasa.gov/iaufwg/."
 
Last edited:
  • Like
Reactions: TEngwall

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
Editing FITS headers is more complex than it looks.
If you are really desparate to edit FITS headers, there is this utility called 'fv'. 'f'v is a FITS file viewer and editor developed at the High Energy Astrophysics Science Archive Research Center (HEASARC) at NASA / GSFC. You could use this to simply delete the ASiair solution from your headers
 
  • Like
Reactions: TEngwall

ricrac50

New member
Sep 18, 2021
2
0
What data in the FITS header should be deleted? Fixing the problem with Plate Solver after integration doesn't provide a solution for when I want to use the NormalizeScaleGradient script. I tried to use ImageContainer and FitsHeader to change the FOCALLEN item, but after viewing this thread I see why that didn't work.
 

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
You need to delete the astrometric solution. This starts with the keyword "CTYPE1" and ends with the keyword "IMAGEH". The examples posted have included a second order SIP distortion model (Simple Imageing Polynomial - the "A...", "B...", "AP..." and "BP..." keywords), resulting in a total of 40 keywords to delete.
You can't do this deletion in PixInsight, since PI "corrects" the file as soon as it reads it. You will need to correct it in some other utility ("fv" - see post #28 above - can do it, but its cumbersome. it would be nice to find a batch utility to process all your files at once, but I'm not sure that any PI script using the standard PI FITS IO will ever be able to see the original, unmodified FITS header).
The only real, proper, long-term answer is to get ZWO to stop putting this totally useless invalid rubbish in the header in the first place.
 
  • Like
Reactions: arrowspace90

ricrac50

New member
Sep 18, 2021
2
0
Thanks! I just might have to dust off my PERL skills and see what I can do. 7 or 8 years ago I could have taken care of this problem in 30 minutes! I'll take a look on the ZWO forum and see if this problem has been brought to their attention.
I'm assuming anything PI processes that rely on photometry won't work with this crud in the files?
 

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
I'm assuming anything PI processes that rely on photometry
Its almost worse than that: because PI "trusts" the astrometric solution, it "corrects" the focal length, and "tidies up" the ZWO solution for PI compatibility (PI doesn't use SIP distortion models). (Actually, it's not photometry that is the problem, it is astrometry - but PCC uses an astrometric solution to search the photometric catalogue, so it indirectly messes up the photometry as well).
 

arrowspace90

Active member
Nov 12, 2020
25
1
You need to delete the astrometric solution. This starts with the keyword "CTYPE1" and ends with the keyword "IMAGEH". The examples posted have included a second order SIP distortion model (Simple Imageing Polynomial - the "A...", "B...", "AP..." and "BP..." keywords), resulting in a total of 40 keywords to delete.
You can't do this deletion in PixInsight, since PI "corrects" the file as soon as it reads it. You will need to correct it in some other utility ("fv" - see post #28 above - can do it, but its cumbersome. it would be nice to find a batch utility to process all your files at once, but I'm not sure that any PI script using the standard PI FITS IO will ever be able to see the original, unmodified FITS header).
The only real, proper, long-term answer is to get ZWO to stop putting this totally useless invalid rubbish in the header in the first place.
I used image solver and it worked with no error message. But then in the next step, when I went to crop the image, it warned me "astrometic solution will be lost".
When I used StarNet, all the warning messages appear again, more of them, and StarNet leaves big star artifacts in the image, artifacts that don't seem to occur in other images I see people do.
 

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
it warned me "astrometic solution will be lost"
This will happen whenever you perform an operation (like cropping) that changes the geometry of the image (since the solution contains information such as the RA / DEC of the centre of the image, which will change when you crop). You just have to solve again.
StarNet leaves big star artifacts in the image
I'm afraid StarNet does that, particularly for larger stars. It is very dependent on properties of the original image, particularly the typical size (in pixels) of stars. StarNet is quite good at removing small stars, but generally leaves ghostly "halos" in the place of larger stars. If you are patient and build multiple star masks carefully, traditional star masking methods are much more robust at removing stars than StarNet - which is a rather crude "one size fits all" method.
 

mmcaulifffe

New member
Sep 27, 2021
4
1
Is there a work around for this for someone (me)who isn't exactly a computer genius? This issue just popped up for me this morning after spending literally all night collecting data on M31.
New to PI, I just "stacked" and edited data from May using PI and an Adam Block tutorial last week acquired from the same ZWO device I used last night with no issues. That said, interestingly enough, I went back to that older data today and suddenly this issue has affected it as well.
Another interesting thing, my flats & dark flats, the focal length is correct.
 

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
This issue just popped up for me
I presume that you mean the original issue in this thread (PI changing the focal length in the header). This only happens when an ASIAir device is used for capture. I believe that it is caused by the ASIAir "Auto-centre" "goto" capability. If you turn this off (and centre your target manually) I think this problem will probably not happen. Since it is caused by a fault in the ASIAir app, there is really nothing much PI can do about it.
 
  • Like
Reactions: arrowspace90

fredvanner

Well-known member
Apr 17, 2019
2,048
328
71
Wells, Somerset, UK
An enthusiastic ASIAir user could provide useful information on this. Does the astrometric solution only get written to the header after an "auto-centre" "goto" (if this is not the case, I don't know where the solution comes from)? If so, what actions "clear" the solution (i.e. stop it being written to the header) - for example is it cleared by a small manual "nudge"? (it surely should be)
 

mmcaulifffe

New member
Sep 27, 2021
4
1
I presume that you mean the original issue in this thread (PI changing the focal length in the header). This only happens when an ASIAir device is used for capture. I believe that it is caused by the ASIAir "Auto-centre" "goto" capability. If you turn this off (and centre your target manually) I think this problem will probably not happen. Since it is caused by a fault in the ASIAir app, there is really nothing much PI can do about it.
Hi Fred,

Thank you for the reply & yes, this was in regards to the OP.

I hope your right! If so, that would be a pretty easy work around until ZWO (hopefully) address's this problem. I'm definitely going to test your idea out next time out and will definitely post results, hopefully it's not another 4 months before another good night LOL
 
  • Like
Reactions: fredvanner

mmcaulifffe

New member
Sep 27, 2021
4
1
An enthusiastic ASIAir user could provide useful information on this. Does the astrometric solution only get written to the header after an "auto-centre" "goto" (if this is not the case, I don't know where the solution comes from)? If so, what actions "clear" the solution (i.e. stop it being written to the header) - for example is it cleared by a small manual "nudge"? (it surely should be)
Second thoughts...

You may very well be onto something fredvanner, I failed to mention it here on this forum but the PI FitsHeader information for my flats & dark flats was fine, stating the obvious, don't use goto for those lol and I shot those immediately after my my all nighter imaging M31
 

mmcaulifffe

New member
Sep 27, 2021
4
1
You need to delete the astrometric solution. This starts with the keyword "CTYPE1" and ends with the keyword "IMAGEH". The examples posted have included a second order SIP distortion model (Simple Imageing Polynomial - the "A...", "B...", "AP..." and "BP..." keywords), resulting in a total of 40 keywords to delete.
You can't do this deletion in PixInsight, since PI "corrects" the file as soon as it reads it. You will need to correct it in some other utility ("fv" - see post #28 above - can do it, but its cumbersome. it would be nice to find a batch utility to process all your files at once, but I'm not sure that any PI script using the standard PI FITS IO will ever be able to see the original, unmodified FITS header).
The only real, proper, long-term answer is to get ZWO to stop putting this totally useless invalid rubbish in the header in the first place.
So I downloaded the "fv" utility and after a few tries I figured out what needed to be deleted to get the PI FitsHeader to show the FL I wanted.
It took awhile with 110 images but after a late night I "stacked" in WWBP, PCC, ABE & DBE saved off the project in the wee hours and will pick up the editing shortly. So far, everything is looking and behaving as expected.

Aside from the time it takes to edit all those lights, is there a down side I'm not seeing?