Warning While Doing ImageIntegration?

[GeneralT001]

Keep getting this message:

Anyone know why?

[Juan Conejero]

This is a known bug:

https://pixinsight.com/forum/index.php?topic=13632.0

I have already fixed it, and right now am working on a new set of updates, which I hope will be available today. If coordinates are important for your work, you'll have to repeat the integration once I release these updates. Sorry for the inconvenience.

[Don]

The problem with the RA keyword is gone following today's update, but I'm still seeing a similar problem with the SITELAT and SITELONG keywords - screen capture attached.

[Juan Conejero]

This is a completely different problem:

34d4m30.000s N
84d34m40.000s W

These are invalid values for the SITELAT and SITELONG keywords. See the SBIG proposal for FITS keyword extensions document. The expected format for these nonstandard keywords is the same as OBJCTDEC, that is, 'DDD MM SS.sss'. Another example of the endless universe of FITS interoperability problems.

[Don]

Hi Juan,

The data files were acquired with Sequence Generator Pro (SGP), and that is where the FITS keywords were generated and added to the files.  The format of the SITELAT and SITELONG keywords generated by SGP has not changed recently.  These errors did not occur until the recent update to PixInsight version 1.08.06.1473, and today's update to 1.08.06.1474 still produces the error.  Data files that processed without error in previous versions of PixInsight now produce these errors.  So obviously something changed in PI 1.08.06.1473.  Apparently you were not trying to parse these keyword values as numbers in ImageIntegration previously - in data files calibrated last month with the previous PI version, the SITELAT and SITELONG keywords were passed into the calibrated files as strings, in the same format that they are in now, but are not present in the integrated files.  In 1.08.06.1473, SITELAT and SITELONG are still passed to the calibrated files as strings, and subsequent processing maintains them as strings until ImageIntegration fails while trying to parse them as numbers.

I'm easy - I don't need the SITELAT and SITELONG data, so to me the error is inconsequential, since ImageIntegration continues and produces the desired output file.  The user provides the latitude and longitude for each site in SGP, but cannot control the format of the FITS keywords.  If you are planning to use these values in future PI versions, some kind of resolution will need to occur.  If not, it is probably better to stop trying to parse them in ImageIntegration, but just pass them as strings if at all.

I will submit a defect report to SGP, citing the document you reference here. 

Don

[Juan Conejero]

Hi Don,

Starting from version 1.8.6.1473, PixInsight uses geodetic observer coordinates when they are available in image metadata. These coordinates are necessary for solar system ephemeris calculations, which are now performed by the AnnotateImage script to locate planets and asteroids on images. More ephemeris-based tools and scripts will be developed in future versions.

So we have to parse and evaluate the SITELONG and SITELAT keywords, which should have valid values. I am sure the SGP developers will fix this problem as soon as possible.

[kor]

Juan,

I completely understand what you are doing here with regards to standardization.  And SGP is modifying their application to be consistent, so the problem is solved for data acquired going forward.

But, I now have years of data that I cannot process with the current version of PixInsight (as my techniques and PI tools get better).  So can you reconsider and give us some workaround that doesn't involve trying to maintain 2 installations of PixInsight?  Maybe an "Input Hints" keyword that converts the value when a file with the old SGP format is loaded?

- Shane

[tchitty]

This is also a problem when using TheSkyX for image capture.

[AccidentalAstronomer]

>This is also a problem when using TheSkyX for image capture.

TheSkyX follows the original SBIG convention as well. Here's an example from a random fits off my computer:

SITELAT       '+27 29 45.00'
SITELONG   '+80 53 03.00'

I see this document quoted (the original SBIG extensions to FITS), often incompletely, and it is getting harder and harder to find online, so I've attached a copy for anyone's reference. The MaximDL document that claims to be compliant actually has the format slightly wrong too. It is actually:
'SDD MM SS.SSS'

Richard

[pfile]

Juan,

I completely understand what you are doing here with regards to standardization.  And SGP is modifying their application to be consistent, so the problem is solved for data acquired going forward.

But, I now have years of data that I cannot process with the current version of PixInsight (as my techniques and PI tools get better).  So can you reconsider and give us some workaround that doesn't involve trying to maintain 2 installations of PixInsight?  Maybe an "Input Hints" keyword that converts the value when a file with the old SGP format is loaded?

- Shane

+1 on this. while i suppose one could write a fits sanitizer to fix this, i have 1000s of frames taken with SGP and honestly i'd rather not touch them. i consider all of my subs to be read-only, and don't want to duplicate them all to fix this fits header problem.

rob

[Juan Conejero]

Hi Shane,

...I now have years of data that I cannot process with the current version of PixInsight

Please note that these errors are non-fatal ones, that is, you can integrate your images even if the SITELONG and SITELAT keywords are invalid. The only issue is that these keywords won't be generated in the final integrated image. Other than this, ImageIntegration will work without problems.

[Juan Conejero]

Maybe an "Input Hints" keyword that converts the value when a file with the old SGP format is loaded?

+1 on this. while i suppose one could write a fits sanitizer to fix this, i have 1000s of frames taken with SGP and honestly i'd rather not touch them. i consider all of my subs to be read-only, and don't want to duplicate them all to fix this fits header problem.

I am very reluctant to implement special adaptations of our code base to solve interoperability problems that we have not generated, unless they are completely unavoidable. An input hint would be doable. However, given that this is a non-fatal error that does not prevent you from using ImageIntegration, I'll have to think about this before making a decision. On the other hand, given that the authors of the software in question have apparently no interest in supporting XISF, which we have created precisely to solve these problems among many others, I have little enthusiasm for this.

[kor]

OK, Juan, I understand.  Someone had posted somewhere (I can't find it now), that ImageIntegration still worked but drizzle didn't, so I got worried.  I will test this myself this weekend.  I should know my now not to take everything that I read (even on high SNR forums) at face value!

As Rob mentioned, it would be rather easy to make a translator--I've used pyfits to correct flats back when I was using an OSC camera.  I will do so if I end up having a problem with old data.  Of course my current project will end up being a mixture of old and new ...

Thanks for your reply and the explanation with it.

One question: Since I use CatlogStarGenerator to make a synthetic reference frame for registration, would I even see this issue?  Does that synthectic frame contain whatever information ImageIntegration needs?

Another question: Since my subs are typically taken over several months from several different locations (up to 700 miles apart if there is a good star party happening), can ImageIntegration decide what to use for the values even when they are in the correct format?

- Shane

[pfile]

Maybe an "Input Hints" keyword that converts the value when a file with the old SGP format is loaded?

+1 on this. while i suppose one could write a fits sanitizer to fix this, i have 1000s of frames taken with SGP and honestly i'd rather not touch them. i consider all of my subs to be read-only, and don't want to duplicate them all to fix this fits header problem.

I am very reluctant to implement special adaptations of our code base to solve interoperability problems that we have not generated, unless they are completely unavoidable. An input hint would be doable. However, given that this is a non-fatal error that does not prevent you from using ImageIntegration, I'll have to think about this before making a decision. On the other hand, given that the authors of the software in question have apparently no interest in supporting XISF, which we have created precisely to solve these problems among many others, I have little enthusiasm for this.

i wouldn't read too much into that - the SGP devs have been in "maintenance mode" for a very long time now. i think given that i16 works well enough for what they are doing, as an SGP customer it would be better (for me, anyway) if they put their effort into something other than file type support.

but if indeed Drizzle still works without this then i suppose all is well, i was under the impression that drizzle would be broken against these wrong FITS files.

rob

[Juan Conejero]

Nothing important has changed in DrizzleIntegration in the latest versions, so this issue should have no repercussion at all. If DI does not work for some reason, then we have a bug that I have not detected in my tests, so please check and let me know.