Fits header values after image solver script

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
Actually, now I'm sorted out a bit, it looks as though spyder (installed with anaconda) may be more flexible for poking about - I just need to learn how to drive it!
 

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
OK. It seems that the FITS header keywords "PV1_1" and "PV1_2" are causing the problem with astropy finding the correct coordinates. If you just delete them astropy works out the correct values. A PI image loaded in the "top-down" orientation, solved in PI, and saved to FITS with these keywords deleted or commented out, has the correct image coordinates (i.e. ther correct bounding box coordinates) in astropy - but with the actual image flipped. You can fix this by overwriting with the flipped image in PI (using PixelMath) - so you can export a FITS file that is correct in astropy, but the WCS solution will then be "upside down" in PI. I'm afraid there is no file layout that is compatible with both.
 

gnewell

Well-known member
Aug 19, 2018
60
2
The vertical flip is still confusing me, because I have to rotate things so north is up AND have things come out right side up.

How much to rotate in what direction when in the process and when in the process to do the vertical flip o_O
 

gnewell

Well-known member
Aug 19, 2018
60
2
Well bottom line I got this image into Stellarium and it lines up with the planetarium stars:
My image (only)
1593907969248.png
Planetarium Stars (only)
1593908000290.png

Sure hope I don't have to go through this again because it is not (yet) an automobile process.

I plan to follow up here with a short video on why I'm doing this (besides just putting my club's images in Stellarium).
 

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
I'm not quite sure what you want. Here is your original image rotated to "N up" orientation:
View attachment Wild_Duck_Cluster_GCN_SJAA_N_up.jpg
I did that by solving the original image (in PI), then reading the CROTA1 keyword from the FITS header, and cut&paste it into the "rotation" process angle, with "clockwise" selected. I then re-solved the rotated image, and added annotation for the image above. If I then delete the PV keywords, convert to greyscale, overwrite with flipped image and export to FITS, I get this in astropy:
1593913464992.png
I'm not quite sure at what point (if any) you want to crop the image.
 

gnewell

Well-known member
Aug 19, 2018
60
2
Thanks.

As I said I got it into Stellarium and lined up so this particular instance is "solved" ;)

It does feel like there is still some work for someone to make astropy and PI saved fits work better together.

If you want to know more about what I am doing with Stellarium there are my scripts and documentation in this google drive folder:

https://drive.google.com/drive/folders/1oO4i83cSjbxuZrYDZTyBXRr8bSYQDrvr?usp=sharing

Those scripts are more updated than the ones currently available via Stellarium production version.

but the short version is:

1) Solve an image to determine its rotation and parity​
2) Make it positive parity if it's not​
3) Rotate the image so north is up (zero degrees rotation) and fill with black​
4) Shrink the image so it's long edge is 1024 (or 512, or 256) pixels​
5) Pad the short edge it is a multiple of 256 pixels and fill with black​
6) Solve the new image, to determine the coordinates of each corner of the image, in the World Coordinate System.​
7) Output the meta data in a format suitable for Stellarium's textures.json file​

Also I'm uploading a demo video as we speak, and will link it here when ready. It demonstrates using your own or club's images in Stellarium, for public outreach, and shows a small part of what is possible in scripting Stellarium.
 

pfile

PTeam Member
Nov 23, 2009
5,311
64
well bob franke does read this forum. which makes me wonder what that readme actually means. PI uses the range [0,1] for f32 fits files. for 16-bit fits files the range [0,65535] would be correct, but not for f32.

rob
 

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
... "ImageSolver introduces a vertical flip". This is not really true. To manage this issue it is important to understand what is really happening.
Internally, PI consistently adopts a "(0,0) at top left" orientation, with the y-axis increasing downwards - reasonably described as "top down" format. It is this orientation of the image that is always solved by ImageSolver.
PI has the option to configure FITS file input to accept images in the widely used "bottom up" (y-axis increasing upwards) format. On input, these images are "flipped" to the PI conventional orientation.
PI does not have an option to flip FITS file output orientation.
Thus, the workflow [export FITS file from "bottom up" app; read "bottom up" FITS file into PI; ImageSolve; write solved image to FITS; import into "bottom up" app] produces an output file with a "flipped solution" - but it's really the image that wasn't flipped. As I describe elsewhere in this thread, this can be "bodged" by overwriting with the flipped image before FITS export, but the tidy solution would really be to use the FITS format "bottom up / top down" status (I refuse to use the absurd and incomprehensible "up bottom" label) to automatically flip images on FITS export.
 

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
I have realised a possible confusion here. If you use the "vertical mirror" function to flip an image in PI, you change the parity of the image (it truely becomes a mirror image). The "flip" to convert a "bottom up" FITS input file to a PI internal "top down" format is not a parity change - it is a conversion between two different descriptions of the same parity image. But FITS input with "bottom up" selected, followed by FITS output (withou any flip) does result in a change of parity.
 

Juan Conejero

PTeam Member
Sep 2, 2004
7,402
67
56
Valencia, Spain
pixinsight.com
The coordinate origin setting in FITS Preferences works for both reading and writing FITS files. I has worked this way at least since 2007. In addition, you can specify the 'bottom-up' output format hint to force writing FITS files with the 'professional' orientation, i.e. with the origin of coordinates at the bottom left corner, overriding FITS Preferences settings. You can use this hint e.g. with the BatchFormatConversion script. So this problem is actually immaterial.

The actual issue here is, as always, FITS. Obsolete, loosely defined, inefficient, eternal source of interoperability problems.
 

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
What you say about FITS is undoubtedly correct, however ... the world is full of legacy FITS-dependent applications, and this thread is essentially about compatibility with such applications. This is fire-fighting a problem that we would all love to go away, but until it does we must fight the fires.
My normal workflow usually has no problems with FITS. I use APT to capture images (in FITS format - usual bottom-up orientation), which I import into PI (using the default FITS settings), at which point I forget about FITS, and everything works fine.
However, this thread is about compatibility of astrometric solutions from nova.astrometry.net, and the export of astrometry solutions from PI to other FITS-based applications. The fact that PI astrometric solutions exported to FITS files end up "inverted", and that this could be rectified by overwriting with the inverted image before export led me to assume that the exported image was not matching the FITS input setting. I have now verified that, as you rightly note above, this is not so (FITS import and export match the selected y-axis orientation). I now understand (at last) that the problem is more subtle (there are probably many readers who understood this already - we all have to learn).
The problem is that the FITS header description of the WCS solution embeds an assuption abot the (x,y) axis orientation. The (x,y) reference pixel (CRPIX1, CRPIX2), the y-axis scale (CDELT2) and the scaling matrix all assume a specific y-axis orientation. When a bottom-up FITS image with a WCS solution in the header is imported, the image is flipped, but the WCS solution keywords still refer to the bottom-up-indexed image. Similarly, when a PI image with a WCS solution is exported to FITS with the bottom-up orientation selected, the image is flipped, but the WCS solution keywords now reference the PI top-down indexed image. So the imported image has an "inverted" solution in PI, and the exported image has an "inverted" solution in other FITS-based apps. Understanding this, it is possible, by vertically mirroring the image at the appropriate points in the workflow, to generate legacy FITS-compatible WCS solutions. Care is needed to avoid an unintended change of image parity.
 

gnewell

Well-known member
Aug 19, 2018
60
2
@fredvanner, I think what would help is a more exacting description of what it takes to make fits images with PI generated solve wcs data work in other programs (including the pixel math).

@Jaun, It seems to me that Fred has identified that certain WCS header data is incorrect, or a at least not properly matching the image data, when images are solved in PI and then exported as fits. Do you agree and if so is there a plan to correct?

Using xisf is not an option, in this case where all external programs are expecting fits (astropy in this case).
 

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
If your objective is to export an existing PI image as a (bottom up) FITS file with a WCS solution in the header that is compatible with other FITS-aware applications, I believe the following workflow will do it.
Clone the image; vertically mirror the clone; ImageSolve the (flipped) clone; use PixelMath to overwrite the image data of the clone with the original image; save as FITS (making sure that "bottom up" is selected). This worked for me.
 

fredvanner

Well-known member
Apr 17, 2019
87
4
Wells, Somerset, UK
It is also worth noting that if you use ImageSolver distortion correction this is not (and can't be) defined in the FITS header. The header is written with a linear solution. This is fine for most normal narrow field images. Solutions from nova.astrometry.net often include SIP distortion coefficients, but PI distortion correction is much more accurate (which you need for things like wide field mosaics). Since there is no defined FITS format to describe PI distortion correction (and no other application would know what do do with it if you did include it), this is really not a FITS header issue.