Fits header values after image solver script

gnewell

Well-known member
Aug 19, 2018
61
2
It seems to me that the WCS Fits header values are not correct, after running the image solver script.

The result is that when I use WCS coordinates from a PI solved image it is in the wrong part of the sky.

I arrive at this by comparing the headers as produce by solving in PI vs. nova.astrometry.net.

An example image:


nova.astrometry.net gives these header values:



Nova.png

Now PI (note that I added the IMAGEW and IMAGEH attributes, after solving):
PI Headers.png

The difference is the sign of the CD1_2 and CD2_2 values.

?
 

pfile

PTeam Member
Nov 23, 2009
5,603
102
when you say "wrong part of the sky" do you mean that something is annotating the image improperly? because the RA and Dec in both headers are just about the same and seem correct for M92.

the scale matrix, if i understand it right, is meant to represent rotation of the image, so even if that is wrong it should still be centered in the same place. unless the difference in the reference pixel location ends up translating the image somehow. what's confusing to me is that the CDELT1 and CDELT2 keywords are used to represent the plate scale and astrometry.net has given a negative number for one of them.


rob
 

gnewell

Well-known member
Aug 19, 2018
61
2
I am using astropy to tell me the corners of the image in the World Coordinate System (WCS).

This is how you place images into the "sky" in Stellarium. With those numbers having opposite signs from the nova solution, the image gets placed in the wrong part of the sky (doesn't use the RA & Dec for image center).

Astropy is looking at that transformation matrix to calculate the coordinates in WCS.

For example, using the nova solution:

Number of WCS axes: 2
CTYPE : 'RA---TAN-SIP' 'DEC--TAN-SIP'
CRVAL : 259.299166924 43.248859047
CRPIX : 2082.68441772 740.475479126
CD1_1 CD1_2 : 0.000107342735222 3.29288988632e-05
CD2_1 CD2_2 : 3.28786572255e-05 -0.000107471505131
NAXIS : 0 0
Image Width x Height: 3318 x 3318
scale: 0.404403 arcsec/pix
Image Parity: -1
Image Orientation: 21.3 East of North


world coordinates of image corners for Stellarium:

"worldCoords" : [[[259.1099, 42.9034], [259.5965, 43.0123], [259.4481, 43.3689], [258.9589, 43.2595]]],

and using the PI solution:

Number of WCS axes: 2
CTYPE : 'RA---TAN' 'DEC--TAN'
CRVAL : 259.278319415597 43.1361876278143
CRPIX : 1659.50407483 1659.54439405
CD1_1 CD1_2 : 0.0001073522347307 -3.29197974242e-05
CD2_1 CD2_2 : 3.28927209613e-05 0.0001074223458549
NAXIS : 3318 3318
Image Width x Height: 3318.0 x 3318.0
Image Parity: 1
Image Orientation: 197.0 East of North


world coordinates of image corners for Stellarium:

"worldCoords" : [[[79.6178, 46.7397], [79.0986, 46.6310], [78.9373, 46.9869], [79.4597, 47.0964]]],

I guess I should go back research to see what the difference between CTYPE : 'RA---TAN-SIP' 'DEC--TAN-SIP' and CTYPE : 'RA---TAN' 'DEC--TAN' is, but I'm depending on astropy to get those details correct.

I have already updated another script with different image parity code, so I will do that in the attached as well, but it shouldn't make any difference in wcs output (but I will check presently).
 

Attachments

gnewell

Well-known member
Aug 19, 2018
61
2
Even after "fixing" all the signs, there is still something else wrong.

I can see how those signs could be flipped anyway, given the negative CDELT2 value in the nova solution.

Man I have been messing with this one image all day! If it would just solve on nova I'd be fine.
 

fredvanner

Well-known member
Apr 17, 2019
201
30
Wells, Somerset, UK
I note the data from nova implies a different (4164 x 1480) image crop, but the key solution parameter are near enough the same.
Provided there are no SIP coefficient in the header (things like "A_1_1", "B_1_2" etc.) then the "-SIP" in the CTYPE should have no effect - and you can safely remove it. I think the parity is equivalent to the difference in the signs in the scalin matrix, and is more or less equivalant to whether pixel(x=0, y=0) is at the top or bottom of the image (FITS is ambiguous about this, so most solvers try both, and encode the answer in the scaling matrix and the parity).
The question is why Astropy has given those odd solutions for the image corners (it looks like it might be RA => RA-180; Dec => 90-Dec). As I understand it, if (u,v) is an (x,y) pixel offset from the centr of the image, then the correponding (RA, Dec) offset (on degrees) is given by:
CD1_1​
 
  • Like
Reactions: pfile

Andres.Pozo

PTeam Member
Jul 7, 2008
939
3
Madrid
This problem is probably caused because some applications store the FITS top-bottom and others bottom-top. With PI you can chose how do you want.
This is one of the (many) reasons Juan has deprecated the FITS format.
 
  • Like
Reactions: Pleiades

fredvanner

Well-known member
Apr 17, 2019
201
30
Wells, Somerset, UK
(bother - it looks like "tab" closes the post ... I'll continue here) this is an attempt to display a matrix multiplication!
(CD1_1 CD1_2) (u) = (RA)​
(CD2_1 CD2_2) (v) = (Dec)​
Given the coefficients of the scaling matrix, this can't get very far from the (RA, Dec) of the image centre - so Astropy is doing something odd - probably because some keyword somewhere is telling it to.
 
  • Like
Reactions: pfile

gnewell

Well-known member
Aug 19, 2018
61
2
I think parity = 1 should just mean that the image doesn't need to be "flipped" in order to map on the sky.

How would I tell PI which fits (top-bottom / bottom-top) convention to use such that the solve agrees with Nova?

We can ask astropy to do the mapping to the sky:

fits returned from nova:
1593630162275.png

same source image solved in PI (no change to fits headers):
1593631184534.png

so astropy is plotting the image differently for solves from PI and Nova. The plot code is here (I just added plt.show as the last line):


For the PI case PI reported the image corners in the process console:

Image center ............. RA: 17 17 06.804 Dec: +43 08 10.20

Image bounds:
top-left .............. RA: 17 15 50.119 Dec: +43 15 33.67
top-right ............. RA: 17 17 47.570 Dec: +43 22 07.70
bottom-left ........... RA: 17 16 26.346 Dec: +42 54 11.80
bottom-right .......... RA: 17 18 23.179 Dec: +43 00 43.54

Is there something I can do to make them agree, noting that the nova case results in correct mapping on the sky, using astropy?
 

Attachments

fredvanner

Well-known member
Apr 17, 2019
201
30
Wells, Somerset, UK
I have had problems like this. One diagnostic is to take a solved FITS file from nova and load it into PI, then use the "Render">"AnnotateImage" script to annotate the image. You will probably find the annotation is "upside down". What to do next depends on what you are trying to achieve. If your objective is to have an image in PI that matches the nova solution I do the following:
  • clone the solved image
  • use the "vertical mirror" button on the toolbar to flip the clone
  • use PixelMath to overwrite the original image with the flipped clone.
You now have an image that matches the nova solution (prove it by annotating) - but it is mirrored. If you don't want it mirrored, then:
  • generate a flipped version of the image, and export to FITS
  • solve in nova
  • reimport and process as above.
I used to do this (mainly because my soft summer-night images simply wouldn't solve with ImageSolver, but would - sometimes - solve with nova); the latest update to ImageSolver seems much more robust and capable, so now I rarely use nova.
 

fredvanner

Well-known member
Apr 17, 2019
201
30
Wells, Somerset, UK
To anticipate a possible confusion:
Andres.Pozo noted above that you can select which FITS parity to assume when loading a FITS file. Unfortunately, there is no similar option when saving a FITS file. I have not found a way to avoid this problem if you want a workflow that processes an image in PI, exports it as FITS for solution in nova, and then reimports it for further processing.
 

gnewell

Well-known member
Aug 19, 2018
61
2
To zoom up to the reason for all of this:

I Wrote a workflow for placing your own (or other's) astroimages in to the planetarium program Stellarium, so that when you zoom into a DSO in the planetarium you see your artwork, vs. the default, or lack of an image that ships with Stellarium (and all the stars line up).

FYI, that workflow is described in chapter 8 of the Stellarium documentation.

That workflow uses astropy to solve images via nova.

But now, I have one image that simply refuses to solve on nova, M11, despite manipulating the levels etc. I also had trouble getting to solve in PI, until I switched from Polygons to Triangle Similarity (why would an open cluster be "hard" to solve when globs solve just fine?).

Anyway my hope was to use PI to generate the data such that I could insert it back into my workflow to get the corners of the image in WCS coordinates.

Since the workflow works with finished astroimages ("un-stretching" them by raising the black point for solving) it doesn't really use fits files for anything. Usually you have a jpeg, and stellarium wants a PNG (where up = North and the sides are wither 512, 1024, or 2048 pixels).

First it creates a jpg version of the image with the black point raised so hopefully only stars are visible, then this is solved to figure out the angle on the sky of the original image (I've noticed that nova is more successful solving jpegs than pngs of the same image).

Next the image is rotated so up is north, and the edges filled in with black. Then it resizes and crops the image so the sides are multiples of 512 and saves the image as a png.

Lastly, it ("un-stretches" and) solves that final rotated, resized, and cropped image, to find the corners of the image in WCS coordinates, and prints them in a JSON format for inclusion in Stellearium's textures.json file of nebula artwork.

So, again, I don't really need fits file format an any point. It's just a format that astropy understands with fits headers. I do see a post here:


that suggests xisf support was being added to astropy, but it doesn't look like that ever happened?

I guess the other thing that would be helpful would be an API to call the PI solver from astropy, avoiding file formats altogether.

But in any case given that the source images in question are jpegs and not fits, can we agree that the PI solve data is incorrect (or is at least incompatible with astropy because PI assumes a different standard for writing fits files than astropy expects?)?

and/or is there a solution to modify the image before solving in PI and is it flipping both horizontally and vertically (I will experiment more I guess)?
 

fredvanner

Well-known member
Apr 17, 2019
201
30
Wells, Somerset, UK
The underlying cause of thre problem is the choice of whether (0,0) is at the top or the bottom of the image, so it is only vertical flips that are involved. If (as it seems) astropy is the opposite of PI then a solution based on the process I describe above will probably work (something like read image file; flip; PI solve; overwrite image with original; export in some format that carries the solution header).
 

gnewell

Well-known member
Aug 19, 2018
61
2
Flipping the image (H and V) prior to solving in PI doesn't solve the problem (posted before I read the immediately above post).
 

pfile

PTeam Member
Nov 23, 2009
5,603
102
glenn, do you have a linear version of the file? i think astrometry.net can directly handle linear files and might actually do better on the star extraction phase. how does the job look on nova? i think you can see the star extraction image (with the green circles) to suss out if the stretched version is problematic somehow.
 

pfile

PTeam Member
Nov 23, 2009
5,603
102
Flipping the image (H and V) prior to solving in PI doesn't solve the problem.
what he is saying is to take the *solved* file that's rendering upside-down in stellarium, clone it, flip it, and then copy the fits header to the flipped image and save it. that should solve the problem.

edit: well, OK he's saying to do the equivalent of that by overwriting the image with a flipped version and preserving the fits header. i guess that doesn't invalidate the WCS solution the way some other operations do.
 

gnewell

Well-known member
Aug 19, 2018
61
2
Crap, I cropped in Photoshop, so I can't duplicate exactly with the linear. Or maybe there's a way to use dynamic crop with overlayed windows in PI?

Anyway, I will send the linear version to nova, however, just to see if it solves.
 

gnewell

Well-known member
Aug 19, 2018
61
2
FYI the linear version failed on Nova. Can't say I'm surprised. This seems to be a real corner case for some unknown reason.
 

pfile

PTeam Member
Nov 23, 2009
5,603
102
can you provide scale hints to nova at all? i know with the command-line version of astrometry.net the things that always helped me were a scale hint and downscaling the image by 2. a center hint might help as well.