Cosmetic Correction now fails in WBPP 4.5.3

jeffweiss9

jeffweiss9

Well-known member
Running WBPP 4.5.3, I find that the important Cosmetic Correction now fails for all frames/filters with the error reported:

Cosmetic correction has no cached data for key 22bfc68606e6ee206b08e3ae9e7c798503f09b5c
Operation queue error: Error: CosmeticCorrection: Cannot execute instance in the global context.
Reason: Nothing to do. Choose appropriate method for locating defective pixels.

The full CosmeticCorrection section of the ProcessLog is attached for the first filter. Apparently it is expecting more information than just checking the AutoDetect box in my Cosmetic Correction icon which used to work (before 4.5.0).

That's the main issue, but also WBPP 4.5.3 just continues after CC failed for all frames/group/filters. For me continuing without CC is a waste of time, so is there a way to require each step in the WBPP Process Monitor to succeed or otherwise immediately terminate?

Thanks,
Jeff
 

Attachments

  • CosmeticCorrection.txt
    2.6 KB · Views: 59
Last edited:
Hi @jeffweiss9,

The problem is that CC is not properly configured; you've enabled the "Use Auto Detect" checkbox without specifying either "Hot sigma" or "Cold Sigma" so CC has not any correction method defined to apply.

You can also verify this by opening your CC icon, adding files, and running it globally. An alert window will appear, showing the same error.

Once CC is properly configured, it will run flawlessly.
 
Unfortunately, WBPP 4.5.3 failed on Plate Solving step, something that never occurred in years of using WBPP.
Log file attached, but it is as if it didn't read FITS header parameters in the data.
Jeff
 

Attachments

  • 20220920040716log.txt
    843.8 KB · Views: 45
Yes, the Image Solver wasn't able to align the image to the region of sky around the image center stored in the fits header. Check this out:

[2022-09-20 04:20:48] Reading image:
[2022-09-20 04:20:48] D:/DSW Data/TOA150 2022 Q1/NGC 3324 (Complete)/Batch1/cosmetized/Light_BIN-1_4500x3600_EXPOSURE-600.00s_FILTER-R_mono/NGC 3324 R -25 600 BIN1 PA000 E 20220129 030958_c_1_cc.xisf
[2022-09-20 04:20:48] Loading image: w=4500 h=3600 n=1 Gray Float32
[2022-09-20 04:20:48] 115 FITS keyword(s) extracted
[2022-09-20 04:20:48] *** Error: No database files have been selected.
[2022-09-20 04:20:48] * Using an automatically calculated limit magnitude of 13.62.
[2022-09-20 04:20:48] * Using the automatically selected GaiaDR2 catalog.
[2022-09-20 04:20:48] Seed parameters for plate solving:
[2022-09-20 04:20:48] Center coordinates: RA = 10 38 27.000, Dec = -58 31 47.00
[2022-09-20 04:20:48] Resolution: 1.135 as/px
[2022-09-20 04:20:48] Starting StarAlignment iteration

These are the info read from the file. Are the values correct? Usually, the coordinates are too far from the image's center, or the sampling is wrong.

You may also try to Platesolve that image using the ImageSolver script that shares the same solver engine; properly tuning the parameters may find where the issue is.
 
You are right again as it turned out that there were several images for most filters, starting with Red, which were obviously off-target when you blinked them and those obviously would never plate solve successfully with the PI ImageSolver script or other method. I checked that both those off-target images and the remaining on-target images all show what I think are the proper values in the FITS header, including the target center and the sampling, and match closely to the parameters that you get in the ImageSolver script by Searching for NGC3324 parameters with the Search button and that target selected. Nonetheless, WBPP v2.5.3 fails again to Plate Solve and hangs on the very first Red on-target image -- as above in your message plus the ImageSolver script fails on all images I tried selected from all the filters. The plate solving fails as you suggest (see attachment). Any ideas appreciated, but I did check that my previous target (NGC4038 & 4039) raw data plate solves without error using ImageSolver and it was successfully processed by WBPP v2.4.5.
Jeff
 

Attachments

  • PlateSolveError.txt
    929 bytes · Views: 47
Last edited:
More info: As an independent source of image paramegers, TSX Pro says the NGC 3324 target has the following parameters, (although entering these parameters in ImageSolver Script, it still failed as before):
1663819207981.png


However, TheSkyX Pro Blind Solver successfully plate solves the offending image itself (the first Red raw sub that WBPP hung up on its _c_cc version in the above post) with the following parameters (using the GAIA2 database which is one of the online databases I tried with PI ImageSolver):

TheSkyX Professional ...All Sky Image Link Succeeded. Found solution:
******** ASTROMETRIC SOLUTION RESULTS ********
From cropped area 2475 x 1980
Center RA (2000.0): 10h 38m 11.68s
Center Dec (2000.0): -58° 33' 50.7"
Scale: 1.1290 arcseconds/pixel
Size (pixels): 4500 x 3600
Angular Size: 1° 24' 41" x 1° 07' 44"
Position Angle: 194° 38' from north through east
Mirror Image: No
RMS: 0.51 (X: 0.35 Y: 0.37)
Number of Stars Used in Solution: 519 (99%)
FWHM: 1.77 pixels, 2.00 arcseconds
***********************************************

Yet when I enter those exact parameters above (Topographic Center RA/DEC, ImageSolver already had the pixel scale correct) into PI's ImageSolver script, it failed once again in exactly the same way.

>>>> Is this a problem with PI's ImageSolver Script ??? If TSX can plate solve the FITS file and overlay it on its map of the sky, why can't PI?

I attach a link to the image discussed here that TSX Pro can plate solve, but PI's ImageSolver script apparently cannot solve
:

NGC 3324 R -25 600 BIN1 PA000 E 20220211 010213.fts

drive.google.com drive.google.com

Jeff
 
Last edited:
This image can be plate-solved without any problems with our ImageSolver script, with default parameters. This is with our Gaia DR3 local XPSD database:

Code: 
Image Plate Solver script version 5.6.4
===============================================================================
Referentiation matrix (world[ra,dec] = matrix * image[x,y]):
+3.03424173e-04  +7.92652237e-05  -8.25395615e-01
-7.93254002e-05  +3.03437169e-04  -3.67721023e-01
WCS transformation ....... Thin plate spline
Control points ........... 1024
Spline lengths ........... l:1024 b:1024 X:1024 Y:1024
Projection ............... Gnomonic
Projection origin ........ [2250.029536 1800.061337] px -> [RA: 10 38 11.703  Dec: -58 33 50.87]
Resolution ............... 1.129 arcsec/px
Rotation ................. -165.354 deg
Reference system ......... ICRS
Observation start time ... 2022-02-11 04:03:10 UTC
Observation end time ..... 2022-02-11 04:13:10 UTC
Geodetic coordinates .....  70 40 36 W  30 22 10 S  1600 m
Focal distance ........... 1096.15 mm
Pixel size ............... 6.00 um
Field of view ............ 1d 24' 40.6" x 1d 7' 44.5"
Image center ............. RA: 10 38 11.703  Dec: -58 33 50.87  ex: +0.000374 px  ey: +0.003774 px
Image bounds:
   top-left .............. RA: 10 31 48.211  Dec: -58 55 19.28  ex: -0.054343 px  ey: +0.004155 px
   top-right ............. RA: 10 42 25.305  Dec: -59 17 02.78  ex: +0.047739 px  ey: -0.035882 px
   bottom-left ........... RA: 10 34 08.346  Dec: -57 50 09.97  ex: +0.026565 px  ey: +0.063604 px
   bottom-right .......... RA: 10 44 27.331  Dec: -58 11 13.64  ex: +0.065681 px  ey: +0.023756 px

I have just loaded the image, then executed the ImageSolver script, reset its parameters, and enabled distortion correction to run it exactly as WBPP does. The image is solved perfectly in a few seconds.

Now this is with the online Gaia DR2 catalog (VizieR service), same limit magnitude of 13.63 that you have reported:

Code: 
Image Plate Solver script version 5.6.4
===============================================================================
Referentiation matrix (world[ra,dec] = matrix * image[x,y]):
+3.03424247e-04  +7.92641837e-05  -8.25417979e-01
-7.93275335e-05  +3.03427543e-04  -3.67726030e-01
WCS transformation ....... Thin plate spline
Control points ........... 972
Spline lengths ........... l:972 b:972 X:972 Y:952
Projection ............... Gnomonic
Projection origin ........ [2250.081925 1800.164459] px -> [RA: 10 38 11.708  Dec: -58 33 50.78]
Resolution ............... 1.129 arcsec/px
Rotation ................. -165.353 deg
Reference system ......... ICRS
Observation start time ... 2022-02-11 04:03:10 UTC
Observation end time ..... 2022-02-11 04:13:10 UTC
Geodetic coordinates .....  70 40 36 W  30 22 10 S  1600 m
Focal distance ........... 1096.16 mm
Pixel size ............... 6.00 um
Field of view ............ 1d 24' 40.6" x 1d 7' 44.5"
Image center ............. RA: 10 38 11.708  Dec: -58 33 50.78  ex: -0.003944 px  ey: +0.011434 px
Image bounds:
   top-left .............. RA: 10 31 48.126  Dec: -58 55 18.96  ex: -0.010403 px  ey: +0.009473 px
   top-right ............. RA: 10 42 25.296  Dec: -59 17 02.82  ex: +0.043401 px  ey: -0.003695 px
   bottom-left ........... RA: 10 34 08.440  Dec: -57 50 09.31  ex: -0.003608 px  ey: +0.006585 px
   bottom-right .......... RA: 10 44 27.169  Dec: -58 11 14.80  ex: +0.155303 px  ey: +0.164953 px

As you see, the image is also solved without problems.

using the GAIA2 database which is one of the online databases I tried with PI ImageSolver
Click to expand...

Most likely this is the problem. Online database services are slow and fail sometimes for a variety of reasons. Using online databases for plate solving is deprecated in PixInsight since years. Everybody should be using our local Gaia DR3 XPSD databases.
 
Thanks very much, Juan-
However, I have never had online database services fail before (and they still work fine for other recent images than this particular set, as I have recently rechecked). I would love to use a local Gaia DR3 database but have not figured out how to install that locally, as ImageSolver immediately fails when I select it as online. I have tried to follow your postings at:

PCC and Gaia DR3

I noticed Photometric Color Calibration only supports up to Gaia EDR3. Is DR3 support coming soon?
pixinsight.com
but I will try again to see if I can finally get the Gaia DR3 database installed and actually working (no luck so far).

Meantime, I was able to get further than before using Gaia DR2 online (Vizier) with -13.8 mag limit but ultimately that also failed me, giving:

"* Reached maximum number of iterations without further improvement."

I have PI v1.8.9-1 Build 1556 so don't know why I should get different results than you are getting with the default parameters. This suggests increasing the maximum number of iterations parameter, (which doesn't seem to exist in ImageSolver) or require high quality stars only (tried this but ImageSolver still fails in exactly the same way with the same message). There must be something you're doing that I'm not.

-Jeff
 
Last edited:
Ok, I managed to install Gaia DR3 catalog as local with the Gaia Process. In fact, the ImageSolver script then converged to a solution on this image for the very first time:

Image Plate Solver script version 5.6.4
===============================================================================
Referentiation matrix (world[ra,dec] = matrix * image[x,y]):
+3.03366944e-04 +7.92664107e-05 -8.25250405e-01
-7.93012645e-05 +3.03375548e-04 -3.67664748e-01
WCS transformation ....... Thin plate spline
Control points ........... 2677
Spline lengths ........... l:2295 b:2523 X:2127 Y:2040
Projection ............... Gnomonic
Projection origin ........ [2249.971929 1800.047402] px -> [RA: 10 38 27.664 Dec: -58 31 46.23]
Resolution ............... 1.129 arcsec/px
Rotation ................. -165.354 deg
Reference system ......... ICRS
Observation start time ... 2022-01-27 04:28:48 UTC
Observation end time ..... 2022-01-27 04:58:48 UTC
Geodetic coordinates ..... 70 40 36 W 30 22 10 S 1600 m
Focal distance ........... 1096.36 mm
Pixel size ............... 6.00 um
Field of view ............ 1d 24' 39.7" x 1d 7' 43.7"
Image center ............. RA: 10 38 27.662 Dec: -58 31 46.20 ex: +0.006257 px ey: -0.002004 px
Image bounds:
top-left .............. RA: 10 32 04.578 Dec: -58 53 13.75 ex: -0.000706 px ey: -0.001960 px
top-right ............. RA: 10 42 40.948 Dec: -59 14 57.25 ex: -0.001334 px ey: -0.010173 px
bottom-left ........... RA: 10 34 24.681 Dec: -57 48 05.19 ex: -0.001921 px ey: +0.003516 px
bottom-right .......... RA: 10 44 42.786 Dec: -58 09 09.63 ex: -0.006600 px ey: +0.002673 px

I'll now try to run WBPP 4.5.3 on this set of images, assuming Local Gaia DR3 is, in fact, the WBPP 4.5.3 default, as suggested above and see if it finally gets past the PlateSolve step without failing immediately on the first image.
 
I'm happy to report that WBPP 2.5.3, with the Gaia DR3 catalog pre-installed* locally by the Gaia Process, successfully ran, including successful plate solving, over this entire data set dataset. The plate solving had to be set as "interactive" in WBPP, so that on the first plate solve instance, I could then change the selection to using the local Gaia DR3 catalog, instead of the Gaia DR2 catalog (VizieR online) that it first tried and failed. Then everything went along smoothly resulting in what looks like a very nice set of master lights.

Thanks very much Juan and Roberto for these outstanding WBPP.
Jeff

*Installation of the Gaia DR3 Catalog: First download the 16 DR3 XPSD files (only) from PixInsight S/W Distribution to the hard drive and then select them in "Gaia Preferences", clicking on the wrench of the Gaia Process Interface and select those files on your disk. Then do a global invocation of the Gaia Process, after which Image Solver uses them when Local/Gaia DR3 is selected in the ImageSolver script.
 
Last edited:
Hi Jeff,

Thank you and happy to know you've solved these problems. Just a few remarks on the Gaia installation procedure:

- There is no need to download the entire set of 16 XPSD database files, unless they are really necessary. For example, for wide field images the first 2-3 files, covering up to magnitude 17-18, are probably sufficient in most cases. For other medium field images the first 9 files, covering up to magnitude 20, may be enough.

- After selecting the database files in Gaia Preferences, a global execution of the Gaia process shouldn't be necessary. The Gaia process is a local database server and once the database files have been configured everything should work perfectly.

- If the Gaia process has been properly configured, the ImageSolver script will use it automatically when its Automatic catalog parameter is enabled. The Gaia process will also be used to find an optimal magnitude limit when the Automatic limit magnitude option is enabled. Similar options are available in PCC. Note that both options are enabled by default in both ImageSolver and PCC. With these options enabled and Gaia well configured, no online access will be performed for plate solving.

The same procedure and remarks are valid for the APASS process and the corresponding APASS DR10/DR9 XPSD database files.
 
You must log in or register to reply here.
Back
Top