cannot reproduce ImageSolver with Distortion Correction checked generates error

I have been troubleshooting the ImageSolver script(ImageSolver.js) to try to find out where the exception was occurring.

I created a new script ImageSolverjpc.js so I could add several console.noteln at various point in the code to write to the Process Control window.

I have noticed that the failure occurs just after this section of code(line 2575 in ImageSolver.js)

The ReferSpline call (in WCSmetadata.jsh) finished and then an exception happens.

newMetadata.ref_I_G = new ReferSpline( pairs.pI, pairs.pG,
this.solverCfg.rbfType,
this.solverCfg.maxSplinePoints,
this.solverCfg.splineOrder,
this.solverCfg.splineSmoothing,
this.solverCfg.enableSimplifier,
this.solverCfg.simplifierRejectFraction );


Jack
 
PixInsight Version: PI-macosx-x64-1.9.2-20241228-c, build 1632, released 24-12-28

Computer: Mac mini 2018
OS: macOS Sequoia Version15.1.2
Processor: 3.2GHz 6-Core Intel Corei7
Memory: 16 GB 2667MHz DDR4

Computer mitigations

1. Disabled Time Machine(manual mode).
2. No external drives are connected.
3. BlueTooth is off.
4. WiFi is off.
5. Ethernet cable is disconnected.
6. Disabled Adobe Creative Cloud.
7. No other Apps are open.
8. Don't have antivirus software.

Directories

1. Re-downloaded all 16 Gaia DR3 files and verified checksum.
2. Put the gdr3 files in a folder in the Documents folder(PIGaiaDR3Database)
3. Put the gdr3sp files in a folder in the Documents folder(PIGaiaDR3SPDatabase)
4. Configured the Gaia process to point to the gdr3 files and to the gdr3sp files.
5. Created a folder in the Documents folder for PISwap files(PISwapFiles). Set up four swap storage directories via
Global Preferences:Directories and Network that all point to the PISwapFiles directory.

ImageSolver setup

1. I have tried the online star catalog. That did not make a difference in the failure.
2. I have been troubleshooting the ImageSolver script.
a. I have found that if the ImageSolver script fails it always fail at the call
this.inverse.spline = this.spline.intializeWithInverse(P1, P2,
this.smoothing,
null,/*weights*/,
this.order,
this.rbfType );

b. I have found that small image files do not fail. (1440x1440, 1038x1997)
However, slightly larger files fail. (1450X1444, 1038x1999) with the error message:
The image could not be fully solved. We have tagged it with the latest known valid solution. (@line3313 in ImageSolver.js)

Larger files(3000,2048) fail with the message:
*** Error : (@line2623 in ImageSolver.js)


c. I ran the Mac's Activity Monitor with 1 second updates and PixInsight memory usage was never greater than 1.8 GByte.

Total memory usage was never greater than 6.25GByte.(Physical Memory 16GByte)

d. I monitored the p1.length and p2.length used in the function ReferSpline. If the length was much greater than 450 an error would occur.


Conclusion
1. As Juan has said, it does not look like an overall memory problem.

2. There appears to be a correlation between the p1.length and the failures

Jack
 
Last edited:
Jkcolli said:
1. As Juan has said, it does not look like an overall memory problem.

2. There appears to be a correlation between the p1.length and the failures
Click to expand...

yes and indeed it does sound very similar to whatever behavior in the JS memory system that PhotometricMosaic triggers on lower-memory macs. i guess some people are going to be stuck with this.
 
I also still have this exact issue with both my 16gb intel based Mac's. The only way it currently works is to disable distortion correction which disables the DDM thin plate spline. I haven't dived too deep into the JS code like @Jkcolli since I'm not proficient in that area but I am glad he has found where it fails as this may give the developers a place to start. I have been thinking about what has changed between 1.8.9 and 1.9.2. This problem did not occur with 1.8.9 at least on 16gb Macs but it may involve the DDM splines as splines caused problems in the past with 8gb Macs. The way they are calculated has changed with version 1.9.2 to the new DDM thin plate spline. I read in another post today that a newer version of PI is coming in the near future and is in testing now. It had more to do with WBPP rejecting good images randomly but maybe also related since distortion modeling is happening there too.
 
Juan Conejero said:
I cannot reproduce this problem. Besides what Nico has said, which is probably the cause of this problem, can you please upload an image where this can be reproduced?
Click to expand...
Juan...I was on the forum a few weeks ago experiencing what many people what saying they were dealing with with wbpp and astrometric solution not working and image solver as well. There were remarks about the hardware issue for those like me who run Mac OS and whther the RAM was sufficient to run 1.9.2. Since then, I have purchased a new Mac with 32GB of RAM and I ran my same date back through wbpp with astrometric solution checked and it ran without error. I then ran my new master back through image solver with distortion correction checked and it worked as well. I just thought I would pass this on to you.
 
  • Like
Reactions: Hab
cloudbait said:
Ok... that's "good" because it should be a lot easier to track down if all you have to do is make adjustments to ImageSolver, without having to go through the whole WBPP sequence repeatedly!
Click to expand...
I was on the forum a few weeks ago experiencing what many people what saying they were dealing with with wbpp and astrometric solution not working and image solver as well. There were remarks about the hardware issue for those like me who run Mac OS and whther the RAM was sufficient to run 1.9.2. Since then, I have purchased a new Mac with 32GB of RAM and I ran my same date back through wbpp with astrometric solution checked and it ran without error. I then ran my new master back through image solver with distortion correction checked and it worked as well. I just thought I would pass this on to you.
 
fredvanner said:
OK. I agree that on closer examination file sync issues are unlikely to be causing this.
From the log it looks as though the initial solution is fine (which is the stage at which all the catalogue access will be performed); the error seems to occur immediately at the start of the iterative distortion correction phase. The fact that this works on other systems makes it unlikely that this is a code error. It looks more like a hardware fault, of a hardware compatibility issue.
Click to expand...
I was on the forum a few weeks ago experiencing what many people what saying they were dealing with with wbpp and astrometric solution not working and image solver as well. There were remarks about the hardware issue for those like me who run Mac OS and whther the RAM was sufficient to run 1.9.2. Since then, I have purchased a new Mac with 32GB of RAM and I ran my same date back through wbpp with astrometric solution checked and it ran without error. I then ran my new master back through image solver with distortion correction checked and it worked as well. I just thought I would pass this on to you.
 
Well, I purchased a new Windows based desktop with 32gb RAM and Intel i9 CPU, etc. Using PI 1.9.2, I processed a dataset that failed on my Intel MacBook Pro with 16gb RAM. Using WBPP with distortion correction selected, the astrometric solution solved without problems. It seems clear that something with PI and several models of Mac are at odds with each other. I was very impressed with the speed of my new computer at processing in PI and will use it for stacking, post processing prior to exporting final images to other software on my Mac for further image 'tweaking'.
 
Is there any progress on this? Just updated to 1.9.2 on my 16GB ram MacBook Pro and having the same issue. Solver works without distortion correction but fails with ***Error: Error with it on. Also fails in WBPP. All worked in previous version of pixinsight (do I need to downgrade?)
 
MBalkham said:
Is there any progress on this? Just updated to 1.9.2 on my 16GB ram MacBook Pro and having the same issue. Solver works without distortion correction but fails with ***Error: Error with it on. Also fails in WBPP. All worked in previous version of pixinsight (do I need to downgrade?)
Click to expand...
Word from 1.9.3 beta testers on Macs is that this has been resolved. Hang tight; I expect that update will be released in the next few days.
 
You must log in or register to reply here.
Back
Top