ImageIntegration lags before starting

[EorEquis]

For the last, oh, 2 weeks or so, I've noticed that ImageIntegration takes an increasingly longer time to "get started".  Upon clicking the Global Apply button, PI freezes, goes to "Not Responding", and is hung for quite some time before finally the Process Console appears, and integration begins.

At first, the delay could be measured in single digit seconds.  This morning, the delay was 42 seconds to begin integrating 18 frames and their .DRZ files.

I've changed no settings, nor made any changes to my calibration processes or files.

• It happens consistently, every time I run ImageIntegration
• It happens regardless of whether drizzle files are included
• Once the process begins, it runs normally and quickly
• No other process is affected.  All other processes start executing normally and in a timely manner.
• The number of frames being integrated does not change the delay.  Right after waiting 42 seconds to begin integrating 18 frames, I attempted to integrate a different 32 frames.  The delay was 41s.

I'm not looking for an instant fix (though if anyone has one, that'd be awesome *heh* ) but rather some ideas as to where to start looking to troubleshoot.

Thanks in advance.

**EDIT**

Sorry, I should have been more thorough about system/software details.

Win7 64 bit, Intel Core i5-3570K, 16GB RAM
120gb SSD Swap Drive
PI 1.08.03.1123 Ripley (x64)

[georg.viehoever]

Which OS? How much RAM? Did you restart PI in between? Did you try to reboot? How much memory does the PI process consume? Is memory consumption increasing with each ImageIntegration run? Are you using the BPP script?

Georg

[EorEquis]

Hi Georg!  Thanks for trying to assist.

Which OS? How much RAM?

I added this to the OP, apparently right about the same time you were asking.  Sorry for leaving that off.

Did you restart PI in between? Did you try to reboot?

Yes, this has happened over several weeks, many restarts of PI and several reboots of the machine.

How much memory does the PI process consume?

~ 100MB when it first launches.

Adding 18 1391x1039 FIT frames and their associated DRZ files to ImageIntegration brings it up to ~ 140MB

Clicking "Apply Global" leaves it unchanged, until processing starts, at which point it climbs slowly to a max of ~ 240MB, and then releases back to ~140MB when integration processing is complete.  There was a very brief (less than 1 second) spike to ~600mb as the integrated image was being rendered on the screen.

Is memory consumption increasing with each ImageIntegration run?

No.  Repeated integrations of the same 18+18 files resulted in the same memory usage. 

Perhaps of significance, if PI is not restarted, reintegrating the same files does not produce the delay.  If PI is restarted, or if other files are selected, then the delay returns.

Asking ImageIntegration to process any new set of frames, with or without drizzle files included, results in the delay to begin processing.  This is true regardless of whether PI is restarted in between or not.

Are you using the BPP script?

Yes, but only for calibration.  Registration and Integration are launched manually as separate processes.

[georg.viehoever]

Are the images on the SSD?
What type of SSD are you using? Samsung 840 EVO?
How much space is left on the SSD?

[EorEquis]

Are the images on the SSD?

No.  In some cases, the images are on an internal drive, in other a USB3 external drive.  Issue is the same regardless of location, and there is no other issue opening/using files from either location in any other PI process or other application.

What type of SSD are you using? Samsung 840 EVO?

Toshiba THNSNH128GCST 128gb (not 120, sorry)

How much space is left on the SSD?

Unless PI is running, it is empty.  Being set as PI's swap drive is its only function.

[EorEquis]

Here is a video that shows the behaviour.

http://spaceonly.net/holding/videos/II_issue.wmv

[pfile]

could this be due to the image noise analysis cache becoming either too huge or corrupt?

you might try turning off "use file cache" and/or "evaluate noise" in ImageIntegration and see if anything changes.

rob

[EorEquis]

could this be due to the image noise analysis cache becoming either too huge or corrupt?

you might try turning off "use file cache" and/or "evaluate noise" in ImageIntegration and see if anything changes.

rob

Tried each/both, no change to the behaviour.  Also tried enabling/disabling pixel rejection and changing weights from All=1 to FITs keyword to Noise Evaluation...again, no change to behaviour.

[pfile]

bummer. i have seem slowness when the noise cache gets huge, so i had hoped it was the same problem.

at any rate this is where some kind of profiler would come in handy. this is actually built in to OSX, but on windows i don't know how you'd go about figuring out what routine PI is spending most of it's time in.

rob

[EorEquis]

Much obliged for the insights and noodling, Rob!

[EorEquis]

Wound up running the Reset PixInsight Settings app, which solved the problem.  now to go put everything back the way I had it.   

[MortenBalling]

It sounds like the cache. Have you tried clearing memory cache?

Be aware that normally cache is your friend, and that even though PI waits for a while, when you hit global, once things start going, cache really speed up the process!

Br

Morten