PixInsight Core 01.08.06.1448 Ripley (x64) crashing

[cliff_n]

Hello

I run PixInsight on Linux (Linux 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux) on pretty good hardware, 16GB RAM, SSD drive for operating system and larger capacity drive for storage.  I have PixInsight's swap storage set up to use the SSD drive and there is about 128GB free space on that drive.  I have been using PixInsight without problems for about a year, but since upgrading to 1.08.06.1448 I am having frequent crashes.  For example, tonight I tried using ImageIntegration to make a master bias from 57 images and PI crashed (i.e. the whole program went away without notice).  I tried again with logging enabled.  This time the ImageIntegration process crashed but the PixInsight program itself did not. The tail of the log file reads:

-snip-
[2019-01-04 23:48:43] Converting to RGB:
[2019-01-04 23:48:43] Computing image statistics:
[2019-01-04 23:48:43]
[2019-01-04 23:48:43] * Waiting for running tasks to terminadone
[2019-01-04 23:48:44] *** Error:
[2019-01-04 23:48:44] *** Error: Out of memory
[2019-01-04 23:48:44] <* failed *>
[2019-01-04 23:50:55]
-end-of-file-

I read several existing forum entries about memory errors and tried lowering the Buffer Size from 16 to 4.  Once again the process ran for three or four minutes and then PixInsight (the whole application) crashed.  The last few lines in the log are:

-snip-
[2019-01-05 00:16:58] Computing image statistics: done
[2019-01-05 00:16:58] MRS noise evaluation: done
[2019-01-05 00:16:58] Scale factors   :   1.01938  1.01990  1.00606
[2019-01-05 00:16:58] Zero offset     :  +3.051804e-05 +3.051804e-05 +1.525902e-05
[2019-01-05 00:16:58] Noise estimates :  5.5185e-04 4.8657e-04 7.0265e-04
[2019-01-05 00:16:58] Weight          :     1.02717    1.04643    1.01975
[2019-01-05 00:16:58]
-end-of-file-

I mean to show that the log had nothing strange -- no errors or warnings -- it just stopped.

I would suspect maybe one of my RAM chips is bad or something, but so far only PixInsight is crashing.  I would appreciate any help in troubleshooting this problem.  I plan to run an extensive memory check overnight, but is there anything else I could try in PixInsight?

[Juan Conejero]

This is a known problem with the new RAW module in version 1.8.6. It eats much more memory than the previous DSLR_RAW module in version 1.8.5. The sudden application termination happens because the Linux kernel applies a kill process or sacrifice child policy in the event of a severe out-of-memory condition.

This problem is now fixed (see the GitHub commit I've uploaded today). The fix will be included in the next version of PixInsight that I'll release in a few days, so I ask for patience.

Anyway, if you convert the bias frames to the XISF format (use the BatchFormatConversion script for example), you can integrate hundreds of images without problems. This issue is specific to the RAW format support module.

[cliff_n]

Thank you for the quick answer.  I will wait patiently.  PixInsight is an excellent program, so a few growing pains are to be expected.

[Juan Conejero]

Thank you. Anyway, I see you are integrating RGB color data. If this is an integration of bias, dark or flat frames, then using RGB data is probably incorrect. If you are integrating frames acquired with a digital camera, such as a DSLR, then you should use the following input hints in ImageIntegration:

raw cfa

(open the Format Hints section of ImageIntegration's interface to access this parameter). In this way, the process will load and integrate monochrome raw CFA frames, which is the correct procedure to generate calibration master frames. With single-channel data, you should be able to integrate more than 57 frames without problems, even with the current RAW module.

At any rate, if you convert the original data to XISF, then everything will be much faster and there will be no memory issues. Hope this helps.