PCL Win32 Exception in Batch PreProcessing

[rvaneynd]

Installed the latest update (including the FITS one from yesterday).

Try to BatchPreProcess 101+ images including Large number of darks, some Bias and Flats.

Tried 5 times and every time at Integrating Channel 1 of 3 I get PCL Win32 Exception.

Computer is an Intel i7 with 12Gb of memory running Windows 7 Pro (all updates installed).

Attached a screenshot of the error.

Raoul

[Juan Conejero]

Hi Raoul,

Are you working with DSLR raw files? In such case, you are most likely running out of memory trying to integrate 140 frames with just 12 GB of RAM. The reason is that raw formats don't allow incremental file access, so ImageIntegration has to load the entire data set in memory to integrate it. The exception happens because some component that is out of our control (perhaps dcraw, or maybe the C runtime libraries) is trying to access a memory block that has not been allocated successfully as a result of scarce RAM resources.

The best solution to this problem is converting the raw frames to a format that supports incremental file reading operations, such as PixInsight's native XISF format. You can use the BatchFormatConversion script to do this. In this way you can integrate thousands of images without problems. Let me know if this helps.

[rvaneynd]

Juan,

Thank you for the quick response.
I am working with FIT files that come out of Sequence Generator Pro. The Camera used is a Canon DSLR.
Each FIT file is approx. 19Mb.

Do FIT files have the same problem?

I will try the conversion and let you know.

Raoul.

[Juan Conejero]

No, the FITS format supports incremental reading, so the problem must be caused by other factors. This does not happen under normal conditions, and AFAIK has not been reported before. Can you upload a data set where this happens consistently?

Anyway, trying with XISF is a good idea, since it can help us to narrow the problem. If you have no issues with XISF, then we know that FITS is the problem.

[rvaneynd]

Juan,

Okay, I did the batch conversion to XSIF (on the light files only) and then run the BatchPreProcessing and it all worked.
As I did receive some updates to PixInsight, I also run the BatchPreProcessing using the FIT files and once again the error attached.
So it might be a FITS problem.

Raoul

[Juan Conejero]

Hi Raoul,

So the problem is in the FITS format support module, that is, in CFITSIO. It seems there are still some thread synchronization issues that only happen in very special cases; it seems you have just found one of them.

I would need a reduced set of your images, if possible, where this problem can be reproduced. Could you please upload them?

[rvaneynd]

Juan,

I have PM'ed you a link to the full data set of light files (2.6Gb total 139 files).
Let me know if you need anything else.

Raoul