WBPP last step to image integration out of memory error

A

Agroleau

New member
Hi everyone. Using WBPP with 50 darks, flats, bias and 700 lights. All goes well until the last step of image integration. I get an out of memory error.

The software runs on windows 2016 server with 64gig ram 2 tb disks and 12 cpu.

Group of 700 Light frames (700 active)
SIZE : 8288x5520
BINNING : 1
Filter : NoFilter
Exposure : 1.00s
Keywords : []
Mode : post-calibration
Color : Gray

Reference image: C:/Batch Preprocessing/calibrated/Light_BIN-1_8288x5520_EXPOSURE-1.00s_FILTER-NoFilter_Mono/DSC_8685_c_1.xisf
Registration completed: 700 images out of 700 successfully registered.
Rejection method auto-selected: Generalized Extreme Studentized Deviate
Local normalization: : reference frame generated by integrating 20 frames
Local normalization reference image: : C:/Batch Preprocessing/master/Light_BIN-1_8288x5520_EXPOSURE-1.00s_FILTER-NoFilter_Mono_LN_Reference__(1).xisf
Local normalization: completed.
Rejection method auto-selected: Generalized Extreme Studentized Deviate
*** Warning: ImageIntegration failed.
!!! Error: Warning: Master Light file was not generated.

Can anyone help me with this issue?
Thank you very much
 
one way to deal with this is to check your windows virtual memory settings and make sure it has a large enough page file or is allowed to create new page files on demand.

another way is to choose a different pixel rejection method - GESD is very memory and CPU intensive.

a third way is to just break up your run into multiple sets so that you're not integrating 700 at a time, then use ImageIntegration to integrate the masters you create.

rob
 
Thank you for your answer Rob,

I tried all the proposed options. It worked once with 300 lights in gray, but if i want color RGBG i still get the same error with 300 lights linear or winsorized pixel rejection.

Any other suggestions?

Also the total process time is around 5 hours if that helps.

Thank you

Alain
 
well... the only thing i can think of is to keep lowering the number of subs in each run. 64GB seems like a lot of memory but at 8288x5520x4 bytes per pixelx3 planes, you're talking about 524MB per image and so just loading everything into memory, you run out of RAM at 125 images. and of course you can't use all 64GB for just PI.

PI shouldn't be trying to load the whole image into memory at once since they are in XISF format, but it could be due to other overheads 300 lights is still too many. if you use ImageIntegration to integrate your subs you have control over how much memory PI tries to use, but when you use WBPP, WBPP does not expose those controls of ImageIntegration so you can't control the memory usage.

rob
 
You must log in or register to reply here.
Back
Top