Multithreaded ABE (& Debayer ?)

[dmarchais]

Hi,
Would it be possible to make the automatic background extractor process multithreaded ? At the moment I have an automatic pipeline using ABE instead of flat field correction (which works almost fine for my needs) and ABE is the bottleneck since I need to run it on each and every sub sequentially, whilst intensive processes like registration and integration use the full army (bi-xeon, 32T) and complete quite quickly. I have also a doubt on Debayer process. Seems multithreaded however only two threads appear in the console log. Do I miss something here ?
Many thanks,
Denis.

 

[Juan Conejero]

The ABE process implementation is very old (2005) and has never been parallelized. It is designed to be used as an interactive tool and is quite efficient for that use. Its parallelization would require a complete redesign, which cannot be justified given the amount of priority tasks that we have ahead.

DBE is already parallelized for sample generation and background model application. As happens with ABE, this process has been designed and implemented as an interactive tool, not as a batch process. DBE and ABE are interactive tools for correction of residual gradients. They are not intended to replace a flat fielding task with the appropriate master flat frame, which is an essential part of any data reduction procedure.

The Debayer tool is fully parallelized at a high level as a batch process in recent versions of PixInsight, and can use all available processors very efficiently. However, in the latest version we have implemented an adaptive memory protection feature that limits the number of threads if the amount of physical memory available is insufficient to use all available processors concurrently. This feature prevents resource exhaustion conditions that were causing serious problems in previous versions, especially because more and more users tend to work with huge data sets and very small RAM sizes.