Hi all,
We have just released an update with a new version of the ImageIntegration module. This update fixes two bugs and provides an important performance improvement for the ImageIntegration tool.
Bug Fixes* The min/max pixel rejection algorithm was generating NaN pixel values in the integrated image under some circumstances. As a result, a subsequent application of some tools and procedures (such as the ScreenTransferFunction tool in automatic stretch mode) was leading to a crash of the entire platform. This has been fixed. Please refer to the original bug report:
http://pixinsight.com/forum/index.php?topic=3357.0* Noise evaluation based on the multiresolution support (MRS) was failing in cases where the MRS algorithm is perfectly applicable. This was due to limitations in the internal PCL implementation, which we have overcome in this version. For more information, refer to the following thread:
http://pixinsight.com/forum/index.php?topic=3320.0In particular, MRS noise evaluation is now immune to black pixels, such as black borders caused during alignment, and implements an iterative routine to find the largest set of wavelet layers where noise evaluation converges to a stable result. The MRS noise evaluation routine is now much more robust and accurate with a negligible performance cost.
Performance ImprovementsThe ImageIntegration tool was initially designed to optimize memory usage to allow for integration of very large image sets on modest machines. As a consequence of these design patterns, its performance was quite poor in terms of scalability on multiprocessor/multicore machines, as has been
demonstrated recently. In this new version I have redesigned significant parts of the ImageIntegration tool to overcome these limitations. The ImageIntegration tool is now much faster and its performance scales really well for multithreaded execution. In addition, new parameters have been added that allow the user to fine tune memory consumption versus performance optimization.
The ImageIntegration tool has now a
stack size parameter. This is the size of the working integration stack structure. In general, the larger this parameter, the better the performance, especially on multiprocessor/multicore systems. The best performance is achieved when the whole set of integrated pixels can be loaded at once in the integration stack. For this to happen, the following conditions must be true:
(1)
Buffer size must be large enough as to allow loading an input file (in 32-bit floating point format) completely in a single file reading operation. For example, the default value of 16 MB allows loading a 2Kx2K monochrome image in a single operation.
(2)
Stack size must be larger than or equal to W*H*(12*N + 4), where W and H are the image width and height in pixels, respectively, and N is the number of integrated images. For linear fit clipping rejection, replace 4 with 8 in the above equation. Note that this may require a large amount of RAM available for relatively large image sets. As an example, the default stack size of 1024 (1 GB) is sufficient to integrate 20 2048x2048 monochrome images optimally with the default buffer size of 16 MB. With a stack size of 4 GB and a buffer size of 64 MB you could integrate 20 4Kx4K monochrome images at optimum performance on a 64-bit version of PixInsight. By
optimum performance I mean that every processor core available on your machine will be used at 100% load during the whole integration process.
Even with suboptimal buffer size and stack size parameter values, ImageIntegration performs now much better than the previous version. Here is one quick benchmark:
- Machine: Custom made workstation, Intel Core i7 990 Extreme Edition @3.47 GHz (6-core w/HT = 12 logical processors), 24 GB DDR3 1300.
- OS: Fedora 14 Linux 2.6.35.14-95 x86_64
- Desktop: KDE 4.6.5
- PixInsight version: 1.7.1.715 Starbuck (development version)
- 19 2048x2048 FITS images previously registered with StarAlignment.
- All images already in ImageIntegration's cache.
- All parameters by default except rejection algorithm set to Winsorized Sigma Clipping.
Times in seconds read from PI's console (best out of 10 runs):
Threads New Old
12 6.10 15.29
10 6.55 15.88
8 6.95 15.71
6 6.70 15.70
4 8.07 17.44
2 12.22 21.41
1 19.13 19.75where "old" corresponds to the version of the ImageIntegration module initially released with PI 1.7.0 (1.5.0.98), and "new" is the latest version we have released today (1.7.0.110).
