New Fast Batch Preprocessing (FBPP) 1.0.0 script

robyx

Administrator
Staff member
Hi all,

I am glad to announce the release of a new script called FBPP (Fast Batch Preprocessing). This script is part of the PixInsight v1.8.9-3 distribution package.

This script has been developed as a simplification of WBPP, maintaining all the flexibility offered in the calibration phase but adopting Fast Integration as the main process used to register, normalize, and integrate the light frames after their calibration. Only a minimal subset of meaningful options is available, and the default parameters have been tested and tuned to make this tool simple and reliable, with a special focus on fast execution.

Below is a short summary of the interface sections along with the corresponding settings available.

Bias panel


FBPP_bias.png

The Overscan setting is available to correct for the bias drift if required.
The integration of bias frames is executed with following default parameters:
  • combination: Average
  • rejection method: automatic (depending on the number of frames and the standard rejection parameters)
  • standard rejection parameters
Darks panel


FBPP_darks.png

The exposure tolerance setting is available. This defines the maximum exposure tolerance difference between the longest and shortest dark frame within a dark frames group.
The integration of dark frames is executed with following default parameters:
  • combination: Average
  • rejection method: automatic (depending on the number of frames and the standard rejection parameters)
  • standard rejection parameters
Flats panel


FBPP_flats.png

The integration of flat frames is executed with following default parameters:
  • combination: Average
  • rejection method: automatic (depending on the number of frames and the standard rejection parameters)
  • standard rejection parameters
  • Large-scale pixel rejection remains disabled
Lights panel


FBPP_lights.png

The calibration exposure tolerance setting is available.This value defines the maximum exposure tolerance difference between the longest and shortest light frame within a light frames group during calibration.
No settings are required as Fast Integration is executed with default parameters to register, normalize and integrate the light frames.

Calibration panel


FBPP_Calibration.png
The calibration panel provides the same overview of WBPP, showing the status of Bias, Dark, Flat and Light groups. The number of settings are reduced to the following:
  • enable/disable calibration masters
  • manually set the calibration masters
  • enable the Master Dark optimization
  • mark a group as CFA, selecting the CFA pattern and the debayer method
Post-Calibration panel


FBPP_post.png

The post-calibration method shows the groups of light frames each of which will be registered, normalized and integrated to generate a master light.
The available setting is the exposure tolerance that defines the maximum exposure difference between the longest and the shortest light frame within a post-calibration group.

Pipeline panel


FBPP_pipeline.png
This panel shows the list of operations that FBPP will perform when executed.

General features


File additions
Adding files to FBPP works exactly as it does for WBPP. FBPP adopts the same file identification strategy by reading the FITS/XISF headers to extract the required metadata in order to properly classify a file as bias, dark, flat, or light, as well as to extract the filter name, the exposure, and other information.
Diagnostic
The FBPP configuration diagnostic is available to preemptively identify possible configuration problems.

Grouping keywords
FBPP still offers the flexibility provided by the Grouping Keywords to organize complex calibration and post-calibration configurations if needed.

Registration Reference Image
The registration reference frame maintains the following options:
  • Auto: FBPP will measure the first 5 frames of each post-calibration group to choose the frame with the highest number of stars as the reference.
  • Manual: The user selects the reference frame.
  • Auto "by keyword": This feature is still available if a keyword in post mode is present, to accommodate the creation of masters belonging to different mosaic panels.
Global options
A few global options remain:
  • "Detect masters from path": If the keyword "master" is contained in a bias, dark, or flat file path the file will be interpreted as a master file.
  • "Rejection maps / drizzle files": Creates the rejection maps to be saved within the master light file, as well as the drizzle files to subsequently run Drizzle Integration if desired.
  • "Smart naming override": This allows the association of metadata to the file using a key-value syntax by properly naming the containing folder or the file name itself.
Cache
FBPP runs with no caching system.

About Fast Integration

Here we briefly discuss the reasons behind the choice of adopting Fast Integration as the main process used by FBPP to generate the master frames.

Fast Integration is a process that has been developed for two purposes: first, it is designed to be fast by keeping all the intermediate images in memory during registration, normalization, and integration. Secondly, it is capable of handling a very large number of files with limited memory resources by implementing a parallelized batched pipeline.

Despite some simplifications adopted to achieve maximum speed (such as using homography instead of thin-plate splines for registration and global instead of local normalization), FBPP is approximately equivalent to WBPP for large datasets that are not affected by strong variable gradients and distortions (like large widefields), although results may strongly depend on data quality. In the latter cases, the thin-plate splines for registration and local normalization with the auto-generated reference frame adopted by WBPP are treasures that can produce significantly better results at the cost of a more resource-demanding task.

WBPP remains the tool with the maximum flexibility and control that can address all scenarios; on the other hand, FBPP is designed to be fast and effective in the vast majority of cases.

As a final note, even though Fast Integration has been designed in collaboration with astrophotographers adopting fast imaging techniques (which require processing tens of thousands of light frames), this tool is also suitable and works quite well with a limited number of light frames (i.e., fewer than 30).



——————————————————————————
Thank you for your attention,

The PixInsight Team at Pleiades Astrophoto S.L.
 
Back
Top