Hi all,
I have made a first revision of Kai's script. You'll find it attached to this post.
First of all I want to say a big thanks to Kai for his nice initiative with this script. His work is excellent and the script is clear and well designed and structured. It has been a pleasure working on it.
The first change you'll notice is a new name: Batch Preprocessing. I think this script has the potential to become a complete preprocessing pipeline for PixInsight, so a more general name was required in my opinion.
The revised version of the script does not rely so heavily on FITS headers. Now you can manually add bias, dark, flat and light frames that the script will load as such without further questions (you are supposed to know what you are doing in such case). Of course, the "Add Files" button is still there and works by selecting frame types automatically, based on standard FITS keywords. Both "Bias Frame" and "BIAS" keyword flavors are now supported.
The independence on FITS headers should make the script much more flexible. It should work now with raw frames generated by applications that don't store frame type keywords (which is a very bad practice IMO) and opens the door to support DSLR and OSC raw frames with little effort.
A change that all of you will love for sure is that the script can now generate instances. I have replaced Kai's settings system based on configuration files (which was efficient and very well designed by the way) with a standard import/export system based on the Parameters PJSR object. This means that the script integrates tightly with the platform, so you can generate process icons with your preprocessing pipelines and store them in XPSM icon files and XOSM project files.
The script now disables ImageIntegration's file cache for integration of bias, dark and flat frames. This greatly improves efficiency, since it prevents calculation of unnecessary noise estimates.
I have simplified the image integration parameters section. Integration and rejection normalization cannot be selected now. This is because these parameters are always fixed as either "no normalization" and "equalize fluxes", depending on frame types (the latter for flat frames), and changing them makes no sense. I have removed CCD Noise Model rejection because it is only of academic interest. I have added support for min/max rejection because it can be necessary sometimes. Finally, I have removed the image weighting parameter because calibration raw frames are never weighted. If you think that one of these parameters should be restored, please argument why, and we'll put it back.
Another change is the way master frames are managed from the GUI. Previously when you selected one of the "Use as master" options the corresponding list of files was cleared. This no longer happens, so you are free to enable or disable these options. This is more friendly in my opinion, and although it increases the risk of doing things wrong, will lead to increased flexibility when we implement a more complete management of file lists (e.g., when we can remove selected frames).
Finally I have made a GUI revamp with new buttons, some reorganization, more graphical elements and extensive use of Qt CSS styling. During this process I have also discovered a couple of minor bugs in the PixInsight Core application (some faulting PJSR routines), for which I have found good workarounds, fortunately.
The script is perfectly usable right now. It is rather stable and robust, and yields excellent results as far as I have tested it. A lot of work remains to be done, however. These are just a few pending tasks:
* URGENT: Decide on a software license to release this script as official, via the update system. Our policy is to never release source code without a clearly stated license. I suggest using our
PCL License v1.0, but this is of course Kai's decision.
* Add support for DSLR and OSC Bayer CFA raw frames. Not difficult now that the script doesn't depend on FITS headers.
* Add more file management features: remove selected frames, use enabled/disabled frame states, manage selections, etc.
* Improve diagnostics and error information.
* Make the script more robust against failed processes.
Hope you like it!
