WBPP v2.2 Released

[robyx]

Hi all,

I am glad to announce a new version of the Weighted Batch Pre Processing WBPP script 2.2.0. This update is distributed with the new PixInsight core application 1.8.8-9 and makes available the new separate RGB channel alignment feature, along with other minor features and bug fixes.

Separated "Calibration" and "Post-process" tabs
Preprocess and post-process views available in WBPP 2.1.2 are now split into two dedicated tabs called "Calibration" and "Post-process".
The Calibration panel remains identical to the preprocess view of WBPP 2.1.2 while the "Post-process" tab extends the previous post-process view by showing some global options that controls the post-process flow and grouping and the list of active/inactive post-processing steps (active are emphasized in bold).

New separate RGB channels workflow
The new separate RGB channels workflow is implemented to recover non-isotropic non-linear distortions causing misalignments between the three RGB channels. This new workflow can be enabled in the post-process panel by selecting one of the Debayer modes available:

Once the "separate RGB Channels" or "Combined + Separated" option is selected, three grey masters will appear in the output list and be marked as "_R", "_G", "_B" in the Color Space column.
Files belonging to these three additional groups, which will be generated for each RGB group, will be aligned to the best reference frame adopting the following configuration:

Registration model = Thin Plate Splines
Distortion correction = enabled
Local distortion = disabled
Generate drizzle data = enabled
Noise reduction (Star Detection) = maximum value between two and the user-selected value
Compute PSF fits (Star Detection) = Distortion correction only

More details about this feature are provided in the PixInsight 1.8.8-9 announcement thread.

This specific configuration is applied only to the separated RGB channels while the combined RGB light frames will follow the regular registration.

Post-processing light exposure tolerance
The exposure tolerance shown under the Light panel is now used only to aggregate light frames in the calibration groups. The exposure tolerance input shown in the Post-processing panel now determines the tolerance to frames in post-processing groups. This allows to keep files with different exposure times separated during calibration and aggregate them into a single master light in post-processing.

Cosmetic Correction enabled by Binning
More than a single Cosmetic Correction instance can now be used, one for each binning value present in the session. WBPP autodetects the list of binning available and allows to assign an appropriate Cosmetic Correction instance to each.

Wider keyword value and filter name character set
To accommodate standard iso date, numbers and other more sophisticated values, keyword values can now contain the following special characters:

• dash -
• round parenthesis ( )
• semicolon :
• full stop .
• blank space " "

Filter names could now contain

• dash -
• blank space " "

such that filter names like "H alpha" or "O-3" are fully recognized.

Other Improvements

• more robust recognition of the frame type
• confirmation dialogue on exit is now presented only if groups are not saved
• files in tree boxes are sorted by name (but the master file always remain in bold on top)

Bugfixes

• manual settings are saved and restored correctly
• matching groups precedence has been fixed in some special cases
• addDirectory feature now works with both uppercase/lowercase extensions
• SNR weight control now affects correct weighting formula parameter
• "assignment to undeclared variable activeFrames" warning removed
• master bias selection when master dark does not contain the bias fixed in some special cases
• processing panel is properly refreshed when CFA properties changes

Enjoy this new upgrade!
Roberto

 

[niccoc1603]

It would be great to have an advanced option in Subframe Weighting to input a custom weighting formula and also a discard option based on outliers probability

 

[Mau Brd]

Hi Roberto, and thanks for this new important WBPP update.

I would like to ask your advice on the following.
After processing an initial group of images in WBPP, I took additional images on the same target, to increase the quality.
Is there any way to avoid reprocessing the first group of images in WBPP?

I thought to initially process with WBPP the second group only (setting the very same reference frame from the first group for alignment), and subsequently to integrate toghether all the aligned and weighted images, but I think now that this is not correct because the WBPPWGHTs are not absolute ones as they are computed with reference to average values of the batch.

A second possibility would be to process the additional images separately, but then not to rely on WBPPWGHT but to measure both batches with SubframeSelector and save the weights. After that integrate all the images using the new weights.

Many thanks in advance,
Mau

 

[Zorion]

Ähem,
sorry for my silly question,
but what do I have to do with the 3 Master-Channels after stacking?

Channel Combination or Image Integration?

CS
Chris
(and thanks for your great work)

 

[Mau Brd]

Ähem,
sorry for my silly question,
but what do I have to do with the 3 Master-Channels after stacking?

Channel Combination or Image Integration?

CS
Chris
(and thanks for your great work)

Hi Chris,
with Channel Combination you get a RGB master that can be processed as usual. The advantage now is that star channel alignment is way better than the classical RGB WBPP integrated image. Naturally you have the option to perform some steps on separated channels - i.e. Linear Fit or ABE - before ChannelCombination.

Kind regards,

Mau

 

[Zorion]

Hi Mau, thanks.

The new advantage was clear to me, but I did not know whether I would destroy this advantage again with only channel combination. I thought Image Integraton would do a better job there. But now everything is clear.

Clear Skies
Chris

 

[Mau Brd]

Hi Mau, thanks.

The new advantage was clear to me, but I did not know whether I would destroy this advantage again with only channel combination. I thought Image Integraton would do a better job there. But now everything is clear.

Clear Skies
Chris

Hi Chris, I see your point. My understanding is that Channelntegration is combining the three channels together while keeping the overall improved alignment, with no loss of information.
Kind regards,
Mau

 

[Zorion]

Grmmpf,
I'm not stupid, I'm super stupid, I have now tried the integration, but of course no Master RGB comes out of 3 mono channels. A mistake in reasoning, sorry.

 

[robyx]

Hi Roberto, and thanks for this new important WBPP update.

I would like to ask your advice on the following.
After processing an initial group of images in WBPP, I took additional images on the same target, to increase the quality.
Is there any way to avoid reprocessing the first group of images in WBPP?

I thought to initially process with WBPP the second group only (setting the very same reference frame from the first group for alignment), and subsequently to integrate toghether all the aligned and weighted images, but I think now that this is not correct because the WBPPWGHTs are not absolute ones as they are computed with reference to average values of the batch.

A second possibility would be to process the additional images separately, but then not to rely on WBPPWGHT but to measure both batches with SubframeSelector and save the weights. After that integrate all the images using the new weights.

Many thanks in advance,
Mau

This is a complicated question since WBPP is not made with the purpose of merging old already calibrated data with new non calibrated data.
There are cases where this could work but only with monochrome images and a meaningful result cold be achieved if you still own the old calibrated and non-registered frames.

In this case what you could do is to:
1. calibrate your new data, eventually apply cosmetic correction, but disable image weighting, registration and integration
2. run a new instance of WBPP removing all calibration master frames and adding the old and new calibrated non-registreted light frames, enable weighting/registration/integration and WBPP will measure all frames, register them together and integrate all frames.

But I would strongly recommend to inspect and measure properly your old and new data and proceed manually in merging them, I expect that some significant differences are non negligible and would require manual tuning to be optimally addressed (NSG script could help to normalize the two sessions each other if needed).

Robyx

 

[mcgillca]

Hi, I'm probably being stupid here, but I'm having problems with creating a new master flat file.

I am using existing master bias files, but want to use new flat files. However, when I add these to WBPP, it simply selects the first of the flats for each filter as the master flat - it doesn't try to combine the new flats to make a master - see screenshot below.

I've just updated to the Sept 20th 2021 version of PI 1.8.8-9 and am using WBPP v 2.2.0 on Mac OS Big Sur 11.6.

Any advice gratefully received,

Thank you,


Colin

 

[robyx]

Hi, I'm probably being stupid here, but I'm having problems with creating a new master flat file.

I am using existing master bias files, but want to use new flat files. However, when I add these to WBPP, it simply selects the first of the flats for each filter as the master flat - it doesn't try to combine the new flats to make a master - see screenshot below.

I've just updated to the Sept 20th 2021 version of PI 1.8.8-9 and am using WBPP v 2.2.0 on Mac OS Big Sur 11.6.

Any advice gratefully received,

Thank you,


Colin

Hi @mcgillca,
most probably your flat full path contains the word "master", that's a criteria to automatically identify a master file. If this is the case then you can uncheck the global option "Include file path to detect Masters", this will constrain the word master to be searched only in the file name. This should make your flat frames to be managed as flat frames and not as master frames.

Robyx

 

[mcgillca]

Thanks, Robyx. You were quite right - however, if I unchecked the global option "Include file path to detect Masters", this didn't pick up the fact that the bias files were masters. Not sure if this was because they were generated with an old version of WBPP?

Instead, I left that in an simply renamed the master directory I keep my flats in.

All seems to be running now though!

Thank you,

Colin

 

[robyx]

Thanks, Robyx. You were quite right - however, if I unchecked the global option "Include file path to detect Masters", this didn't pick up the fact that the bias files were masters. Not sure if this was because they were generated with an old version of WBPP?

are you sure these were created by BPP/WBPP and not by hand? in the first case the FITS keyword IMAGETYP should be "Master Bias" and this marks them as master files. In case of manual integration this keyword is not set as such and it could be that they needs the work "master" in the file name or in a containing folder.

All seems to be running now though!

Great to hear!!

 

[mcgillca]

are you sure these were created by BPP/WBPP and not by hand?

Pretty sure - they are dated April 2019, and are XISF files. The image type is listed as "Bias" though rather than master bias, but they are in a subfolder called "master" which is why they needed the "Include file path to detect Masters" to work. Not sure how this was picked up in older versions since they worked fine in those!

Oh well - will recreate the bias files at some point and that should sort it out properly.

Thank you very much for your help,

Colin

 

[mcgillca]

And having just looked at my masters, I'll need new bias frames anyway...

 

[tgainey]

I am relatively new to using this Weighted batch pre-processing script as have been a long time user of the previous (now obsolete) BachPreporcessing script.
I have found it frustrating that although it saves my file list on exit, when I next use it it forgets that I always use CFA files and I have to remember to reset this for my Flats and Lights every time.
Does the bug fix "processing panel is properly refreshed when CFA properties changes" mean that the script will now remember that my files are CFA for future use? (The old BachPreporcessing script did this).
Thanks

 

[robyx]

I am relatively new to using this Weighted batch pre-processing script as have been a long time user of the previous (now obsolete) BachPreporcessing script.
I have found it frustrating that although it saves my file list on exit, when I next use it it forgets that I always use CFA files and I have to remember to reset this for my Flats and Lights every time.
Does the bug fix "processing panel is properly refreshed when CFA properties changes" mean that the script will now remember that my files are CFA for future use? (The old BachPreporcessing script did this).
Thanks

Hi @tgainey,

I am not sure if your question means that you still experience that issue in version 2.2 or if you're just asking if version 2.2 fixed it, but the answer is the second, WBPP 2.2 remembers the CFA settings on exit and reloads them at the next launch.
If this is not the case please report more details regarding your WBPP configuration.

Robyx

 

[tgainey]

Hi @tgainey,

I am not sure if your question means that you still experience that issue in version 2.2 or if you're just asking if version 2.2 fixed it, but the answer is the second, WBPP 2.2 remembers the CFA settings on exit and reloads them at the next launch.
If this is not the case please report more details regarding your WBPP configuration.

Robyx

Robyx
My experience is that v2.2 does not preserve the CFA settings as you have indicated. I have run the script with one set of data and set CFA options on Light and flat frames. The next time I run the script these are turned off again. I have also tried dragging and dropping an icon to my desktop to try to preserve the CFA options and upon re-opening the script the CFA options are turned off again.
This is frustrating as the CFA options are hidden and you have to scroll to the right to enable them each time.
What more detail do you need?
Thanks

 

[robyx]

Robyx
My experience is that v2.2 does not preserve the CFA settings as you have indicated. I have run the script with one set of data and set CFA options on Light and flat frames. The next time I run the script these are turned off again. I have also tried dragging and dropping an icon to my desktop to try to preserve the CFA options and upon re-opening the script the CFA options are turned off again.
This is frustrating as the CFA options are hidden and you have to scroll to the right to enable them each time.
What more detail do you need?
Thanks

Ok, now it's clear.
Can you upload a screenshot of the control panel (the calibration tab) just before exit and just after reopening it? keep the window as wide as possible so I can see all columns for each group.
In case would you be willing to share the files you've added to WBPP for me to test?

 

[tgainey]

Robyx
I apologize for confusing you. I was using WBPP v2.1.2 not v2.2 and that version has the issue with not remembering the CFA settings.
I had assumed I was using the latest one you advertised on Sept 3rd as my Pixinsight system still says it is up-to date at 1.8.8-8 so I guess I need to wait for 1.8.8-9 to be released to get your latest script? I cannot see any other way to update to the latest script until this happens.