WBPP 2.4.3 Released

lukenash707

New member
Aug 9, 2021
3
0
Yep, I did include dark flats so perhaps that's why it doesn't want to use it. I think I've figure it out now. It will use the bias if I optimize the master dark for the lights and also if I don't include the dark flats. Since I took dark flats and the correct exposure darks, I guess I don't need the bias frames?

I've included the screenshots just in case I am still missing something
 

Attachments

bulrichl

PTeam Member
Nov 2, 2016
1,444
226
La Palma, Canary Islands
You are right, if you capture flat-darks and don't want to use dark frame optimization, you won't need bias frames. Alternatively: you can try to use bias frames for the calibration of the flat frames and see if the integration result differs at all. If not, you can skip the flat-darks.

Whether dark frame optimization is advantageous depends mainly on the sensor that is utilized in the camera.

By the way, you are using an outdated version of WBPP (2.4.1 / current version: 2.4.4) and PixInsight (1.8.9? / current version: 1.8.9-1). It is worthwhile to update.

Bernd
 

thomsdak

Active member
Dec 19, 2016
25
2

TomT

Member
Feb 23, 2022
5
0
Good morning. Been searching but haven't seen an answer.

I left my new Dell grinding away overnight on its first set of WBPP files and when I awoke it was looking for user intervention to combine reference files. I reviewed the selections then hit generate.

I later noticed in the attachment that no reference files or normalized reference files were located ????

Any thoughts? Perhaps WBPP kept processing awaiting input and it got to the stage where it needed the files but were not generated as it was waiting on me?
73150B41-65F5-4F3A-993A-F671865647E8.jpeg
 

robyx

PixInsight Staff
Jul 17, 2019
723
279
45
Zurich
Good morning. Been searching but haven't seen an answer.

I left my new Dell grinding away overnight on its first set of WBPP files and when I awoke it was looking for user intervention to combine reference files. I reviewed the selections then hit generate.

I later noticed in the attachment that no reference files or normalized reference files were located ????

Any thoughts? Perhaps WBPP kept processing awaiting input and it got to the stage where it needed the files but were not generated as it was waiting on me?View attachment 15019
hmmm I think you pressed Esc button to dismiss the interactive view, is this the case?
This is a small flaw in the execution since when the window is dismissed with the keyboard it actually does not keep the selected frame as reference frame, it selects it only if you click the "Select" button.

I'll fix it in the next release anyway.
 

heyjp

Member
May 15, 2015
12
0
So what exactly is a low resolution monitor? I have a maxed out 14" M1 Pro MacBook with 3024 × 1964 built-in monitor. When I open Pixinsight full screen then open WBPP 2.4.3 and 2.4.5, WBPP takes up most of the screen (see attached screenshot) but the file menus are condensed into a single pulldown. If I stretch the length of the WBPP window a tenth inch, all the menus expand. Lots of grey space at the bottom available.

Jim
 

Attachments

robyx

PixInsight Staff
Jul 17, 2019
723
279
45
Zurich
So what exactly is a low resolution monitor? I have a maxed out 14" M1 Pro MacBook with 3024 × 1964 built-in monitor. When I open Pixinsight full screen then open WBPP 2.4.3 and 2.4.5, WBPP takes up most of the screen (see attached screenshot) but the file menus are condensed into a single pulldown. If I stretch the length of the WBPP window a tenth inch, all the menus expand. Lots of grey space at the bottom available.

Jim
Hi @heyjp,
yep, the low-resolution concept is a bit misleading, what happens is that the WBPP window is initially laid out and if we detect that the auto-size spans beyond the available screen size we have to reduce it introducing the collapsed panels.
I have updated how this collapsing GUI works and in the next release you'll be able to manually enable\disable this collapsed GUI appearance, so hopefully, this will not disturb you anymore.
 
  • Like
Reactions: heyjp

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,693
682
58
Valencia, Spain
pixinsight.com
So what exactly is a low resolution monitor? I have a maxed out 14" M1 Pro MacBook with 3024 × 1964 built-in monitor. When I open Pixinsight full screen then open WBPP 2.4.3 and 2.4.5, WBPP takes up most of the screen (see attached screenshot) but the file menus are condensed into a single pulldown. If I stretch the length of the WBPP window a tenth inch, all the menus expand. Lots of grey space at the bottom available.

Jim
Your laptop's monitor has a logical resolution of 1512 x 982 pixels at standard Retina resolution. This is quite low and close to the minimum screen resolution required by current PixInsight versions (the minimum reasonable resolution is Full HD = 1920 x 1080 px).

This happens because Retina displays on macOS use a fixed scaling factor of 2:1. In other words, two physical pixels are used to represent one logical pixel, and this scaling is applied transparently to the entire graphical interface, including screen drawing operations, mouse event coordinates, touch events, window geometries, etc. What applications see and work with is logical pixels, not physical. This happens on all operating systems with high-dpi monitors, but on Linux (X11, Wayland) and Windows we can decide on scaling factors, while on macOS/Retina we are forced to work at 2:1.
 
  • Like
Reactions: robyx

jkovach

Member
Jul 12, 2021
13
3
Grand Rapids, MI
I love the new execution monitor, but I have a couple requests:

1. Show the elapsed time for the current process.
2. Show elapsed time for processes that finished with warnings or errors.
 
  • Like
Reactions: robyx

TomT

Member
Feb 23, 2022
5
0
I love the new execution monitor, but I have a couple requests:

1. Show the elapsed time for the current process.
2. Show elapsed time for processes that finished with warnings or errors.
Estimated completion time / duration would be nice as well. Thanks
 

sbellavia

Member
Feb 22, 2022
15
4
Hi,
This is a brilliant process, and I am very appreciative!
But I cannot get it to work when using multiple filters, with a monochrome camera, using the filenames that are written from APT.
In order to recognize a Dark-Flat, Flat, or Light, APT adds "_DF..", "_F", "_L..", etc. to each filename, near the beginning. For example:

Cygnus_Loop-OIII_DF_10073_2022-07-04_04-16-59_Bin1x1_0.34531s_G100__-10C

So if I have the Grouping Keywords to be "Ha", "OIII", "RED", etc., it will properly separate the groups, but will not calibrate the flats with the Dark-Flats, nor will it apply the created Master Flat to the Lights.
If I turn off the "Pre" on any of them, it then no longer separates the exposure times, and they all become lumped together.

After 2 days of many iterations, trying to get this to work, using very small sample sets of real data, the only solution I have come up with is to manually rename all the Dark-Flats and Flats to have an "_L" rather than the "_DF" and "_F" in the filenames. I usually use 6 filters with 30 Dark-Flats and 30 Flats for each, so that would be very time-consuming.

In the attached images, I have renamed the OIII Dark-Flats and Flats, but not the Ha.
As you can see, the OIII will work properly, but the Ha will not, as I did not rename any of those files.

Another option that works is to create the Masters, before going into WBPP, and only changing the names of those to all be "_L" for each group. Perhaps less labor-intensive, but not ideal, and could lead to possible mistakes.

Any help is appreciated!
Thank you
Steve
 

Attachments

Last edited:

robyx

PixInsight Staff
Jul 17, 2019
723
279
45
Zurich
Hi,
This is a brilliant process, and I am very appreciative!
But I cannot get it to work when using multiple filters, with a monochrome camera, using the filenames that are written from APT.
In order to recognize a Dark-Flat, Flat, or Light, APT adds "_DF..", "_F", "_L..", etc. to each filename, near the beginning. For example:

Cygnus_Loop-OIII_DF_10073_2022-07-04_04-16-59_Bin1x1_0.34531s_G100__-10C

So if I have the Grouping Keywords to be "Ha", "OIII", "RED", etc., it will properly separate the groups, but will not calibrate the flats with the Dark-Flats, nor will it apply the created Master Flat to the Lights.
If I turn off the "Pre" on any of them, it then no longer separates the exposure times, and they all become lumped together.

After 2 days of many iterations, trying to get this to work, using very small sample sets of real data, the only solution I have come up with is to manually rename all the Dark-Flats and Flats to have an "_L" rather than the "_DF" and "_F" in the filenames. I usually use 6 filters with 30 Dark-Flats and 30 Flats for each, so that would be very time-consuming.

In the attached images, I have renamed the OIII Dark-Flats and Flats, but not the Ha.
As you can see, the OIII will work properly, but the Ha will not, as I did not rename any of those files.

Another option that works is to create the Masters, before going into WBPP, and only changing the names of those to all be "_L" for each group. Perhaps less labor-intensive, but not ideal, and could lead to possible mistakes.

Any help is appreciated!
Thank you
Steve
Hi @sbellavia, you were close to the simple solution :) write the explicit "dark" "flat" and "light" label instead of using the custom labels D, DF etc, this will let WBPP know what kind of file he's loading when he cannot infer it from the FITS header IMAGETYP keyword.

If you prefer, you may move your light/flat/darks into subfolders called "darks" "flats" or "lights" to obtain the same result.

Note that this works only in two cases:
1. you don't have the IMAGETYP keyword in the fits header or you have it but the content is "weird" (i.e. does not contain "light" or "flat" etc)
2. if you're not adding a custom image type (in such case your custom preferences have precedence)

Absolutely avoid matching frame types, filter names, and frame durations using grouping keywords since these are fundamental information that WBPP already handles internally without the need for any custom grouping keyword. They are handled on a completely different and deeper level with respect to grouping keywords and are foundational information for the WBPP behavior.

So, remove the grouping keywords and use the "dark", "flat", and "light" labels to fix the configuration.

Robyx
 
  • Like
Reactions: sbellavia

sbellavia

Member
Feb 22, 2022
15
4
Hi @sbellavia, you were close to the simple solution :) write the explicit "dark" "flat" and "light" label instead of using the custom labels D, DF etc, this will let WBPP know what kind of file he's loading when he cannot infer it from the FITS header IMAGETYP keyword.

If you prefer, you may move your light/flat/darks into subfolders called "darks" "flats" or "lights" to obtain the same result.

Note that this works only in two cases:
1. you don't have the IMAGETYP keyword in the fits header or you have it but the content is "weird" (i.e. does not contain "light" or "flat" etc)
2. if you're not adding a custom image type (in such case your custom preferences have precedence)

Absolutely avoid matching frame types, filter names, and frame durations using grouping keywords since these are fundamental information that WBPP already handles internally without the need for any custom grouping keyword. They are handled on a completely different and deeper level with respect to grouping keywords and are foundational information for the WBPP behavior.

So, remove the grouping keywords and use the "dark", "flat", and "light" labels to fix the configuration.

Robyx
Thanks!

I will give it a try.

But I already have all my darks, dark-flats and lights in subfolders with exactly those names, and it was not working?

And are you saying to NOT have "Red", "BLU", etc. in the naming groups? When I tried having no keywords, it lumped them all together, and did not separate based on exposures (and they all have different exposures for the flats and dark-flats).

I will try again, but I don't think I understand it. :/
 
Last edited:

sbellavia

Member
Feb 22, 2022
15
4
I am definitely going in circles here.
I tried using subfolders, since I do not know how to change the naming conventions in APT
And it lumped everything together, as it did before.
And it does not recognize the exposure for the Master Dark, which I have in a separate library.

I checked and this is the FITS header for a Dark-Flat:

IMAGETYP= 'Dark Frame' / The type of image

And it has the exposure, so I don't know why it is lumping them all together if I do not use the Keywords.

EXPTIME = 0.34531 / The total exposure time in seconds

I haven't a clue how to get this to work so I end up with what I need. :/
 

Attachments

Last edited:

robyx

PixInsight Staff
Jul 17, 2019
723
279
45
Zurich
I guess your FITS headers do not contain the necessary data. Again you have two ways, one is to fix the problem and configure APT to write the proper filter names into the fits headers (which is missing) or add "FILTER_HA" and "FILTER_O3" to your file names or use the smart naming to inject this information using a proper folder structure.

One example is the following (in your case it works because image type and filter name is missing, in other cases you need to check the "smart naming override"):

- bias
- dark
- flat
-- FILTER_Ha -> files are recognized as flat frames with "Ha" filter
-- FILTER_O3 -> files are recognized as flat frames with "O3" filter
- light
-- FILTER_Ha -> files are recognized as light frames with "Ha" filter
-- FILTER_O3 -> files are recognized as light frames with "O3" filter

separating files in subfolders caller FILTER_<your filter name> makes the WBPP attempt to recover the filter name from the file path if he cannot find this information from the fits hedaer. This should work.

Regarding the exposure time of your master dark, it means that the EXPTIME/EXPOSURE keyword is not stored in the FITS header. Again, you can fix this by either adding this information into the FITS header manually or adding to the file name "_EXPTIME_240" or "EXPOSURE_240" (both are valid) if your master dark is 240 sec.
 

robyx

PixInsight Staff
Jul 17, 2019
723
279
45
Zurich
I gave that a try. Still no good.
I will look into changing the naming structure from APT, but that will not help me with the 7 or so years of data I was hoping to reprocess.
disable the grouping keywords, you're really misusing them.
I think you're quite close to the solution :)
 

robyx

PixInsight Staff
Jul 17, 2019
723
279
45
Zurich
1. It is not separating the Dark-Flats, like it does the flats (though it does show the different exposures)
one consideration: darks of 0.18 and 0.35 seconds are indeed bias frames. you can add them as bias frames and WBPP will do the rest (it will use the master bias to calibrate the flat frames).

If you're willing to keep the two dark sets separated then reduce the dark tolerance to zero.

It will work in both ways.
 

sbellavia

Member
Feb 22, 2022
15
4
one consideration: darks of 0.18 and 0.35 seconds are indeed bias frames. you can add them as bias frames and WBPP will do the rest (it will use the master bias to calibrate the flat frames).

If you're willing to keep the two dark sets separated then reduce the dark tolerance to zero.

It will work in both ways.
Yep!
I just did that (and was editing my post, but not quick enough)
it works now!
I will now get in the habit of naming my subfolders as suggested

Thank you for all your time and effort!!
 
Last edited:
  • Love
  • Like
Reactions: sbenz and robyx