Author Topic: WBPP doesn't seem to recognise EXPOSURE keyword  (Read 2015 times)

Offline slang

  • Member
  • *
  • Posts: 60
WBPP doesn't seem to recognise EXPOSURE keyword
« on: 2020 January 05 14:52:49 »
Hey.

I'm attempting to try WBPP with xisf files created with NINA. NINA writes exposure time in fitsheader as EXPOSURE. WBPP 1.4.2 doesn't seem to observe this, and collates groups of images as "0 Seconds"

Attached screenshot shows several (120/180 Second) images being grouped as 0 Seconds.

Inspection of fitsheaders of these files shows EXPOSURE keyword, and other other older (fits, not xisf) files that 'work ok' seem to use EXPTIME.

Is it possible to update WBPP to use either EXPOSURE as well?

Cheers -
--
Mounts: Orion Atlas 10 eq-g, Explore Scientific G11-PMC8
Scopes: GSO RC8, Astrophysics CCDT67, ES FCD100-80, TSFLAT2
Guiding: ST80/QHY OAG/QHY5L-II-M
Cameras: Canon EOS 450D (IR Mod), QHY8L, QHY163m/QHYFW2-US/Astronomik LRGBHaSiiOii

Offline Philippe B.

  • PixInsight Old Hand
  • ****
  • Posts: 399
    • CIEL AUSTRAL
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #1 on: 2020 January 06 05:20:16 »
I got same kind of issue when I created my masterDark with manual way
The workaround was to write in the FIT HEADER the line "EXPTIME" and its value in secondsYou can use FIT HEADER menu, of for several files, I think you can use a "batch" method.

This is the fastest way.

The other way should to edit script and look for EXPTIME you could replace by EXPOSURE but this is just if you are familiar with javascripts  :tongue:
Anyway, it should be fine if the author could upgrade the script with both EXPOSURE and EXPTIME options to adapt to all types.


Offline slang

  • Member
  • *
  • Posts: 60
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #2 on: 2020 January 06 12:40:19 »
Hey Phillippe.

Thanks for that. I've tried a few options (I have 50 - 80 files I need to do).

There was a script for manipulating values in fits headers (can't locate it right now) but that didn't help with adding new headers, let alone creating a new header (EXPTIME) and setting it's value to be the same as what is stored in EXPOSURE.

Creating process icons and applying to an image container allows you to replace the entire fits header with a set of static headers values, but that of course erases most of what you want to keep.

There are several references to WBPP not reading EXPOSURE (including reply#12 on https://pixinsight.com/forum/index.php?topic=14218.0). Changing the filenames to include, to include EXPTIME_300 for example enables the smart naming to pickup the missing value. However, as both EXPOSURE and EXPTIME are defined and both used by other software, it seems very strange that WBPP only acknowledges one.

I might see about hacking the script - not sure about that though... my javascript foo is not high.

Cheers -
--
Mounts: Orion Atlas 10 eq-g, Explore Scientific G11-PMC8
Scopes: GSO RC8, Astrophysics CCDT67, ES FCD100-80, TSFLAT2
Guiding: ST80/QHY OAG/QHY5L-II-M
Cameras: Canon EOS 450D (IR Mod), QHY8L, QHY163m/QHYFW2-US/Astronomik LRGBHaSiiOii

Offline slang

  • Member
  • *
  • Posts: 60
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #3 on: 2020 January 06 13:12:10 »
Have had a look through the code - there is reference to EXPTIME as well as EXPOSURE, but I don't understand enough to work out why.

An alternative way is to use the 'add custom' in WBPP, and manually set the Exposure (filter and other settings)...

Still, this should be working properly with EXPTIME...
--
Mounts: Orion Atlas 10 eq-g, Explore Scientific G11-PMC8
Scopes: GSO RC8, Astrophysics CCDT67, ES FCD100-80, TSFLAT2
Guiding: ST80/QHY OAG/QHY5L-II-M
Cameras: Canon EOS 450D (IR Mod), QHY8L, QHY163m/QHYFW2-US/Astronomik LRGBHaSiiOii

Offline dghent

  • Newcomer
  • Posts: 9
    • Personal site
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #4 on: 2020 January 07 00:44:21 »
Hi all,

Yes, it would be very helpful if WBPP (and anything else in PI that attempts to grok the exposure length of a given fame) would observe that there are multiple FITS keywords that are possible here and try to understand them. The FITS 4.0 standard does not prescribe any specific keywords related to exposure length, but the document Dictionary of Commonly Used FITS Keywords that accompanies the standard on HEASARC's website states the following:

Quote
4.  Keywords that give the date or duration of the observation:

    TIME-OBS TIME-END DATE-END
    EXPOSURE EXPTIME TELAPSE ELAPTIME ONTIME LIVETIME

And specifically regarding EXPTIME and EXPOSURE:

Quote
KEYWORD:   EXPOSURE
REFERENCE: HEASARC   
HDU:       any
DATATYPE:  real
UNIT:      s
COMMENT:   exposure time
DEFINITION: The value field shall contain a floating point number giving
the exposure time of the observation in units of seconds. The exact
definition of 'exposure time' is mission dependent and may, for example,
include corrections for shutter open and close duration, detector dead
time, vignetting, or other effects. This keyword is synonymous with the
EXPTIME keyword.

KEYWORD:   EXPTIME
REFERENCE: NOAO   
HDU:       any
DATATYPE:  real
UNIT:      s
COMMENT:   exposure time
DEFINITION: The value field shall contain a floating point number giving
the exposure time of the observation in units of seconds. The exact
definition of 'exposure time' is mission dependent and may, for example,
include corrections for shutter open and close duration, detector dead
time, vignetting, or other effects. This keyword is synonymous with the
EXPOSURE keyword.

Additionally, when it comes to XISF files, the Instrument:ExposureTime image property should certainly be preferred over any FITS keywords.

Offline Armando

  • Newcomer
  • Posts: 22
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #5 on: 2020 January 07 11:19:23 »
Hi,

I think that editing WeightedBatchPreprocessing-engine.js @ line 798
to have

Code: [Select]
if ( !forcedExposureTime && imageType != ImageType.BIAS )
should fix the issue.

CS
Armando Beneduce

Offline dghent

  • Newcomer
  • Posts: 9
    • Personal site
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #6 on: 2020 January 07 11:33:44 »
I just looked into the source for WBPP, specifically WeightedBatchPreprocessing-helper.js

It's not looking at the FITS headers at all for this information - it's looking for the equivalent of FITS keywords in the file name of the image!  ::)

So for WBPP to work with this info, the script is demanding that people name their files a certain way, even if the metadata it is looking for is in the FITS keywords or XISF XML blob? I must admit I'm a bit surprised by this approach. Is there no way for the script to find what it needs out of the file's metadata?

Offline Armando

  • Newcomer
  • Posts: 22
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #7 on: 2020 January 07 11:38:56 »
As already stated the bug is caused by a wrong boolean expression (a missing parenthesis if you prefer) - WeightedBatchPreprocessing-engine.js @ line 798

CS
Armando Beneduce

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #8 on: 2020 January 07 11:52:20 »
I just looked into the source for WBPP, specifically WeightedBatchPreprocessing-helper.js

It's not looking at the FITS headers at all for this information - it's looking for the equivalent of FITS keywords in the file name of the image!  ::)

So for WBPP to work with this info, the script is demanding that people name their files a certain way, even if the metadata it is looking for is in the FITS keywords or XISF XML blob? I must admit I'm a bit surprised by this approach. Is there no way for the script to find what it needs out of the file's metadata?

you misunderstand the code, it does that if there are no fits keywords. like if you are using CR2 files... the only way to convey the important info about the exposure is in the filename.

rob

Offline dghent

  • Newcomer
  • Posts: 9
    • Personal site
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #9 on: 2020 January 07 12:30:31 »
Ah, I see now. Thanks for pointing that out.

Offline slang

  • Member
  • *
  • Posts: 60
Re: WBPP doesn't seem to recognise EXPOSURE keyword
« Reply #10 on: 2020 January 07 17:51:38 »
Fantastic Rob. Thank you very much for that - I eventually came to the wrong conclusion, and had even looked/duplicated that code segment. Of course, I just don't understand what I'm doing or what the logic was, and would not have realised that was the error.

Have edited my local copy and confirm that it now gets and uses EXPOSURE and EXPTIME from .xisf metadata, as would make sense.

Cheers -
--
Mounts: Orion Atlas 10 eq-g, Explore Scientific G11-PMC8
Scopes: GSO RC8, Astrophysics CCDT67, ES FCD100-80, TSFLAT2
Guiding: ST80/QHY OAG/QHY5L-II-M
Cameras: Canon EOS 450D (IR Mod), QHY8L, QHY163m/QHYFW2-US/Astronomik LRGBHaSiiOii