Bug in BatchPreprocessing - subtracting wrong darks from flats

[bitli]

In some case BatchPreprocessing (v1.34) is subtracting the dark intended for the light from the flats, which is likely not what is expected.

In line 1453 [edit: of BatchPreprocessing-engine.js] we have:

Code: [Select]

IC.masterDarkPath = this.getMasterDarkFrame( binning, exptime );
IC.masterDarkEnabled = !IC.masterDarkPath.isEmpty();
For flats there is usually no adequate dark, in which case it takes whatever darks are present, usually the one prepared for the lights. This results in one of the messages at the beginning of the calibration of the flats (I guess nobody looks back so far in the log).

Code: [Select]

* Searching for a master dark frame with exposure time = 0.1s -- best match is a master dark frame of unknown exposure time.
* Searching for a master dark frame with exposure time = 0.1s -- best match is 421sYou get the former if the master dark was not done with BPP, in which case it has no exposure time, the later if the master dark as exposure time in its header.

Although it makes sense to use whatever dark is provided when calibrating lights, it is generally wrong to use the same ones for the flats. May be you can get away with this when using 'optimize', this is not correct in the general case.
The mechanism was probably intended to support flat darks, but it does not work if you do not provide them.

As a work around I added the line marked BITLI after 1458
 

Code: [Select]

  if ( imageType == ImageType.FLAT )
   {
      IC.outputDirectory = File.existingDirectory( this.outputDirectory + "/calibrated/flat" );
      IC.masterDarkEnabled = false;   // BITLI - Do NOT subtract dark from flat, as they may be the dark intended for lights
   }
A better solution would be for getMasterDarkFrame not to return any dark if: we are calibrating flats AND there is no matching dark within the tolerance AND (maybe) we are not using optimize AND we are also calibrating lights (the last condition to allow calibrating just flats with dark flats).

I suppose that because darks levels are usually very small compared to flat (and because the darks are scaled to a very small value or rejected if you use optimize) this had no obvious impact. However I had to calibrate very dark flats (no pun intended) and obtained different results from BPP and manual process, therefore my investigation.

-- bitli

[pfile]

along these lines i wish there was a way to make BPP calibrate flats with the master bias, as long as the flats are shorter than some threshold. my flats are always short and i've never done any flat darks; scaling my 1800s darks down to 2s does not make much sense to me.

i guess there's nothing stopping me from trying to make that change to the script, though.

rob

[bitli]

My correction would solve your problem, although it is unconditional. Check the log to be sure -- bitli

[bitli]

I think this problem is still present in 1.8.1-1092, by inspecting the code (I am short of time to make a real test).
-- bitli