Unexpected results comparing manual, BPP and WBPP processing

Buzz

Well-known member
I have been doing a comparison of my manual calibration in PI using ImageCalibration on the QHY183 (mono CMOS) with the PixInsight BPP and WBPP scripts and found some anomalies which could cause unexpected issues. I have posted this in bugs, as I would expect the two scripts to implement fundamental calibration the same, using the same settings.

In each case I processed a single light file, so I could easily compare calibration results (no light integration involved)

In summary, in my simple experiment it appears that BPP and WBPP are not calibrating lights using the same rules (using equivalent settings).

I have a master calibration library and for the experiment I had already created Master Darks, Bias and Flats. Master Darks and Bias are simple stacks and Master Flats are calibrated with a matching Master Dark and the usual normalisation and integration. No bias subtraction or scaling was involved. (I use refractors and flats are good across several sessions)

Loading these masters, in BPP, I checked the corresponding master boxes, unchecked optimize dark frames, unchecked CFA and checked calibrate only.

Comparing my manual light calibration method (which only calibrates with Master Flats and Master Darks) with BPP created identical results. Interestingly, if I cleared the Master Bias file from BPP, the result was still identical (checked with image analysis tools)

I loaded up the same files (Master Flat, Dark, Bias and light) and settings in the WeightedBPP and disabled the additional setting 'calibrate master darks' too. The result was awful, heavily clipped to black.

If I remove the Master Bias and run WBPP script again, I achieve the same result as the manual method / BPP script. I started afresh, repeated the comparison and got the same result.

This is a surprising and potentially significant result - which indicates that there are more differences between BPP and WBPP than just image weighting. As it appears right now, I would defer to BPP. One possible explanation is that WBPP bias-subtracts a master dark but BPP does not?

There is no documentation, so I tried to decipher the batch scripts but I'm more familiar with C#.

These scripts are useful for many, especially those starting out and it would be useful to understand them better and correct any potential issues.

If it helps, I put my masters and a light frame into dropbox: https://www.dropbox....ZWTvfM9lta?dl=0
 

Juan Conejero

PTeam Member
Sep 2, 2004
7,555
96
56
Valencia, Spain
pixinsight.com
The BPP script is now obsolete and will be removed in version 1.8.8-6. WBPP and BPP use different strategies and working parameters internally, so the differences you are observing are expected behavior.

WBPP is well described in these documents:

 

ngc1535

PTeam Member
Feb 1, 2014
532
68
AdamBlockStudios.com
Buzz,

Do not disable Calibrate Master Darks. You should have seen warning concerning this.
I expanded on the documentation that Juan relates above in a YouTube series on WBPP.
It has Roberto's unofficial endorsement (meaning, I really really tried to get things right
by working directly with him). Please review and see if things make more sense?


-adam
 

Buzz

Well-known member
Buzz,

Do not disable Calibrate Master Darks. You should have seen warning concerning this.

-adam
Thanks Adam - to clarify, does your suggestion also apply to when you already have a master dark that is perfectly matched in temp/duration to the image frames?

I found the video (Part 4), about 14 mins in. (They are really good BTW, I know it is not easy firsthand, as I have made a dozen or so during lockdown ).

Your video suggests a use of calibrating master darks is for when dark scaling is needed.
I fully understand but some of my CMOS sensor's mean bias levels exceed that of their dark frames. My concern is that I wish to prevent any Bias subtraction from a Dark frame during the calibration process. Even if it was being used for internal calculations - I think this circumstance would screw it up?
 
Last edited:

ngc1535

PTeam Member
Feb 1, 2014
532
68
AdamBlockStudios.com
When you have "Calibrate Master Darks" checked, it means you are giving WBPP a dark that contains the Bias. This is normal since a master dark is normally just a simply integrated set of raw darks with some form of rejection. WBPP will then do the right thing, subtracting the master bias only when necessary (e.g. optimization). If you are bringing WBPP an already bias-subtracted dark (why? no really... why is that important to do?) then you should uncheck the box.

So.. you are giving WBPP a master dark with the bias intact correct?

When you are using darks and flat darks with matching exposure times (with its bias in it) with raw flats and lights there is no mystery calculation that occurs (and the bias is not used either). The darks are straight up subtracted from the lights/flats. This is why I suggest this for CMOS as a good safe check since some CMOS sensors misbehave. Things get weirder when optimizing for these misbehaving sensors.

If you are giving WBPP a master dark that already has its bias subtracted- but raw lights and flats that have the bias- this can be problematic which is why I suggest load the raw darks into WBPP and let it create the master dark (with the bias). Due to the electronic signature of some CMOS sensors- you cannot simply subtract the "bias" from the flats and get good results.

Bottom line BPP and WBPP produce identical results- the math is equivalent. However, WBPP expands on the management of darks compared to BPP which is what is presumably causing the issue for you. If this doesn't make sense... you can provide the files.

-adam
 

Buzz

Well-known member
Adam - your explanation makes sense. I have nothing to fear. You brought to light another subtlety with CMOS sensors that short and long exposures may bring about different noise mechanism/contributions.
Incidentally, my masters are not bias-subtracted. (I recall that a master dark from MDL5 would annoyingly bias subtract)

On a slightly different tack- I am trying to help a colleague calibrate his Kepler4040. All exposures have a faint level change in the pattern of four quarters. This GPixel sensor however must have another noise mode since a 'perfect' calibration using flats, flat darks, lights and matching darks still has a residual four quarter shading. I suspect it is some form of exposure non-linearity that our classical calibration schemes cannot entirely cope with. I think these larger sensors use parallel processing modules - and so it is difficult to get perfect matching.

Anyway - many thanks for your replies and your excellent videos.
regards
Chris Woodhouse FRAS
 

ngc1535

PTeam Member
Feb 1, 2014
532
68
AdamBlockStudios.com
The Kepler 4040 is an entirely different beast. It is a dual gain double read sensor...that indeed, I was not able to successfully calibrate in my review of its data. I don't think this has yet reached the mainstream... once it does... someone will likely figure it out..but I do not think this is true yet.
-adam
 

Buzz

Well-known member
Yes - I think the consensus is to avoid the double gain and just go with one. Even so, all our tests were with low gain and it still is a head-scratcher. QHY is developing a camera with it too and it will be interesting to see if they have the same issue. I contacted GPixel themselves but our astro needs are so specific, their tech support could offer no further insights.