I read the article and the most scholarly paper to which it links, here:
http://ericfossum.com/Publications/Papers/2014%20JEDS%20Review%20of%20the%20PPD.pdfI have to say that I consider Clark's conclusions to be largely erroneous. It appears to be based on a fundamental misconception of the dark current suppression issue that affects Canon DSLRs.
This issue was identified by Craig Stark, documented here:
http://www.stark-labs.com/craig/resources/Articles-&-Reviews/CanonLinearity.pdfI have reproduced his results through my own tests documented here:
http://www.blackwaterskies.co.uk/2015/02/pixinsight-dslr-workflow-part-2a-dark.htmlTo summarise, Clark's proposal is that dark current suppression technology is a relatively new introduction to DSLR sensors and that it is responsible for the improvements in (apparent) reductions in dark current and the accompanying dark current noise that has occurred between approximately 2005 and 2014. Clark dismisses the idea that dark-current suppression in these cameras is implemented in firmware/software.
Clark reaches these conclusions by reference to largely visual comparisons of dark frames from Canon 1DMkII and 7DMkII cameras as well as some fairly dubious techniques of histogram stretching in Photoshop rather than by sound statistical methods.
My view:
- The pinned photo-diode (PPD) that Clark claims to be the magic technology here has been in production use in DSLR sensors since at least 1995. No doubt incremental improvements have been made over time, but looking at the literature the implementation doesn't appear to have dramatically changed between 2005 and 2014. This technology does indeed reduce dark current and dark current noise in the sensor elements, but nowhere is it claimed that it eliminates these issues entirely except (by implication from his overall article) by Clark.
- Clark contradicts his own assertions that dark current suppression by software/firmware is not used and is not necessary in modern DSLRs. He refers to the ability of Canon cameras to take a second exposure of equal length with the shutter closed and subtract it from the light frame on-camera to reduce dark current, clearly acknowledging that dark current and accompanying dark current noise still exist, and that software suppression techniques are available as an option.
- The tests performed by Stark (which anyone can reproduce) show that (for the Canon cameras tested) apparent dark current reduces with increasing temperature and/or exposure time (contrary to what would be expected), but that dark current noise increases. PPD technology reduces generation of dark current and thus the dark current noise that accompanies it; it does
not "subtract" the dark current in any way. The only reasonable conclusion is that remaining dark current (after accounting for the PPD improvements)
was accumulated but has later been suppressed since the accompanying dark current noise remains in the image.
- Clark claims that it would be impossible to suppress the dark current in firmware/software without having a large library of calibration frames stored on camera for different exposure lengths and temperatures. This is arrant nonsense, as an optical black area (a set of masked off pixels at the edge of the sensor) can be used to measure the accumulated dark current for each light frame. It might be that a single factory-created "master dark" is stored in the camera firmware and scaled using the optical dark measurements , or it might more simply be an offset calculated from the optical dark that is subtracted globally from each pixel.
I don't think anyone is disputing that the quality of sensors and the accompanying camera technology has improved over the years, leading to cleaner images with less noise, but to pin it solely on a 20+ year-old piece of technology and dismiss any suggestion that manufacturers use firmware to clean up images is not credible.
What does this mean for DSLR users trying to calibrate their light frames? I reached the same conclusion as Vincent: Do not assume anything and perform your own tests for your particular model of camera. The firmware will vary between manufacturers and between models over time, so there is no hard and fast "rule" that can be applied to all DSLRs.
My rules of thumb are that after performing your own tests:
- If your CMOS camera behaves normally (i.e. dark current and dark current noise increase proportionately with time/temperature) then creating and subtracting a master dark frame will work.
- If your CMOS camera suffers from repeatable banding, amp glow or fixed pattern noise, then you should turn off PI's dark frame optimisation and use a time and temperature matched master dark. Otherwise it will likely over or under-correct and those artifacts will only be partially corrected (or indeed made more visible). This is certainly the case for my CMOS-based ZWO ASI1600MM-cool. A temperature and time-matched master dark without optimisation works very well and deals with the amp-glow that becomes visible and increases after about 60 seconds of exposure. With optimisation turned on, PI under-corrects by applying a scaling factor of about 0.45 despite a matched dark frame. This is unsurprising since the optimisation process is global and I expect it is being thrown by the significant variation across the frame.
- If your CMOS camera suffers from non-repeatable banding or similar issues (as my Canon 500D does), then all bets are off. Dark frame subtraction is likely to be problematic and just as likely to make your images worse rather than better. Personally I found that lots of subs, large dithering scales (15 pixels plus between frames), subtraction of a master bias made from 300+ subs to deal with repeatable fixed pattern noise and judicious use of CosmeticCorrection and the CanonBanding script worked for me.
