Bias frames - pixel rejection - temperature regulated DSLR

[astropixel]

Hi.

I have a question about pixel rejection with temperature regulated DSLR bias frames - My question is, referring to the info below, do I need to bother with pixel rejection at all? What am I missing here?

The bias frames attached were acquired with a temperature regulated DSLR @ 0C. The camera is a cold finger Peltier mod regulated by Arduino PWM switching a MOSFET. Accuracy at best is +/- 0.2C at worse +/- 0.5C.

The frame on the right is a single bias frame straight out of the camera. The typical cross hatch can just be seen - if you look closely. k values range from 0.0035 - 0.0045 - mostly 0.0039

The frames on the left are from the same set (40 in total). The top frame was integrated with Windsorized Sigma Clipping. The bottom frame was integrated with No Rejection. k values are identical pixel by pixel. A broad sample across the frames found no variation - k values range from 0.0038 - 0.0042. There are no horizontal lines as in the single frame.

Noise analysis for the single frame σK = 5.183e-05, N = 8989064 (88.45%), J = 4

For the Windsorized stack - σK = 1.572e-05, N = 8668784 (85.29%), J = 4

For the No Pixel rejection stack - σK = 1.594e-05, N = 8542290 (84.05%), J = 4

I don't fully understand the data above. An explanation would be helpful.

Also I am not happy with the results that I am getting with integrated dark frames, but I will post separately.

Many thanks,

Rowland.

[astropixel]

Answering my own question, I cannot see an observable difference following calibration, with or without pixel rejection, except during integration of light frames, where rejection is necessary.

The most noticable reduction in image quality was found with bias subtracted dark and light frames - this is the PI default. Best results were had by integrating raw dark frames and calibrating the light frames with the uncalibrated master dark - however, flats must be bias subtracted, independently. The resulting increase in fine detail and image smoothness of the integrated light image is most noticable.

The issue seems to be bias integration, which is producing an unreliable masterbias with my cooled DSLR data set, adding noise to the image - what am I doing wrong? Consequently, I have resorted to applying the bias in the dark, rather than using a separate master bias.

EDIT: Images attached. Left side is bias subtracted frames as per Batch Preprocessing. Center and right side are as discussed - no bias and dark containing the bias, with low and default integration pixel rejection values for comparison.

[astropixel]

Well. I conclude that DSLR bias frames are problematic and perhaps non-essential. I will try taking dark frames at the same shutter speed as the flats and calibrate the flats separately.

I think the data destruction is self evident in the image on the left.

[pfile]

maybe with all this dark current modification they are doing you need an output pedestal while calibrating the darks? otherwise you get negative pixels which are truncated to 0…

rob

[astropixel]

Thanks Rob. I was going to mention bias pedestal - 100 is the usual in-camera addition - but earlier results were not promising. I suspect that cooling has unmasked an issue not so evident with uncooled images - I hadn't identified it as a problem before.

[Phil Leigh]

I run two "DSLR" cameras @ -25 or -30c.
Creating a superbias in Pi was very informative. You can clearly see that there is a pedestal of 2048 (16-bit scale) with the 600D for example... and I've also examined the bias frames from many other models of Canon DSLR and it seems that Canon Engineers change the pedestal on almost every model!

I always use my superbias to calibrate my flats and lights. I have not used a dark frame in over 2 years. There's no point because when properly cooled the dark is very close to the bias... and dithering handles the rest. Canon dark frames are not linear in the way that CCD frames are so the biggest benefit to me of having a cooled DSLR is being able to forget all about dark frames...

To put it another way, the bias frames are important and essential... darks frames are best avoided!

[astropixel]

What I'm seeing then, is dark frames that are predominantly bias, or so it seems. That is the visual impression anyway.

Imaging is at -5C, 0C or 5C, which is adequate for my purposes and a dithering routine that works very well.

I believe the Canon 1000D/XS has a bias pedestal of 256 and not 100 as previously stated.

Is it necessary to apply a literal pedestal value when using a superbias? And if not using a superbias, what value should I use. At a guess 256? But my experience with all of this has been somewhat negative - consequently, a bit hesitant to waste time playing around.

Superbias module v1.8 is yet to come from what I read?

[Phil Leigh]

Hi Rowland,
I did my "superbias" the hard way with 200 actual bias frames.
Looking at my master bias and darks for my 600D and 1100D they both show a pedestal of 2048 (16-bit scale) which would be 512 on the native 14-bit scale of the cameras.

I don't apply any pedestal subtraction given that all the frames I use (light/flat/bias) are raised by 2048 (512) so it is a constant. I think if you start subtracting the pedestal you can get into trouble if you end up with negative numbers which would presumably truncate to zero as Rob says.

If you open your bias or dark in Pi and look at image statistics in 16-bit mode (not normalised mode) you tell what the pedestal is from the mean value - it will be close to a power of 2 such as 1024 or 2048...

[Ignacio]

I have worked with a 1000D, a 450D and a 6D, all cooled to about freezing temperatures (at the sensor level). I have found that it is best to integrate darks into a master dark without calibrating them (no master bias substraction). Then, calibrate the master dark during calibration of lights. In this way you avoid/minimize truncation (when using 20-30 darks).

I always use a lot of bias frames to build a master bias (200), as you only do it once and they are easy to take. This is key to gte good dark optimization later on. Typically, I use a very mild windsorize clipping (4;6), and then get a superbias master with Carlos' script.

With this workflow I have gotten very consistent dark calibrations, which I always optimize, getting factors very close to 1 (two or three digits) when temperature matches. And grater / smaller than 1, consistent with temp differences.

Hope it helps,
Ignacio

[astropixel]

Thanks Phil and Ignacio. That explains my problems with bias pedestal literal values, whereas, typically, I would use the 0 default.

You present two slightly different perspectives. At temperatures of -25 to -30C, dark current is all but zero. Because I live in a relatively warm and often humid climate, near zero Celcius and slightly below is just within reach. Dark current is still an issue, but reduced. Because my camera is temperature regulated throughout an imaging session, I can count on consistency between frames.

have worked with a 1000D, a 450D and a 6D, all cooled to about freezing temperatures (at the sensor level). I have found that it is best to integrate darks into a master dark without calibrating them (no master bias substraction). Then, calibrate the master dark during calibration of lights. In this way you avoid/minimize truncation (when using 20-30 darks

).

I'm half way there, I think, but missing the superbias. EDIT: at -5C, I notice that bias and dark frame k values are almost the same.

My thinking on all of this, although I wasn't completely sure about what was happening in the camera, is that all frames except bias, possibly, get the same in camera treatment and should be consistent.

On another point, I understand that the batch preprocessing script applies a pedestal value, but the standalone image calibration module does not. This introduces a level of confusion. Where, how, is the pedestal value determined and how could that be manually applied to the image calibration module. Partly, my overthinking gets me into deep water. What am I missing here?

[Phil Leigh]

Hmmm.. I'm not aware of any pedestal subtraction in BPP... How would it know what pedestal to subtract and why would it do that anyway?

[astropixel]

That's what I said? Anyway, not being fully aware of the internal vagaries, I can't be sure.

[astropixel]

You are correct. No pedestal correction by BPP. In that case, is the pedestal corrected in the ImageCalibration module? As the default literal value is zero and the default setting, from memory is fits header, I think not and not my experience.

[bitli]

You may want to look at http://pixinsight.com/forum/index.php?topic=7006.15

The BPP erroneously subtract the dark from the flats, even if there is no dark flat  You can compensate this by creating a dark flat (copy a bias under the name 'flat' and do no calibrate the flat), but at this point it may not be worth using BPP for DSLR, IMHO.

The use of a PEDESTAL resolve the technical problem that PI (as many other programs) does not like negative values in FITS files, but it does not solve the problem that you will not get much correlation if you optimize a dark with negative values and that the bias are useless for dark and lights if you do not optimize.

-- bitli

[astropixel]

Interesting. I have followed convention and yet my thoughts agree with your post in several areas. DSLR images should not be considered as dedicated astro CCD images for the purpose of calibration - they are not the same. Bias frames are surprisingly repeatable and good flats are very important. It can be said of darks and lights that they are not unadulterated RAW data and I think they should be processed with identical settings - there is no basis for conformity to assume a predictable result, applying CCD calibration settings to DSLR frames. It seems that several agree about this.

I think I have resolved my limited understanding of the DSLR pedstal issue in a subsequent thread. Speaking about cooled DSLR frames the bias value is near the dark values. If a dark frame is needed/desirable, then a very minimal pixel math addition 0.0002 to the dark frame prevents truncation with bias subtraction.

I think the DSLR calibration thing is becoming clearer and pointing to a better method of calibration for DSLRs.