75% pixel rejection during ImageIntegration of DSLR bias frames?

[chris_todd]

So in an attempt to fully understand the BatchPreprocessing script (and the DSLR_RAW workflow_), I attempted to run through the process manually.  I converted my RAW bias files to FIT, then loaded them into ImageIntegration using (almost) all the settings detailed in Vicent's tutorial.

The output I get seems to suggest that about 75% of the pixels in some channels are being rejected:
Integration of 20 images:
Pixel combination ......... average
Output normalization ...... none
Range rejection ........... range_low=0.000000 range_high=0.980000
Pixel rejection ........... Winsorized sigma clipping
Rejection normalization ... none
Rejection clippings ....... low=yes high=yes
Rejection parameters ...... sigma_low=4.000 sigma_high=4.000

* Using 806 concurrent pixel stack(s) = 975.65 MB

* Integrating channel 1 of 3:
Integrating pixel rows:     0 ->   805: 100%
Integrating pixel rows:   806 ->  1611: 100%
Integrating pixel rows:  1612 ->  2417: 100%
Integrating pixel rows:  2418 ->  3223: 100%
Integrating pixel rows:  3224 ->  3464: 100%

Pixel rejection counts:
/Volumes/LaCie/Astrophotography/Texas Star Party/MastersCalibrationFrames/MasterBias/WednesdayNightBias/IMG_0188.fit
    1 :  13524162  75.030% ( 13519820 +      4342 =  75.006% +   0.024%)
/Volumes/LaCie/Astrophotography/Texas Star Party/MastersCalibrationFrames/MasterBias/WednesdayNightBias/IMG_0198.fit
    2 :  13524188  75.030% ( 13519702 +      4486 =  75.006% +   0.025%)
/Volumes/LaCie/Astrophotography/Texas Star Party/MastersCalibrationFrames/MasterBias/WednesdayNightBias/IMG_0199.fit
    3 :  13524234  75.031% ( 13520075 +      4159 =  75.008% +   0.023%)

Does that sound right?

Edit:  Forgot to mention these were taken with an unmodified Canon T2i.

[ruediger]

The output I get seems to suggest that about 75% of the pixels in some channels are being rejected:

Your "RAW Format Preferences" are "Create RAW Bayer image (RGB image)", so you have 75% black pixels in the Red and Blue color plane, which all get rejected during ImageIntegration.

The new Batchprocessing script instead uses "RAW Bayer CFA image (monochrome)", so the rejection values are much smaller.

Monochrome RAW images save a lot of hard disk space (and time for reading/writing) and still lead to the same results. So you might change your RAW Format Preferences (or use "input hints" when possible)

Rüdiger

[chris_todd]

The output I get seems to suggest that about 75% of the pixels in some channels are being rejected:

Your "RAW Format Preferences" are "Create RAW Bayer image (RGB image)", so you have 75% black pixels in the Red and Blue color plane, which all get rejected during ImageIntegration.

The new Batchprocessing script instead uses "RAW Bayer CFA image (monochrome)", so the rejection values are much smaller.

Monochrome RAW images save a lot of hard disk space (and time for reading/writing) and still lead to the same results. So you might change your RAW Format Preferences (or use "input hints" when possible)

Rüdiger

Thanks, Rüdiger, I'll try the format hints tonight.

[chris_todd]

Thanks, Rüdiger, that worked (and saved me several hundred megabytes of disk space!)