Flat frame over-correcting?

[gregwjones]

I am just restarting my interest in this hobby after a very long gap. I started using PI just a few months ago and am running into a problem when I apply flats to my light frame calibrations.

After I run Image Calibration my light frames are bright white in the corners. The original light frames do have significant vignetting because I am using a C8 with a .63 reducer/corrector and a full-frame Canon DSLR. I am using a LED artist tracing light-box for exposing the flat frames. My flat frames have a mean of 7500 ADU (14-bit space). The MasterBias has a mean of 2048 and the MasterDark has a mean of 2049.

Is this normal for frames with significant vignetting or can I do something to fix it?

The calibrated light frames look fine (but with significant vignette) when processed with just a MasterBias and MasterDark.


Thank you in advance for any help.

[pfile]

usually when this happens the problem is with the calibration of the flat subs. how did you calibrate the flats?

rob

[gregwjones]

The 25 flats were first integrated using Average, Multiplicative and with the rejection set to Percentile Clipping, Equalize fluxes to create an uncalibrated MasterFlat.

Then the 25 light frames were integrated using a MasterBias, a MasterDark (uncalibrated) but its "Calibrate" box was checked during the integration and the uncalibrated MasterFlat from above with its "Calibrate" box checked during the integration.

This is shown in the attached screenshot.

[pfile]

the flats are pretty short; what i would do is calibrate the flats separately using only a master bias frame, then integrate the calibrated flat subs and use that master flat when calibrating the lights (but uncheck calibrate in the flats area, of course.)

with such a disparity between the short duration and the dark duration you can run into problems. to be honest i've never done it the way you are doing but i can imagine that ImageCalibration may find no correlation between the flat subs and the master dark and just subtract the master dark without scaling it, which i think would definitely result in mis-calibration of the master flat.

rob

[dld]

I agree with Rob, your flats seems too short and this may cause the problems described at http://www.myastroscience.com/proper-flats-with-dslr under "Choosing ISO for Flats". Since you're using an artist's tracing light-box, use a sheet of paper to dim it further in order to increase the exposure time. And be sure that there are no other light sources around

[pfile]

yes this is a very good point - with such fast shutter speeds you can start to see either shutter artifacts or flickering artifacts from your panel. it is probably a good idea to dim the panel somehow (as dld has described, using paper) and lengthen the exposure.

rob

[gregwjones]

I reprocessed my flats as Rob described and it did improve things. The corners are no longer bright white but now are speckled with bright white and very dark pixels.

I reviewed the batchstatistics for the original flat frames and the mean value ranges from 5500 to 14000. So, I will be collecting new flat frames with reduced intensity and longer exposures. (thanks dld)


Does the rejection algorithm and normalization for Flats have to be percentile clipping and equalize fluxes?

Since I have 25 flat frames can I use linear fit clipping to create the MasterFlat? If so, does the normalization have to remain equalize fluxes?


Thank you for the help.  I will report back with the results.

[pfile]

supposedly 'equalize fluxes' is the only right normalization method for flats. however, the rejection method can be whatever you like (which is really just dependent on the # of subs.) i think you can use linear fit or windsorized sigma clipping with that # of frames. but since flats aren't dithered/registered if there are any hot/cold pixels in them they probably will survive whatever rejection method you use as they will sit right on top of one another in each frame.

rob

[pfile]

one other thing i noticed - your format hints pane is not visible - did you use the "raw cfa" hint or do you have the DSLR_RAW handler set to return undebayered, raw images? if DSLR_RAW is debayering the images then the calibration will never be optimal.

rob

[gregwjones]

I do have the format for RAW set to "Pure RAW". So, my input frames are not debayered.

[gregwjones]

I collected new flats last night and the results are not much better. The 25 flats are at least consistent with each other now. The exposure was 1.25 seconds and the range of mean values is 7600 to 8100 ADU (14-bit). I created a calibrated MasterFlat with just the MasterBias substracted from the individual flat frames. I then calibrated the 25 light frames with the MasterBias and the MasterFlat. I did not use a MasterDark because the exposure for the light frames was only 60 seconds and Canon does some in-camera processing of the data before writing the raw file.

I think the vignette on my optical train is just too severe to be fully corrected by flat frames. The corners in the raw light frames have an ADU value of less than 10 above the Bias offset. So the corners are basically full black. I think the C8 I am using was never intended to fully illuminate a full-frame DSLR sensor.

With this severe vignette and the other optical aberrations away from the center, I intend to crop the calibrated light frames down to the size of an APS-C sensor. I will use an Image Container and the Crop process to do this.

I will continue to try to use these flat frames to correct just the center part of the frames.

[Stu]

IN terms of dimming the light box I have an Aurora flat panel that's too bright.  I just take 2-3 pieces of plain paper and put it over to dim it enough so I can get a flat at least 0.5 seconds long