No bias calibrated flat - almost complete banding elimination

[astropixel]

It has taken some time to figure this out, but it always comes back to calibration. In this case possibly a little out of the ordinary, which may be of interest to canon dslr users.

Basically, the banding issue came down to using a bias calibrated flat. I created a flat with no bias correction, Convert raw to fit and integrate. The frames are the same used on the previous calibrated flat.

I wont go into detail, but here are the results. With a calibrated flat all the banding was in the R channel - with no bias correction slight banding appeared in the B channel.

[Carlos Milovic]

How many bias are you using? At which ISO setting are you taking the light subs and bias? Is the same ISO as the flats?

I'm using a new technique for bias frames, that was developed between Vicent and myself, called "SuperBias". It was supposed that he was going to publish some results from this technique... but that has been delayed. Maybe you'll see some improvements with it. I'll upload a windows 64bits release tonight for you to try. Ok?

[astropixel]

Thanks Carlos. ISO 800, light subs, bias and dark frames. Flats ISO 100 (acting on advice, flats were taken at a lower ISO). Perhaps this is incorrect with PI? I think that is the traditional approach to flats.

Now that I think of it, I used to take flats at the same ISO as lights and don't recall experiencing problems.

I will look for the download.

[georg.viehoever]

Hi,
I did see these issues with flats, darks, biases and lights all taken at the same ISO 800.
Georg

[Carlos Milovic]

If you are using flats of different ISO, then I should recommend taking flat-bias (ie. bias frames at ISO 100 just for your flats). Always take more than 20 frames and average them, without any normalization. You may use a sigma rejection, but I don't think it is too critical.

[astropixel]

Thanks Georg and Carlos. That of course makes sense. I had not considered that bias might be function of ISO. Perhaps not, if it were possible to take a zero time image, but at 1/4000th it's not really zero time. OK, I'll give that a go and take some 100ISO bias - good experiment anyway.

I must admit to being a little puzzled by pixel rejection associated with master frames production. There is no necessity with only a few frames but with 30 - 50 Windsorized seems to be the best choice. Is it absolutely necessary

[pfile]

i agree with others in the thread - i have seen banding in my flats (canon 50d - which of course also affects the calibrated frames as well). i have tried flats at the same ISO as my lights (800 or 1600) and also flats at iso 100. i always calibrate the flats with bias and darks of the same ISO, but this never made any difference. i never thought to *not* calibrate the flats and see what would happen.

i've mentioned this a few times but no one has ever commented: when i debayer a CR2 with DCRAW, the histogram is always much farther to the right than when i debayer a calibrated CR2 (really a fits file) with BatchDebayer. i can't help wondering if this has something to do with the problem. i think carlos or vincent has even remarked that my master flat was underexposed, but i think that's the result of the BatchDebayer rather than the flat being actually underexposed. the flats were metered with the camera, so they should be reasonably okay...

i should mention that the low histogram does not seem to be due to the calibration - if i debayer a CR2 directly with BatchDebayer the result is the same.

[Carlos Milovic]

A possible explanation to the low histogram values is that the raw data stores data with only 14bits. So, the higher values should be near 0.25... I may be wrong, since I have not worked with CR2 data, only old CRW from the 300D. And I have not seeing banding with it...

[pfile]

i suppose it could also be that DCRAW does not do a linear conversion when it debayers an image? i just don't know.

[Carlos Milovic]

I'm more inclined to believe that it rescales the data, and clips some of it. Now that I remember, when I was processing the solar eclipse images (cr2 files), I got way much information on the highlights by debayering manually, not by dcraw.

[astropixel]

Carlos, you are correct. No banding in RGB channels (observable to the eye) with a 100iso flat calibrated by a 100iso bias. Same flat frames as before - new bias.

EDIT: Based on a single calibrated light debayered and channel extracted.

My stack of 800iso lights is calibrated with an 800iso master bias and dark and a 100iso master flat (calibrated with a 100iso bias). What a mix!

Phil, I calibrated a single frame 3 or 4 tries with different combinations of master frames and deduced that the problem had to be with the flat. Comparing an STF master bias with an STF light highlighted remarkable similarities - holding one frame over the other showed the banding lining up through the transparent frame.

[astropixel]

May be I spoke to soon.

There is marginally more banding once all images have been integrated (calibrated flat) and it was necessary to use the banding reduction script.

Noticeable, whether using a calibrated or not calibrated flat the process of ABE, BN and CC produced the results expected. Very little processing required to get good results.

I did note that Windsorized clipping did a better job than linear with default settings for hot and cold pixel rejection.

[pfile]

A possible explanation to the low histogram values is that the raw data stores data with only 14bits. So, the higher values should be near 0.25... I may be wrong, since I have not worked with CR2 data, only old CRW from the 300D. And I have not seeing banding with it...

now that i think about this more, this makes sense. it could be that dcraw just scales the values to fit in i16 since dcraw is intimately familiar with the raw format. could be completely unrelated to the banding problem.

[georg.viehoever]

You could check this by taking a fully saturated picture with your Canon (e.g. 10 seconds by daylight), and then load the images into PI using the different RAW converter settings. If you see values that are different from 0.25, then indeed someone scales the RAW values.

Georg

[pfile]

You could check this by taking a fully saturated picture with your Canon (e.g. 10 seconds by daylight), and then load the images into PI using the different RAW converter settings. If you see values that are different from 0.25, then indeed someone scales the RAW values.

Georg

i messed around with this today and i found that if i overexpose a flat, the histogram peak is about 80% across the histogram display - this is when DCRAW performs the debayering.

i am still completely puzzled with these flats.

recently i picked up an EL panel to replace my lightbox. i just shot some flats with it through a canon 200L lens at f/4, at ISO100.

if i let the camera meter the exposure, it gives me 1/8th of a second.

through elimination i found that 0.6s would give me a histogram well over to the right, but not overexposed, again after debayer with DCRAW.

i made 50 flat subs and 50 dark flats subs and went thru the whole calibration process only to find severe banding in the flat again. tried again with uncalibrated flat subs, same problem.

i just don't understand how there could be so much banding at low ISO and relatively long exposures. i mean, i've seen banding in my bias frames but those are 1/8000 exposures. here there should be plenty of light and the signal should be way, way above the read noise.