Flats Calibration Problem

[Hytham]

I have encountered a problem similar to this, but it was easily resolved by re-imaging the flats with a higher ADU count.  Unfortunately this time around, it is to no avail.

Here is what I'm encountering.  The following image is fully calibrated with 100 bias, 20 darks, and 20 flats and as one can see it's not exactly what I, or anyone, would expect.



I have tried re-calibrating using flats from a previous session, but unfortunately the result has been the same.

Individual files:

https://www.dropbox.com/s/90o3rl9slnmswf8/bias_0.05sec_-20.0C_1x1_frame1.fit

https://www.dropbox.com/s/weolw0np4xakxin/dark_1200sec_-20.0C_1x1_frame1.fit

https://www.dropbox.com/s/hvvh0cyxa6w77h7/heart-soul_OIII_flats_29.2sec_1x1_-...

https://www.dropbox.com/s/6aqxgpmtlxubiuv/heart-soul_HA_1200sec_1x1_-20.0C_OI...

Please disregard the "HA" in the name in the last link.  It's purely cosmetic (I copied the event from one to the next and neglected to change the name prior to start) and the fits header does in fact confirm I was using the OIII filter.

Thank you for any insight.

[Geoff]

Can't really help here, but I ran your frames in PI and got similar results.  Then I tried CCDStack and the result was much the same, so I guess it's a problem with your frames rather than anything you are doing in the processing.  I was surprised by the amount of vignetting in your original frame.
Geoff

[Graham]

The vignetting in the light frame is rather unusual. If I didn't know better, I'd say that your objective was dewed up.

[mads0100]

Someone mentioned synthetic flats on here but didn't tell us how to create them in PI. Can anyone enlighten me on the procedure? I have some similar problem due to some shifted dust...

Chris

[pfile]

wasn't the conclusion on CN that the flats are overexposed?

rob

[Hytham]

CN thought it was over exposed flats, but after retaking the flats at the suggested exposure length for my chip, the calibrated result is still the same.

I had two dew heaters running on medium power and the low for the evening was still 5 degrees above the dew point, but that still does not stop me from periodically checking for dew.  At the time of the meridian flip there was not a trace of dew on the objective. 

The reason for the significant vignetting is due to the .73x reducer that produces a 44mm image circle on a 52mm diagonal chip.  It's pretty heavy in all of my images, but looks like it's most pronounced when the OIII 3nm filter is in use.

[oldwexi]

Hi Hytham!
The image circle of your telescope is to small
for a proper flat.

Lets simplify what happens during calibration with flats,
the light is divided by the normalized flat, normalized simplified means
white will have the value 1.000 , black the value 0.000 ...
So, if you divide any pixel by 1 the value is unchanged
if your divide it by 0.5 the value is doubled.
If you have  in your flat an area in  the corner which
is near to black means has a value of 0.01
The value of the pixel in your light corner is multiplied by 100.
This should explain the bright corners in the calibrated
light. This is brought to more extreme using the OIII filter
which creates even more dark corners in the flat.
(And yes, you have heavily overexposed your flat to get the
corners in the flat less dark.)

How to counteract at this. You have to change the
profile of the flat, means increase the brightness
of the dark underexposed corners in the flat.

I did some correction on your flat to lift the brightness
in the corners and reduce the overall brightness.
I used process Curves in steps for this. Than i brought the overall
brightness of the flat down with PixelMath.
This flat than i used for the Calibration.
The calibrated light still showed a small gradient.
Running one step with DBE finally got me a still linear
single relatively flat light.

Find a screenshot about the actions here:
https://dl.dropboxusercontent.com/u/57910417/Correct_FLat_processes.jpg
Left the new flat, right to it the old light.
right to it the calibrated light, right to it
the light treated with DBE. The histogramm at the right border
represents the histogramm of the new flat.
Below you find the Curves process i used 6 times.

Find the new flat here:
https://dl.dropboxusercontent.com/u/57910417/f.fit

Find the process container for correcting the flat here:
https://dl.dropboxusercontent.com/u/57910417/Correct_FLat_processContainer.xpsm


Find the new Light as fits file here:
https://dl.dropboxusercontent.com/u/57910417/snew1.fit

Find the stretched JPEG of your single new calibrated light here:


Hope this helps

Gerald

[niteman1946]

Hi Gerald, you're always the teacher.

This is what I think you are doing:
1.  Complete the flat master in normal fashion (either calibrated or not).
2.  Apply S-Curve RGB/K curve to flat master as needed to minimize/eliminate corner darkening.  Season to taste/verify with histogram?
3.  Calibrate Lights with newly modified (and flattened) flat master (calibrate or not depending on what was done in no.1).


Mark

[oldwexi]

Hi Mark!
Yes, These are the 3 steps in general,
1.  Complete the flat master in normal fashion (either calibrated or not).
2.  Apply S-Curve RGB/K curve to flat master as needed to minimize/eliminate corner darkening.  Season to taste/verify with histogram?   YES all the time!!!
3.  Calibrate Lights with newly modified (and flattened) flat master
    First in the quick and dirty mode with PixelMath:
    (light - dark)  / ( Masterflat - bias)
again 2. and 3. until its the way i like it.

Gerald

[niteman1946]

Thanks Gerald,

I've been following Hytham's posts both here and on CN with interest.  My Atik 383L+mono suffers from the same problem when used with the Meade f/6.3 reducer.  More so with the luminance filter, and less so with the narrowbands.  But still enough with the NBs that i have to crop.

I'm looking forward to trying and perhaps using your technique, until i can fix my optics (if possible).

Mark

[Hytham]

Gerald, thank you very much for all of the work and the explanation.

What I find most intriguing about all of this is that this is not the first time I have used the OIII 3nm filter with the reducer on my Tak.  My previous attempts worked successfully when I imaged the Elephant Trunk Nebula with the exact same configuration - no change at all other than the object imaged.

I fail to understand how in one scenario, the flats are applied correctly to an image with the same image circle size, but it now fails in this instance? 

[oldwexi]

Hytham!
Do you have one sigle  raw uncalibrated Image of this elephant trunk nebula?
Would like to calibrate it with your actual flat which i have to search for the reasons
of having success and not having success.
Gerald

[Geoff]

The above results puzzle me.   Any problems in the imaging train should be faithfully reflected in the flat, so that proper calibration should result in an evenly illuminated image. The above conversation seems to show that there is a problem in extreme cases. This suggests to me that the convential methods of flat fielding are not exact, but only a good approximation to a "proper" calibration, which breaks down if pushed too far.
Geoff

[Hytham]

Gerald,

Yes sir I do and here is the link:

https://www.dropbox.com/s/nsqndyvauu3kse4/Elephant%20Trunk_1200sec_1x1_OIII_frame4.fit

Geoff ... I'm with you.  I always thought that the entire purpose of the flat was to correct these problems, but the logic here seems to be missing.   

[oldwexi]

Hi Hytham!
Thanks for providing the image for a test.
My result is - using your original flat you cant get good results.
You must have used with your elephant trunk processing another flat?

I did calibration of your image with the original and my corrected flat and made a screenshot.
https://dl.dropboxusercontent.com/u/57910417/calibration_elefant_trunk.jpg

Have a look at the screenshot from left to right:
left top your original flat, left bottom my corrected flat,
to the right middle your elephant trunk Image
right to it top the calibrated Image using your original flat
to the right bottom after additional ABE,
to the right top the calibrated Image using the corrected flat
to the right bottom after using additional ABE.

Also have a look at the statistic window which shows the analyses of your original flat
in conjunction with the Histogram it looks strange and my opinion is this flat cannot work without treatment.


Gerald