donut disaster

[astrodoc71]

Just wondering if anyone has discovered a way to eliminate dust donuts after the fact, meaning after a failed calibration. I understand the right answer is to recalibrate, take new flats etc and I am working on that but curious if there is a way to fix this afterward. Unfortunately DBE does not fix them. I have tried a million permutations and combinations. Interestingly the closest I have come to fixing it is in the non-linear stage, extracting individual layers and doing pixel math on a larger structure layer, then recombining. Problem is that data is clipped with this approach. Unfortunately I believe that in my frustration I inadvertently deleted the file so I don't have it to post, but if anyone has discovered a general approach to the problem that has been effective for you I would like to know what that is.
Thanks
Dave

[MikeOates]

Dave,

I would have thought using the most recent flats you have should go a long way to sorting them out. In my experience the dust does not seem to change much or at all from night to night. But I suppose that could be very different if the system is changed a lot. Worth a try?

Mike

[astrodoc71]

Hi Mike,
Actually I did try to increase and decrease the flat brightness of the flats I have and that didn't seem to help. What happened originally was I got white donuts in the calibrated image. That to me would imply my flat is too dark. They are not embossed so I believe rotation and focus are OK but I would have thought that changing the brightness would have helped. It didn't seem to. I tried to do this using simple pixel math addition to brighten and also tried subtraction to darken. Maybe that isn't the right way. I tried testing the results of brightening or darkening the flat with division of single raw image as a simple test without normalization. I would think this would give me some hint I was on the right track even though it is not exactly correct operation??
Thanks
Dave

[astroman2]

Hi,

I have made artificial flats from the star images themselves. Depending on the field this can give acceptable results.

Regards
Scott

[gvanhau]

Hello Dave

I've found a method that usually helps me when Flat frames are doing overcorrection or under correction (meanly a scaling problem).

The trick consist on doing a linear fit between a half calibrated light frame and the master flat.
I will write e tutorial on this as soon as I have the time, but the procedure is as follows:

If your frames are mono do the following:
1.- thake one of the light frames before calibration.
2.- using pixel math substract the the master Dark (and master Bias) from the choosen light.
3.- now apply a linear fit to your master Flat using as reference the image resulting form 2
4.- use the new master flat (resulting fom 3) in your normal calibration of lights.

If your frames are osc (bayer color)
Then the procedure is similar, but you have first to split the light frame, Master Dark,, Master Bias and Master flat into 4 individual images (using splitcfa) apply the procedure to the four parts and recombine the four resulting flats parts into one full flat and use this result in your normal calibration.

Note this does not hep if during the session the dust has moved, it only helps if there is a scaling problem.
I dont know yet why this is hapening, but for me it helps.


Regards
Geert

[astrodoc71]

Thanks guys.
This is an interesting idea Geert. I will give this a try. When I went back to try and retake the flats it looks like the dust has moved so I am stuck working with the original flats if at all possible.
Regards
Dave

[dnault42]

I've fixed these two different ways.  One is using the light frame data to create a synthetic flat which is then used to calibrate the image(s).  I've got a tutorial on that here.  The other is to model the donuts with PixelMath and then subtract them out similar to what I've done here with large halo removal.

Regards,
David

[jkmorse]

Geert & David,

Great ideas! Thanks,

Jim

[astrodoc71]

Thanks David,
More complex but clearly effective!
Regards
Dave

[jkmorse]

Geert,

Had a chance to use your technique on a set of flats that had been giving me lots of trouble and it solved my issues perfectly.  Thanks for the idea!

Best,

Jim

[VoidPointer]

Well, this topic is over 3 years old but...

I just ran into the same Problem and luckily discovered your post pretty quickly after a google search for overcompensating flats.
I used what you described for monochrome flats on flats/lights from a CFA color camera (before debayering anything) and it worked just well without splitting up anything into individual channels beforehand. The resulting flat after linear fitting the overcompensating master-flat to one of the lights was very good.

Thanks for sharing that here. It helped me a ton.

Any idea why this is happening? I never had this happening before and I have not changed anything in my workflow...

Cheers!
Lars

[dnault42]

Hi Lars,

I'm glad the process worked for you.

Flats can go wrong several ways:
- Something changed in the optical setup, like new dust has accumulated that wasn't accounted for between when the flats and lights were taken.  This could also be something like the orientation of the camera relative to the optical system has changed, although if the problem is specific to dust donuts then orientation isn't the most likely cause.
- If not all your temperature, ISO and binning settings changed between flats, lights, darks, etc.  Some cameras are affected by ambient temperature differences even if the sensor probe registers the same value between lights, flats, and darks, assuming the camera has temperature regulation.  Cameras that don't have temperature regulation are very difficult to calibrate well and most people rely on building libraries of darks and bias at various temperatures and ISOs so they can select what matches any given flat or light the best.
- The flat field device used provides uneven illumination.  If you were collecting good calibration data before and suddenly these are off then it is possible that something went wrong with the flat field acquisition setup.

Regards,
David

[VoidPointer]

David, thanks for you reply...

I went back and checked all sorts of things (thanks for the detailed list of culprits).

Even though I expected that this would remain a mystery, I did find the error eventually:

The master dark I was using for light frame calibration was not containing the bias signal. I think this happened because I failed to turn off normalization in the ImageIntegration process when integrating the dark-frames. After re-creating that master-dark with no normalization, things turned out just well...

Cheers,
Lars