Trouble with 3d Doughnut

[Lightpath]

Hello everyone-

I'm new to Pixinsight but I must say I am thoroughly enjoying it.  I've got a very strange problem with one of my classmate's images, however.  I have this odd 3d doughnut that I can't seem to process out.  I'm following Richard Bloch's amazing youtube tutorial.

This is the final processed image, , you can see the 3d doughnut in the upper left middle:

http://imgur.com/RdHsBW2

Here is the master flat with 13 flats, 7 darks, and 25 biases:

http://imgur.com/FbxJZI7

Here is one of the original frames:

http://imgur.com/UFwhPSx

Does anyone have an idea of how to process out the doughnut, or how I should alter my processing to deal with this?

This was taken by a classmate on our school's TEC 180.  All of his longer exposure images show this.  I have pixinsight so I'm trying to help figure out a way to deal with the doughnuts.  We can't touch the equipment to clean it.

thanks very much everyone-

Mark.

[Nocturnal]

Hi,

Looks like your flats don't match your lights. Given how shockingly dirty your flats are it's actually a bit of a surprise the image came out this good :-) Looks like whatever boulder caused that large disk moved a little between imaging and taking flats.

You said the equipment can not be cleaned by you but I strongly recommend showing this flat to whoever maintains the gear. It is unacceptably dirty, in my opinion anyway :-) Hopefully they will clean the cover glass of the camera and the filters.

[Lightpath]

Ha!  Yes, shockingly dirty is a good way to put it.  Usually it takes spoiled data like this to get things cleaned.  The techs are excellent but very conservative.

So...  So here's the problem-

The original flats were taken BEFORE he focused.  Then he focuses the system after the observatory opens.  Does this not basically ruin the original flats in the first place?  I keep telling everyone we should be taking flats AFTER we focus, but that would waste acquisition time, and no one is willing to do that.  Am I wrong in this?  He imaged for hours without re-focusing.

Then I came along mid way through the night, re-focused the system, and imaged a target for an hour, and turned the system back over to him.

Since I re-focused, and when he took over he did not change the focus back to where he had it, he says his data is completely ruined and I am at fault.  So I'm trying to find some way to process the images so he isn't so mad at me.  *I* don't think it's my fault, but I would rather win his respect by helping him than simply telling him I don't agree with him.

For the record the focus changed 76 steps.  On the feathertouch 3" focuser that's not much.
 
Any help here would be appreciated.  Is it possible that creating an artificial flat might help?

[Lightpath]

By the way-  the sub I posted was unregistered, that's why they don't match up perfectly.

[Nocturnal]

Well "it depends" applies. I have no idea how much 76 steps is but it sounds substantial. The reason I think that piece of dirt moved is that the other artifacts were corrected properly.

In any case, flats do not really correct for dirty optics. Once your signal gets reduced by a speck of dirt it is gone forever. In broad terms the defect can be corrected by multiplying the dimmed pixels so they match their surroundings but the lost signal is gone. Noise will also be multiplied and depending on how dark the artifact is this can remain visible even after flattening. It is for the same reason we try hard to center our targets because that is where the most signal will fall on the sensor. Flattening will give the illusion of even illumination but that is all it is, an illusion.

My advise remains to show the flat to the techs and ask for the optics to be cleaned. Then you should also do the flats with the system focused properly. Note there is no need to take flats again when the scope is refocused during imaging due to temperature fluctuations. As long as the image is in focus the path for the light is identical. Well perhaps vignetting can change a little when the focuser length is changed to compensate for a change in length of the OTA but those are details.

If you really want to know how critical your flats are simply take a few at different focus points and flatten one with the other. If they are equal you end up with a blank image.

[Lightpath]

I am totally going to show the image to the techs.  I'm sure seeing this result they'll clean the optics.  We had the same issue on the 20" RC, and the tech cleaned up the optics (at least the filters and the sensor window) and now we don't have any of these problems with the 20" RC.  but we had EXACTLY the same problem-  A huge doughnut that would not correct out.

How would you advise going about attempting to "flatten" the field?  Is the data useless now?

My previous calculations show that the focuser moves 1.25 microns per step, and that the cfz is 122 microns wide.  That translates to a CFZ that's 97 steps wide.  So if the focus changed by 76 steps it may, or may not, have moved out of the critical focus zone.

Mark.

[Nocturnal]

In some cases the data is simply toast. For me this would be one of those cases but perhaps someone has some tricks they can share.

[Lightpath]

Ok, thanks for your insight and help.  If anyone has any ideas, like maybe photoshop, I'm all ears. 

[Geoff]

By the way-  the sub I posted was unregistered, that's why they don't match up perfectly.

You didn't register subs before calibration I hope? Subs must be calibrated before registration.

[Lightpath]

By the way-  the sub I posted was unregistered, that's why they don't match up perfectly.

You didn't register subs before calibration I hope? Subs must be calibrated before registration.

Correct.  Subs were calibrated, then registered, then stacked.

Mark.

[Geoff]

In that case I'm inclined to support Sander's suggestion that the dust mote may have moved between taking flats and taking lights. The peculiar 3D look could perhaps be explained that way (dark on one side, light on the other).  If the dust mote has moved, there would be different calibrations applied on opposite sides of it in the direction of motion.

[pfile]

david ault has this tutorial here about making synthetic flats: http://trappedphotons.com/blog/?p=756

obviously with a flat like yours this is going to be hard. but maybe you can somehow model just that one giant spot and merge the real flat back with the synthetic flat. not going to be easy...

rob

[msmythers]

Just to show that you can sort of repair this type of problem without taking the final image to another piece of software here is what I was able to do quickly in PI. I used PixelMath, TGVInpaint, Rangemask and Curves. A little more care on my part and the repair could be better.


Mike

[Lightpath]

HOLY CRUD that's good!

I'll try going through the processes you mentioned, and reading up on them.


Can uh...  You teach me how to do that?



thanks for showing that it can be done, you're awesome!

[msmythers]

Thanks.

I don't think there is an easy way of teaching this. It's taken me 5 years to get to this point but I'm a slow learner.


Mike