PixInsight Forum

PixInsight => General => Topic started by: Lightpath on 2017 April 19 13:00:42

Title: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 19 13:00:42
Hello everyone-

I'm new to Pixinsight but I must say I am thoroughly enjoying it.  I've got a very strange problem with one of my classmate's images, however.  I have this odd 3d doughnut that I can't seem to process out.  I'm following Richard Bloch's amazing youtube tutorial.

This is the final processed image, , you can see the 3d doughnut in the upper left middle:

http://imgur.com/RdHsBW2

Here is the master flat with 13 flats, 7 darks, and 25 biases:

http://imgur.com/FbxJZI7

Here is one of the original frames:

http://imgur.com/UFwhPSx

Does anyone have an idea of how to process out the doughnut, or how I should alter my processing to deal with this?

This was taken by a classmate on our school's TEC 180.  All of his longer exposure images show this.  I have pixinsight so I'm trying to help figure out a way to deal with the doughnuts.  We can't touch the equipment to clean it.

thanks very much everyone-

Mark.
Title: Re: Trouble with 3d Doughnut
Post by: Nocturnal on 2017 April 19 13:18:24
Hi,

Looks like your flats don't match your lights. Given how shockingly dirty your flats are it's actually a bit of a surprise the image came out this good :-) Looks like whatever boulder caused that large disk moved a little between imaging and taking flats.

You said the equipment can not be cleaned by you but I strongly recommend showing this flat to whoever maintains the gear. It is unacceptably dirty, in my opinion anyway :-) Hopefully they will clean the cover glass of the camera and the filters.
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 19 13:54:05
Ha!  Yes, shockingly dirty is a good way to put it.  Usually it takes spoiled data like this to get things cleaned.  The techs are excellent but very conservative.

So...  So here's the problem-

The original flats were taken BEFORE he focused.  Then he focuses the system after the observatory opens.  Does this not basically ruin the original flats in the first place?  I keep telling everyone we should be taking flats AFTER we focus, but that would waste acquisition time, and no one is willing to do that.  Am I wrong in this?  He imaged for hours without re-focusing.

Then I came along mid way through the night, re-focused the system, and imaged a target for an hour, and turned the system back over to him.

Since I re-focused, and when he took over he did not change the focus back to where he had it, he says his data is completely ruined and I am at fault.  So I'm trying to find some way to process the images so he isn't so mad at me.  *I* don't think it's my fault, but I would rather win his respect by helping him than simply telling him I don't agree with him.

For the record the focus changed 76 steps.  On the feathertouch 3" focuser that's not much.
 
Any help here would be appreciated.  Is it possible that creating an artificial flat might help?
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 19 13:59:30
By the way-  the sub I posted was unregistered, that's why they don't match up perfectly.
Title: Re: Trouble with 3d Doughnut
Post by: Nocturnal on 2017 April 19 14:16:31
Well "it depends" applies. I have no idea how much 76 steps is but it sounds substantial. The reason I think that piece of dirt moved is that the other artifacts were corrected properly.

In any case, flats do not really correct for dirty optics. Once your signal gets reduced by a speck of dirt it is gone forever. In broad terms the defect can be corrected by multiplying the dimmed pixels so they match their surroundings but the lost signal is gone. Noise will also be multiplied and depending on how dark the artifact is this can remain visible even after flattening. It is for the same reason we try hard to center our targets because that is where the most signal will fall on the sensor. Flattening will give the illusion of even illumination but that is all it is, an illusion.

My advise remains to show the flat to the techs and ask for the optics to be cleaned. Then you should also do the flats with the system focused properly. Note there is no need to take flats again when the scope is refocused during imaging due to temperature fluctuations. As long as the image is in focus the path for the light is identical. Well perhaps vignetting can change a little when the focuser length is changed to compensate for a change in length of the OTA but those are details.

If you really want to know how critical your flats are simply take a few at different focus points and flatten one with the other. If they are equal you end up with a blank image.
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 19 14:21:12
I am totally going to show the image to the techs.  I'm sure seeing this result they'll clean the optics.  We had the same issue on the 20" RC, and the tech cleaned up the optics (at least the filters and the sensor window) and now we don't have any of these problems with the 20" RC.  but we had EXACTLY the same problem-  A huge doughnut that would not correct out.

How would you advise going about attempting to "flatten" the field?  Is the data useless now?

My previous calculations show that the focuser moves 1.25 microns per step, and that the cfz is 122 microns wide.  That translates to a CFZ that's 97 steps wide.  So if the focus changed by 76 steps it may, or may not, have moved out of the critical focus zone.

Mark.
Title: Re: Trouble with 3d Doughnut
Post by: Nocturnal on 2017 April 19 14:25:58
In some cases the data is simply toast. For me this would be one of those cases but perhaps someone has some tricks they can share.
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 19 14:54:40
Ok, thanks for your insight and help.  If anyone has any ideas, like maybe photoshop, I'm all ears.  :)
Title: Re: Trouble with 3d Doughnut
Post by: Geoff on 2017 April 19 15:42:51
By the way-  the sub I posted was unregistered, that's why they don't match up perfectly.
You didn't register subs before calibration I hope? Subs must be calibrated before registration.
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 19 15:48:17
By the way-  the sub I posted was unregistered, that's why they don't match up perfectly.
You didn't register subs before calibration I hope? Subs must be calibrated before registration.

Correct.  Subs were calibrated, then registered, then stacked.

Mark.
Title: Re: Trouble with 3d Doughnut
Post by: Geoff on 2017 April 19 17:04:49
In that case I'm inclined to support Sander's suggestion that the dust mote may have moved between taking flats and taking lights. The peculiar 3D look could perhaps be explained that way (dark on one side, light on the other).  If the dust mote has moved, there would be different calibrations applied on opposite sides of it in the direction of motion.
Title: Re: Trouble with 3d Doughnut
Post by: pfile on 2017 April 19 17:15:58
david ault has this tutorial here about making synthetic flats: http://trappedphotons.com/blog/?p=756

obviously with a flat like yours this is going to be hard. but maybe you can somehow model just that one giant spot and merge the real flat back with the synthetic flat. not going to be easy...

rob
Title: Re: Trouble with 3d Doughnut
Post by: msmythers on 2017 April 19 17:50:01
Just to show that you can sort of repair this type of problem without taking the final image to another piece of software here is what I was able to do quickly in PI. I used PixelMath, TGVInpaint, Rangemask and Curves. A little more care on my part and the repair could be better.


Mike
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 19 21:48:25
HOLY CRUD that's good!

I'll try going through the processes you mentioned, and reading up on them.


Can uh...  You teach me how to do that?

:)

thanks for showing that it can be done, you're awesome!
Title: Re: Trouble with 3d Doughnut
Post by: msmythers on 2017 April 20 00:27:03
Thanks.

I don't think there is an easy way of teaching this. It's taken me 5 years to get to this point but I'm a slow learner.


Mike
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 20 09:20:19
Screen shots of your processes, or steps maybe, would give me a great starting point, however I'm still looking into the processes you said you used.  I may have more questions as I go.  :)

Thanks very much for showing what's possible-

Mark.
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 21 10:28:47
Just to show that you can sort of repair this type of problem without taking the final image to another piece of software here is what I was able to do quickly in PI. I used PixelMath, TGVInpaint, Rangemask and Curves. A little more care on my part and the repair could be better.


Mike

Do you have a hint as to how you created the initial mask with pixel math?

Thanks-

Mark.
Title: Re: Trouble with 3d Doughnut
Post by: msmythers on 2017 April 21 11:42:08
Mark

Here is the PixelMath end of this. I did not save a project of the previous data so I am having to do this all over again. Someone good with PixelMath would do what I did much more efficiently as I am terrible with PixelMath.

Anyway the sequence is  first make a black image the same size as the original. This is the base for your circle mask.
Next clone your black image.
Next create the white circle using the inellipse function in PixelMath. If you open the expressions editor and go to the inellipse function you will see the parameter format for this function.
Last apply the circle mask to the black image clone and with PixelMath use the original image and copy to the black image.


The TGVInpaint part requires modifying PI with a new, not released TGV module. You can find out more about it here. https://pixinsight.com/forum/index.php?topic=8970.0 (https://pixinsight.com/forum/index.php?topic=8970.0)
Here is a link showing one of it's uses. https://drive.google.com/file/d/0B13r3kYqTf8NcXQzVnJuVmlUeGs/view?usp=sharing (https://drive.google.com/file/d/0B13r3kYqTf8NcXQzVnJuVmlUeGs/view?usp=sharing)

Now the RangeSelect mask take some real work to get things just right for this image. This is not a plug in some numbers and your there. I actually use the RangeSelect tool on the masks created by the RangeSelect tool. I also had to use Curves on the masks created by the RangeSelect tool.
 
 

Mike
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 21 14:17:55
Thanks Mike, there's certainly a lot to learn here.  I'll start tinkering!
Title: Re: Trouble with 3d Doughnut
Post by: Lightpath on 2017 April 26 12:43:40
I'm actually quite proud of myself-  I managed to get the mask sorted out!  Now I'm trying to figure out TGVInpaint.  :)

Thanks again, I'm not sure I'll figure this all the way out, but I'm learning a lot about PI in the process.