Cosmetic correction - When is it used in the workflow?

[MikeOates]

Hi,

I have searched the forum, but can't find a definitive answer. When is Cosmetic correction used in the workflow?

Is it right after calibration of DSLR images but before debayering, or can it be used after debayer, or does it not matter? I presume it's best used before integration.

I know it can be used as part of the Preprocessing script, but I don't know at what point in that script it's applied. I am currently doing the calibration manually using Vicent Peris' tutorial http://www.pixinsight.com/tutorials/master-frames/en.html

Thanks,

Mike

[jerryyyyy]

I would love to have a better answer for this too.  I have been working on the Cat's Eye Nebula with 3600s images and it almost all my subs have satellite trails in them... looks like a tick-tack-toe game.  Either that or this nebula is surrounded by aliens.

I understand the variants of sigma clipping might work, but you need a lot of framts.  Never gotten this to work right it seems either in the preprocessing script or in CC itself run alone....

My alternative is Photoshop, which is a pain. 

[papaf]

The BatchPreprocessing Script does it as the very first thing. You can tell by looking at the script output.
If this is the correct way, I don't know, but I suppose it is.

[NKV]

After calibration, before debayering.

[jerryyyyy]

Do any of these techniques actually take out satellite trails?  I have heard some senior users pretty sceptical.  I have had no sucess, but maybe I need 50 subs. 

[Carlos Milovic]

Integration with a rejection algorithm takes out satellite trails very effectively.

[pfile]

After calibration, before debayering.

agree, does BPP really do it first? that does not actually make sense.

[papaf]

My reply was from memory. It's entirely possible it does it after calibration. Moreover, I don't use darks so the process just flies by, and I could well have missed the calibration phase entirely!

[georg.viehoever]

Cosmetic correction is only necessary if your images are not dithered (i.e. stars dont move between images). For me, this is only the case for widefields (<=85 mm). For standard telescope shots (750mm), my messy mount does the dithering "automatically". If dithering exists, pixel rejection does the trick of removing hot pixels, satellite trails etc. .
Georg

[jerryyyyy]

Cosmetic correction is only necessary if your images are not dithered (i.e. stars dont move between images). For me, this is only the case for widefields (<=85 mm). For standard telescope shots (750mm), my messy mount does the dithering "automatically". If dithering exists, pixel rejection does the trick of removing hot pixels, satellite trails etc. .
Georg

In my case, the images are dithered, but in a concrete example.  I have 6 3600s O-III images of the Cat's Eye nebula.  I am trying to collect >10.  I get 1/night.  2/6 currently have sattelite trails... seems the longer the sub, the more time over target, and the more likely to have a sattelite trail.  If I look at a typical trail in detail, it is as much as 5 pixels wide.  So if the rejection scheme looks around one of those pixels, it will see another pixel of similar intensity, so will not reject. 

I am using the BatchPrepocessing Script and using the "Average" Combination and

Winsorized Sigma Clipping with Low of 4.0 and High 1,96 settings. 

I am not sure what algorithm would work even with 20 images if the trails are 5 pixels wide. 

Does not seem CosmeticCorrection would add anything. 

What am I missing here? 

[pfile]

CC won't add anything. what can happen in the normal calibration flow is that there are still some hot/cold pixels left over after calibration. CC can clean those up.

CC can't do anything about airplanes or satellites, etc.

II's rejection functions don't look at any neighbor pixels (to my knowledge) but simply consider a stack of pixels at a time. the outlier values are thrown out, the rest are averaged together.

with small pixel stacks, discarding this bad data can be as bad as averaging it in. the SNR of the pixel stacks with fewer pixels is less than the surrounding pixels, and often this means you can see a "ghost" of the rejected satellite trail. just the fact that there are so many pixel stacks near one another with rejected pixels allows your eye to pick up on it; if the rejected pixels are distributed randomly around the image you won't notice the SNR hit.

the only cure for this is to have more subs in your stack. so you might have to do 20h+ (since you are using 1h subs) before you can "safely" do rejection on large structures like sat trails. that's another reason why people only expose to the sky or read noise limit - it's easier to get more subs with shorter exposures.

[jerryyyyy]

CC won't add anything. what can happen in the normal calibration flow is that there are still some hot/cold pixels left over after calibration. CC can clean those up.

CC can't do anything about airplanes or satellites, etc.

II's rejection functions don't look at any neighbor pixels (to my knowledge) but simply consider a stack of pixels at a time. the outlier values are thrown out, the rest are averaged together.

with small pixel stacks, discarding this bad data can be as bad as averaging it in. the SNR of the pixel stacks with fewer pixels is less than the surrounding pixels, and often this means you can see a "ghost" of the rejected satellite trail. just the fact that there are so many pixel stacks near one another with rejected pixels allows your eye to pick up on it; if the rejected pixels are distributed randomly around the image you won't notice the SNR hit.

the only cure for this is to have more subs in your stack. so you might have to do 20h+ (since you are using 1h subs) before you can "safely" do rejection on large structures like sat trails. that's another reason why people only expose to the sky or read noise limit - it's easier to get more subs with shorter exposures.

Thanks, I tried the 1800s images and the 3600s just have more H-alpha signal.  They are now blown out/saturated, so I think I am stuck with the 3600s to get maximal signal.

I did learn one thing along the way, in some old posts searching on "satellite" I found the suggestion of using "Median" rather than "Average" combination and the trails are substantially attenuated.  The overall stacked picture does not look much the worse for wear since I guess with a normal error distribution median and mean will overlap.  I seem some limitations to this so looks like overall bst to stick with "average".

This was a discussion of a sucessful effort with 14 subs:

http://pixinsight.com/forum/index.php?topic=2928.msg20080#msg20080

He used the Windorized clipping, but I do not seem to have enough images for that. 

Currently I am trying the Averge Sigma Clipping... which on reading is supposed to be most appropriate for 6-10 images.  I am trying different Sigma High values and lookinbg at the rejection maps. 

Guess I need more images.... hopefully can squeeze out 2/night. 

[MikeOates]

Jerry,

I don't know if you have seen this, but the following page may help:

Interpolation Algorithms in PixInsight:
http://pixinsight.com/doc/docs/InterpolationAlgorithms/InterpolationAlgorithms.html

Mike

[jerryyyyy]

Jerry,

I don't know if you have seen this, but the following page may help:

Interpolation Algorithms in PixInsight:
http://pixinsight.com/doc/docs/InterpolationAlgorithms/InterpolationAlgorithms.html

Mike

Thanks very much.  I'll look at it over the weekend.  What I have discovered (I think) is that you can systematically test the various pixel rejection options by stacking using the BatchPreprocessing Script, then run the Registered images through ImageIntegration.  I think your reference will help me understand the II output in terms of improvement of SNR. 

Last night I was able to capture two more 1h images of my main target (Cat's Eye) before the fog came in.  When I moved up from 6 to 8 images that really clobbered the satellites trails... what a difference a better estimate of central tendency can make....

Thanks again. 

[pfile]

im going to have to move my telescope down to your house. here it gets foggy around 6pm on most days. no imaging since 7/4.