Author Topic: Confused about Purpose of the "Cosmetic Correction" Script  (Read 9178 times)

Offline Terry Danks

  • PixInsight Addict
  • ***
  • Posts: 137
I've watched Harry's videos on the Batch Preprocess script several times. (Thanks, Harry!)

But I'm confused at why the Cosmetic Correction script, demonstrated in the same video, exists. It seems to "turn off" hot pixels but isn't that a function of dark frames? So, if the hot (and cold) pixels are removed during the dark subtraction process, what is the point of Cosmetic Correction?

Terry

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #1 on: 2012 July 04 09:20:02 »
Hi Terry

Dark frames are taken to eliminate the "dark current", which is generated by thermal excitation of the semiconductor that is the ccd or cmos chip. Most of the pixels behave in a very similar way, generating a current that is linearly dependant of the exposure time. But, there are a few pixels, that do not follow that linearity, and generate either way more electrons, or too few. These are the so called hot and cold pixels.
The problem with them, is that due the nonlinearity of their behaviour, dark frames cannot properly correct those pixels. So, they must be fixed in other ways.
Dark current is corrected through a subtraction, and information of the target is "preserved". To correct hot or cold pixels (also bad columns or pixels) we replace their contents with information from neighbour pixels. So, in those cases, data is lost in the pixel, and we try to get a good estimation of which it may be. That is another reason why the dithering technique is recommended, so you move hot and cold pixels , and you may recover the data from the whole object.

In PixInsight you have two processes that performs this task. CosmeticCorrection, and DefectMap. The first one, uses a dark frame as input to generate a map of cold and hot pixels, which are replaced. In the second, you supply the map of pixels that should be replaced.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Terry Danks

  • PixInsight Addict
  • ***
  • Posts: 137
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #2 on: 2012 July 05 08:58:43 »
Thank you, Carlos, for the explanation. My understanding of the dark frame function was incomplete and naive.

Offline kerrywaz1

  • Member
  • *
  • Posts: 68
    • Spirit Bear Observatory
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #3 on: 2012 July 05 09:07:44 »
Good explanation.

I too have watched Harry's video tutorial about the Batch Processing script and I have a question about the Defect Map generation: how is this defect map generated? Is there a tutorial yet for this defect map generation?

Thanks,
Kerry
Deep Sky Instruments RC14C
FLI PL16803 w/MOAG + StarlightXpress Ultrastar
Pyxis 3"
Astro Physics AP1200
Tak FSQ-106EDXII
Apogee Alta F16M
Hap Griffin modified Canon EOS 5D Mark II
PixInsight, MaxIm DL, PhotoShop CS5

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #4 on: 2012 July 05 09:34:41 »
Hi Kerry
Dark frames are a good starting point to generate a defect map (as the CosmeticCorrection process does). For example, if you want to isolate the hot pixels, just use the Binarize process to create a black or white image that contains all the hot pixels. You have to carefully set the threshold value, using previews (not the real time one). Then, you have to invert the binary image, so "bad" pixels are black and "good" pixels are white.
The same is true for cold pixels. This time, most of the image will become white, leaving the cold pixels black, as you set the threshold value for the Binarize process.

You may apply the same strategy to deal with bad columns (it is easier to use light frames in this case). Also, you may use MorphologicalFilter to remove isolated pixels (with the opening or closing operations), or even the CloneStamp.

If you want to merge several maps into one, just use PixelMath, multiplying the maps, or taking the minimum value (since they are binary images, it is the same).
Just remember, bad pixels should be black (zero value). Any other value is considered as a good pixel.


Also, the DefectMap process allows you to use different operators to replace the data marked as bad. You may perform a convolution, where the pixel weights are the same as the pixel values in your defect map (and hence, this is the reason why bad pixels should have a zero value). Or, use morphological/statistical methods, that do not consider weights.

I already sent to Juan the documentation for this process, and a also a few changes to the code fixing bugs and adding the possibility to deal with bayered data. I hope he finds time to compile them and redistribute through the update system.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline kerrywaz1

  • Member
  • *
  • Posts: 68
    • Spirit Bear Observatory
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #5 on: 2012 July 05 10:43:58 »
Thanks, Carlos.

One of Harry's excellent tutorials would really help with this Defect Map generation process for us PixInsight lightweights. Are you listening, Harry?  ;D
Deep Sky Instruments RC14C
FLI PL16803 w/MOAG + StarlightXpress Ultrastar
Pyxis 3"
Astro Physics AP1200
Tak FSQ-106EDXII
Apogee Alta F16M
Hap Griffin modified Canon EOS 5D Mark II
PixInsight, MaxIm DL, PhotoShop CS5

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1458
    • http://www.harrysastroshed.com
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #6 on: 2012 July 05 12:15:39 »
Hi

No sorry gone a bit deaf  :o

Harry

Oh go on then will do  :laugh:
Harry Page

Offline gboyle

  • Newcomer
  • Posts: 9
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #7 on: 2012 October 19 00:49:25 »
I suspect I have some similar questions to others...
First off, when I have defined my cosmetic correction process, should it be applied to my light frames before or after dark subtraction?
Secondly, should I be applying it to my darks in any way - either to raw darks before making my master darks, or to the master dark?
(I wonder this because if I take the hot pixels from an uncalibrated light frame and then subtract a dark which still has the hot pixels in it I may have negated the impact I wanted..?)
I guess the batch preprocessing script might have this all figured out but I may be doing the separate stages myself for a while.
Thanks

Offline chris.bailey

  • PixInsight Addict
  • ***
  • Posts: 235
Re: Confused about Purpose of the "Cosmetic Correction" Script
« Reply #8 on: 2012 October 19 04:01:55 »
I appply CC to my light frames post calibration using the master dark AND autodetect with the sigma for both set to about 3. Either on its own still leaves a few warm pixels behind but together they seem to kill 99.999% of them very nicely.