Author Topic: Hot pixel removal from DSLR subs  (Read 2936 times)

Offline philherbert

  • Newcomer
  • Posts: 4
Hot pixel removal from DSLR subs
« on: 2016 January 17 12:49:29 »
Hi,

I'm new to PixInsight (still in my trial membership), and although I can see the potential, I'm having a real problem with hot pixels. My setup is using an unmodified Canon 350D, with a new guiding setup taking 10 minute subs. I know there's a lot of noise on such an old camera, so I've take a series of 10 minute darks, as well as bias and flat frames. I didn't use dithering for the subs (next chance with clear skies, if that ever happens). However no matter how I combine them either using different BatchPreProcessing setttings, or a manual workflow from Light Vortex Astronomy, I still end up with very noticeable hot pixels.

I've tried CosmeticCorrection but I can't find any settings which make much difference there either. I'm open to suggestions! I'm hoping that I'm missing something obvious, but can't spot it. The hot pixels are in the dark subs, but don't seem to be having much of an effect. The end result is as shown (cropped, 1:1 after a histogram stretch).

I'm starting to wonder whether 10 minute subs for lights and darks is asking too much of an old 350D?

Phil


Offline biosif

  • Member
  • *
  • Posts: 52
    • http://www.astrobin.com/users/iosifbodnariu/
Re: Hot pixel removal from DSLR subs
« Reply #1 on: 2016 January 17 13:13:43 »
Tak FSQ85, FLI ML 8300, CGEM.

Offline philherbert

  • Newcomer
  • Posts: 4
Re: Hot pixel removal from DSLR subs
« Reply #2 on: 2016 January 17 14:59:49 »
Thanks Iosif.

I had seen that but nothing appeared to work. I can't get any level of cosmetic corrections to take out the hot pixels. And when I tried the pixel maths I got an error in the formula used. I might have another try to fix it but 30 mins of trying didn't help yesterday!

Any other ideas?

Offline philherbert

  • Newcomer
  • Posts: 4
Re: Hot pixel removal from DSLR subs
« Reply #3 on: 2016 January 18 08:26:33 »
OK, I had another try at the PixelMath approach, and at least managed to get it to run (it seemed to be a problem with a carriage return in the copied text, or something similar). It didn't take out all of the hot pixels, but quite a noticeable amount.

Interestingly there are quite a few hot pixels which occupy more than one actual pixel, and it is these that it fails to remove, although they do improve somewhat. I wonder if this is related to using an old camera with so much noise that the hot pixels spread out?

OK strike that. Repeated applications of the same formula with PixelMath takes out the majority of hot pixels! It also appears to work with no noticeable affect on image quality, even after 7 applications! Wow.

Offline dnault42

  • PixInsight Addict
  • ***
  • Posts: 125
Re: Hot pixel removal from DSLR subs
« Reply #4 on: 2016 January 18 11:59:26 »
Phil commented on my website with this question and I responded there, but I thought it might be useful to post that same response here.

Here were Phil's questions:
1. Would you recommend this only for use on debayered subs?
2. Is there a way to tune the hot pixel removal from the PixelMath expression? The “f=9.0” looks like the place I could try perhaps?
3. Is there a way of running this multiple times automatically do you think?


And here is my answer:
1) Yes, the subs would need to be debayered first. This is because, while Bayered, the intensity at each adjacent pixel may be very different due to the filter transmission and brightness of the object in each portion of the spectrum that the filter allows through (this is why with Bayered monochrome data you may see some pixels as much darker than others when zoomed in). The expression could be modified to work on a Bayered image but it would be difficult and easier to do with a script or process.
2) Yep, the f=9.0 symbol is essential a sigma multiplication factor. It isn’t exactly a sigma value because I’m using a biweight midvariance function instead of standard deviation, but it works effectively the same way. Smaller values will mark more pixels as hot or cold and replace them with surrounding data, so they are more aggressive but they also have a higher chance or removing real data, like star peaks.
3) If you bring up the ProcessContainer you can drag the ‘new instance’ icon from the PixelMath window into it 5 or more times. When that ProcessContainer is applied to your image it will run that PixelMath expression however many times you added it in. You can save that ProcessContainer off, just like the PixelMath process, so it can be used whenever you want.

I will say that when I created some of these PixelMath expressions, tools like CosmeticCorrection did not exist. The ‘Use Auto detect’ option does essentially this same thing. It does use standard deviation rather than bwmv which is not as robust but still works well. It also only works in a global capacity on files, so you can’t run it on a single image or use the ProcessContainer method I described above to run it in multiple passes. It does however handle Bayered data (the CFA option – which stands for color filter array).


The reason why your hot pixels cover more than one pixel is that debayering uses an interpolation process (when using bilinear or VNG) to determine what the color is at each pixel location.  Since a hot pixel's 'signal' is typically very strong relative to the rest of the image the color associated with that hot pixel is spread across the surrounding pixels.

Tools like CosmeticCorrection and DefectMap, which are CFA aware, can operate on the the Bayered data knowing that the filters are arranged in a 4x4 pixel pattern to correctly determine what the sigma is and what the replacement value should be.

Regards,
David

Offline philherbert

  • Newcomer
  • Posts: 4
Re: Hot pixel removal from DSLR subs
« Reply #5 on: 2016 January 19 12:43:39 »
Thanks very much for the help and advice David! I've not had a chance to try out your suggestions yet.

Phil