Author Topic: Trying to kill a few hot pixels  (Read 2664 times)

Offline mikey2000

  • Newcomer
  • Posts: 19
Trying to kill a few hot pixels
« on: 2018 January 19 06:59:33 »
Hi again!  More newbie questions!

My stack is finished and I'm going well with the stretch.   I find just a handful of extra-bright pixels left over - they aren't 100% hot but they still stand out.

I'm having trouble finding a way to remove them.

I've tried Cosmetic Correction.  The Auto Detect seems to 'see' them.  In the real time preview, the 'Show Map" option gives a black view with little spots exactly where my problem pixels.  However, when I run the process on that one file, the output file still has the defects.  Also, if I turn off 'Show Map', the real time preview doesn't show anything being removed.  I have no mask set.

I'm sure I'm doing something simple and wrong.  Any pointers?

Here are two screenshots



Offline mikey2000

  • Newcomer
  • Posts: 19
Re: Trying to kill a few hot pixels
« Reply #1 on: 2018 January 19 07:00:40 »

Offline bulrichl

  • PixInsight Guru
  • ****
  • Posts: 524
Re: Trying to kill a few hot pixels
« Reply #2 on: 2018 January 19 09:13:56 »
CosmeticCorrection is intended to be used before ImageRegistration on monochrome images (in the case of a DSLR camera or One Shot Color [OSC] camera: on monochrome Color Filter Array [CFA] images), i. e. in the workflow just after ImageCalibration. When you align and integrate, hot pixels are multiplied - this is not what you want.

Bernd

Offline mikey2000

  • Newcomer
  • Posts: 19
Re: Trying to kill a few hot pixels
« Reply #3 on: 2018 January 19 09:31:24 »
Yep, I used CosmeticCorrrection as part of the pre-processing before debayering.  However, when I get near to the finished file (after stretch etc), I still have some red or even some tiny blue blobs (just a few pixels here and there, maybe a few thousand out of a 24mpx image).

I know how to get rid of these in photoshop (with a dust/speckle + masks sort of thing) but I assumed there would be a way in PI.

Surely there must be...

Offline bulrichl

  • PixInsight Guru
  • ****
  • Posts: 524
Re: Trying to kill a few hot pixels
« Reply #4 on: 2018 January 19 10:12:11 »
OK, then go back and modify the CosmeticCorrection settings. You can use a combination of 'Use Master Dark', 'Use Auto detect' and 'Use Defect List'. However, for the last option it is necessary to build and save the Defect list previously.

Bernd

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: Trying to kill a few hot pixels
« Reply #5 on: 2018 January 19 11:28:59 »
if you dithered during capture then you should be able to reject the hot pixels during integration... did you use any pixel rejection mode and did you check the high rejection map to see if there is anything in there?

rob

Offline mikey2000

  • Newcomer
  • Posts: 19
Re: Trying to kill a few hot pixels
« Reply #6 on: 2018 January 19 11:48:51 »
Rob,  I'm still learning but I think I even understand your question.. (Two days ago, you would have got a 'blank look' in reply!)

I haven't look at high rejection maps yet.  I'll run the integration again and look more closely.   As it is, I don't dither (my camera isn't automated enough to get the dither timings correct)   I noticed in DSS that my images always have a little dx and dy, maybe even a fraction of a degree of rotation.  I hope that's enough to count as dithering.  I don't see where PI displays this sort of dx/dy misalignment information.

It's curious though.  When I open a raw, I see the hot pixels. If I hover over them inspection tool, I expected to see 1.000 in of the colours (ie, hot/stuck pixel) but the values don't go that high. It seems I have 'warm pixels' :-)

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Re: Trying to kill a few hot pixels
« Reply #7 on: 2018 January 19 12:16:03 »
if you see 0.25 then they are hot pixels.

the reason being that most DSLRs have 14-bit a/d converters. when PI opens one of these files, it puts the 14-bit data into the lower 14 bits of 16 bit integers. therefore the max of 11 1111 1111 1111 is represented as 0011 1111 1111 1111, which is 16383. 16383/65536 is about 0.25.

as StarAlignment is running, it prints a transformation matrix for each sub exposure. after that matrix it also shows the "human readable" X and Y offsets. this stuff all flies by in the console so you'd have to scroll back to find it.

if rejection was not turned on, then there will not be a high rejection map. if you have enough subs (~15+) the best option is to use Windsorized Sigma clipping. i usually end up with both low and high sigma sliders in the vicinity of 3, but you need to inspect the rejection maps to see if there is any structure of your object in them... if so, the rejection slider(s) are set too low.

usually that kind of steady march in X/Y in your subs is the result of differential flexure between the guider and imager. if unguided, it can be the result of inexact polar alignment. in either case, sometimes nearby hot pixels start to overlap due to that diagonal motion and then they become hard to reject. this is the source of the "rain noise" that's often seen in DSLR images that were undithered.

if you end up there, then cosmetic correction is the way out. note also that sometimes due to dark scaling during calibration of your images, that hot pixels are only partially corrected, and end up *becoming* warm pixels... which due to the fact that they are not strong outliers anymore, can also be hard to reject.

rob

Offline ngc1535

  • PixInsight Old Hand
  • ****
  • Posts: 326
Re: Trying to kill a few hot pixels
« Reply #8 on: 2018 January 19 23:16:28 »
Mikey,

If you have reached a point in processing where you have red/green/blue blobs and they were not taken care of earlier through standard acquisition and processing (dithering, calibration, cosmetic correction...etc). Usually these things are completely independent in color space. Hot pixel rejection relies on a criterion of brightness... but you could instead use color to identify and then "kill" the offenders since it is likely the color is primary (this appears to be the case in the images you show). Just something to keep in mind in the future as another solution for similar problems. (I suspect a flurry of pixel math expressions could follow for example.)

-adam

Offline mikey2000

  • Newcomer
  • Posts: 19
Re: Trying to kill a few hot pixels
« Reply #9 on: 2018 January 26 03:19:44 »
Adam (ngc1535)  thanks for your message with the video:

https://drive.google.com/open?id=1KPWBDrVMByq0xb8PnZc-JOSrxDWIdiz2

I tried your technique and it worked very well.    Now, I'm a newb with PI but I still managed to follow the steps and even learned a few new things along the way.

The blobs in your video are very similar to the blobs I experienced.  I'd recommend anyone to take a look.

In summary, Adams technique is as follows. 
Quick PixelMaths to create a mask from pixels that are way more red than they are green/blue
Dilate the mask a little
MultiscaleMedianTransform on the R layer to patch over the blobs
(optional) Noise Generation on the patches to match the background.

The process worked on my blue blobs as well with only minor adjustment to the pixelmaths formula.

I suppose the ultimate goal is to create a stack without any blobs in the first place but until I master that, I will use this technique quite often, I suspect.

With thanks to Adam