Applying Defect Map issues

[varmint]

I'm trying out the new Defect Map process incorporated as a part of v1.6.1 but could use a little help.

I'm trying to use a Master Dark file, which has a bunch of hot pixels in it, as the Defect Map and trying to apply it to an image in the hopes that it would replace the hot pixels with the median value of the surrounding pixels in the target image.  Unfortunately nothing seems to be happening.

So either I am misunderstanding how the Defect Map tool works, or I have something setup incorrectly (as I highly doubt this is a bug).  I have loaded both frames into the workspace, in the settings dialog for the Defect Map I'm using my Dark File as the Defect Map.  I'm choosing Median and Square as the parameters.  I then drag the blue triangle from the Process Icon onto my target light frame and the Processing Console indicates a calculation has occurred, but I see no difference in my target image.

Any help would be appreciated, maybe this is the wrong tool for the job?

[Nocturnal]

As I understand it (based on forum posts and tooltips) the defect map is not supposed to be a dark frame. It's supposed to be an image where every pixel with a value > 0 will be considered a defect and will thus be fixed in the image the process is applies to. So this means you first need to construct your defect map.

I wrote a defect finder/fixer program a while back that I need to port to PCL. It'll behave the way you expect it to.

[mmirot]

Kind of hard to use with out way to generate a map.

Max

[Nocturnal]

Someone will say to use pixelmath

[Andres.Pozo]

It's supposed to be an image where every pixel with a value > 0 will be considered a defect and will thus be fixed in the image the process is applies to.

I think it is the opposite. The description of the process in the Process Explorer says:

DefectMap uses a defect map image to find defective pixels, then it replaces all defective pixels with data computed from neighbor pixels.
In a defect map image, defective pixels are signaled by zero pixel values.

[Nocturnal]

Doh! That makes little sense, frankly. I would want the map to indicate where the fixes should be applied. You do that with non-zero values in an otherwise dark image. Oh well

[mmirot]

I need a few screen shoots on how to set this up. 

Max 

[varmint]

It's supposed to be an image where every pixel with a value > 0 will be considered a defect and will thus be fixed in the image the process is applies to.

I think it is the opposite. The description of the process in the Process Explorer says:

DefectMap uses a defect map image to find defective pixels, then it replaces all defective pixels with data computed from neighbor pixels.
In a defect map image, defective pixels are signaled by zero pixel values.

So if I read this right, you apply the Defect Map Process to your actual Image, and if the Image has pixels with 0 value then it'll replace them... I'll give this a try tonight.

As an aside, I should read the header on the processes more often... 

[varmint]

I did a quick test tonight and had what I'd call a partial success.  I think I can improve things with a better map.

Here's what ended up working for me:
"Created" a map by using Pixel math to invert my master dark frame (formula was " Abs (dark_frame - 1)).  Since many of my "hot" pixels had a value of 1 in my dark frame, inverting converted them to 0's.

Then I used the Defect Map with this new map and applied it to one of my sub-frames.  For those pixels that were really 0 in the map it did fix the sub-frame pixel.  My issue is that the map really does have to have all values =0 for the pixels to be fixed in the image.  Many of my hot pixels in the dark frame were "almost 1" but not quite 1 so when I inverted they were "almost 0" but no quite 0, so the pixels were not touched.

I'll try to put together a couple of screen shots for you Max.

[Carlos Milovic]

Hi guys

Since this is my frankestein, I'll try to aswer you questions

Doh! That makes little sense, frankly. I would want the map to indicate where the fixes should be applied. You do that with non-zero values in an otherwise dark image. Oh well smile

It is just a matter of how you interpret the data. You may just as well think as a defect map the other way around. 1 is good data, 0 is bad data Also, there is a little practical reason behind this. To calculate the kernel for the convolution or morphological filter (for each pixel a new kernel) it is better to interpret the defect map locally as this kernel (for every neighborhood). So, 0 pixels values means that they are not considered in the operation. In fact, this point of view enables to use gray pixels partially, as weighting factors in the kernel calculation. So, if you force the users to give a image "ready for inner use", there is a gain in processing speed. You don't have to invert the image every time you try the process.

Now, the problem is how to prepare the defect map.
The most direct approach is with a pixel inversion (Invert process), and then either use HistogramTransform or Binarice processes. The later just sets a threshold value, where the result is a black or image image depending at which side of the threshold is the original value. Almost the same as setting the low and high clip in HT in the same point. If you don't want to use different weights (only for convolutions), just use the binarization.

And that's all... a bit odd, maybe, but works really fine (in my tests, at least) A script may automatize the defect map generation from a dark frame, and then use this process, if you want to perform batch processing, or just leave it to do all the calibration stuff without much interaction. As some of you know, I'm more a "Divide & Conquer" guy than "Do it All at Once"...

[mmirot]

Do you need to prepare 2, one for hot and one for cold?

Max

[Carlos Milovic]

No need. As long as you follow the 0-bad, 1-good strategy, they could be anything. Since they are considered as "bad" pixels (or columns), they are discarded completely and the neighborhood is inspected/operated to find the new pixel value. No interpolation, or other previous information feed is used. Just convolutions (bluring filters) or morphological transforms.

Having said that, if you start from the dark frame, it would be easier to use two defect maps, one for the hotter pixels, and one for the coolest. You may merge both maps with PixelMath, but I think that this would be computationally the same. Also you may use curves instead of a histogram transform (using a inverted U shape).

[varmint]

Here are two screenshots of my test go tonight.  Using my inverted dark master applied to a clone of a single subframe.  I had some setting wrong during image capture (so I've deduced) that had an auto-dark subtracted from my light frames, leaving holes.

The "fixed" frame is on the left, the un-altered source frame is on the right, the calculated map is in the middle, the dialog box for the Defect Map is lower right (dragged the instance blue triangle onto the "try1" image).  In the second image (annotated and attached), Pixel A is the one that is truly 0, Pixel B is the one that is "close to 0".  The corresponding map (from a Dark frame) shows the same Pixels.

Screenshot 1
Screenshot 2, annotated

Question for Carlos, I tried using a clone of my light frame but got a memory allocation error that failed the process, it shouldn't matter what file you use for the map correct? (As long as you're not trying to use a map of the same image your trying to adjust of course).

If you use the Histogram Transformation tool , you're essentially clipping some of the dark pixels to generate the map?

Thanks in advance

[Carlos Milovic]

Yes. Please note that a median filter is a child of a Morphological filter. So, in this case, pure black (0.000) values are considered bad pixels, and everything else is a good one; no weighting. You should use HT to ensure that they are pure black, or even better, the Binarize process (with the RTP is really easy to use it).

[mmirot]

perhaps you could illustrate with a few screen shots Carlos.

Max