Author Topic: Error when DefectMap applied with Mean/Square and qurstion on ImageIntegration  (Read 4806 times)

Offline netwolf

  • Newcomer
  • Posts: 17
Hi All,

Some back ground first
Image captured with CCDSoft from Sbig8300mono camera. Conveted to 32bit fits in PI.
Lights: 6x120sec RGB each channel binned 2x2, 6x300sec L channel unbinned,
Dark Frames 6x120sec binned 2x2 and 10x300sec unbinned

Using the great videos by Harry and following some help I search and found from PI website and forums. I manged to Do all processing in PI.

Initally I did not have Dark frames so and my friends and I noticed we were all getting small one red one grean one blue pixel all over our image. We belived these to be hotpixels offset by Alignment. So we got the my friend whos equipment we used captured and sent me the dark frames. I made the masters in PI using the Tutorial i think by Vincent. I did the Calibration and at the end I found I Still was seein the Hot Pixels. After searching the fourms I realised that with PI ImageIntegration process the optimise function by design only elimated real noise. It seems to not remove Hot pixels. So based on advise I tried CosmeticCorrection tool this along with later use of ACDNR seem to remove the hot pixels. I have attached a small sample from one corner of my RGB combined image to show what I mean, this is just before ACDNR is applied, which seems to remove this.

But I wanted to go back and step through each segment of my processing to ensure I really understood the settings and what they were doing, apply some science to it. This is the really coold thing I like about PI, there are many measurements that can be used to evaluate quality rather than just judge by visual appeal. So it thought I will go back to my master darks and extract a Defect map, using the DefectMap extrack tool by NKV (wow some really helpfull scripts from this man). Using this I got the Bad Pixel map, now I am trying to apply to my orignal Light frames. I tired to apply to both the Calibrated (master dark removed) and non-calibrated Lights but the system returns this errro.

Channel #0
Fixing bad pixels, morphological median:   1%
*** PCL Win32 System Exception: At address 000007FEFD12A49D with exception code C0000005 :
Access violation: invalid memory read operation at address 000000001083954C

But if I chose Gausian/Square it seems to work ok. So for now I will do this.

Question why is this error occuring? Is it the update to 1.7?

Next question which is better, to apply Defectmap to non-calibrated Lights or to the Calibrated lights?

Also I am having some gradients in my Lights, which I am removing post Image Integration, now I have learned that I should probably do a DBE Map and apply to all before Image integration. One anomaly I noted was this.

Using Image Integration tool, with Windzorised Sigma Ciping i set the clips to 5 and 5. I noted from another hepfull thread here that the maximum SNR is sqrt(Number of Frames). Well if you use Scale+Zero Offset this is true, if I do 10/10 high/low respectively I get max SNR, such tat J SNR/Maths.sqrt(number of frames) returns .99%. But above 5 it seems to be above .99. But never reaches the theoretical limit of sqrt(number of frames). But I read that to help with Gradients one alternative method to individually doing a DBE, is to use Equalize Fluxes. But this method is returning SNR > sqrt(number of frames). How is this possible? Is it possible that the Gradient is causing some internal calculation error in Average SNR values?

Also is there somewhere where I can find a complete list of Outputs (in Process Console) for Each Process and Script that explains what the output means? Well at least maybe for the ones that affect SNR. Perhaps in future some graphical representation of this method could be provided.
Juan I know you get some criticisim on PI Interface being not to automated, but in my case I find this absolutely perfect. There are many softwares out there that take the Science out of it and you just push buttons to get a desired visual result. I like PI for the fact that it returns some control over each step to the User. I am all for automation, but I think Informed automation is best.

Regards
Fahim




 
« Last Edit: 2011 July 10 05:24:53 by netwolf »

Offline netwolf

  • Newcomer
  • Posts: 17
Well some more testing

Defectmap has to be aplied after calibration, else Calibration creates black pixels at the spots.

Post calibration I still did a CosmicCorrection. And then Staraligmnet on one frame each of RGB. Combined these and I am still seeing these funny pixels. See attached same corner as in prvious post. I am sure ACDNR will remove this but the question now is why are they occuring.

Its not the bad pixels, unlees the Defect extract from Master dark does not get all the bad pixels.

So what is causing these? Is cant be noise as it is to much of a coincedence that they appear to be in one place just offset by a one pixel. And that to L shapped. What am I missing?



Offline gvanhau

  • PixInsight Old Hand
  • ****
  • Posts: 345
Hello Fahim

I had some similar behavior, but last night, after some tweaking with the pixel rejection algorithm and parameters, it was completely removed.


Regards
Geert
Geert Vanhauwaert

Offline netwolf

  • Newcomer
  • Posts: 17
Geert,

can u please give some s details on how and when u apply this pixel rejection algorithim?

thanks
Fahim

Offline gvanhau

  • PixInsight Old Hand
  • ****
  • Posts: 345
Hello

I managed the solution during Image Integration and proceeded more or less like in this article:

http://jordigallego.fotografiaastronomica.com/articles/Image_integration_JGallego.ppt

I can give you my specific settigns after office, but the results are very dependent on the image data.

Regards
Geert

Geert Vanhauwaert

Offline netwolf

  • Newcomer
  • Posts: 17
Thanks Geert, I already found that one and went over it but it did not help. Monitoring previews and adjusting Sigma cliping values does not remove these hot pixels.

What I have is hot pixels that

1. Are not being removed by Dark frame subtraction because Optimise setting is used.
2. Are not being removed by Defect map subtraction (extracted from Master Dark using Defect map extract script from thsi forums). But I think i need to look at this again and maybe increase the threashold when generating the defect map this might help me remove the hot pixels better.
3. Wait for new releases of ImageCalibration where Optimise routine also removes hot pixels. Image calibration in all other programs I have tried, CCDsoft, CCDStack, Nebulosity removes the hot pixels. But IC in PI seems to have this problem. But reading the forums it seems PI belive this to be by design and neccesary. Refer to following links refrencing this.
http://pixinsight.com/forum/index.php?topic=2981.0
http://pixinsight.com/forum/index.php?topic=2471.0


I think in need to increas the threasholds for creating the Defect map and then apply them again. The only other way is to wait till ACDNR is applied post Histro Stretch, which seems to get rid of it. Problem with this is that ACDNR is aplied once the image is non-linear. But there are still some proces's i want to use while the image is linear that dont work so well with this hot pixels still there. Like Deconvolution. So I think i have to

1. Go back and redo Bad pixel map extract and increase threashold to get more of the hot pixels but this is "trial and error"
2. Do same but use Cosmetic correction to remove the Hot pixels. Same as above but does it in one step.
« Last Edit: 2011 July 14 12:49:35 by netwolf »

Offline gvanhau

  • PixInsight Old Hand
  • ****
  • Posts: 345
I'm still curious... I tought i had the same problem as you:

In debayered frames, before star aligment you can see several bright spots in diferent colors spread over the images, almos all in the same position.
After star aligment, those spots are still there, but at slight diferent positions because of drift or dithering.
After integrating the pictures you get spots in the final picture spread in small areas grouped by color.

What I did was beeing more restrictive in pixel rejection, and those spots disapeared.

Regards
Geert

Geert Vanhauwaert

Offline netwolf

  • Newcomer
  • Posts: 17
Geeret

This was taken with a monochrome with filters so no debayerign for these.

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
BTW, I would not recommend DefectMap for CFA images... for them use the ColorCorrection script, that handles these kind of images pretty well. The reason is that DefectMap looks for any valid pixel in the neighbourhood to perform the interpolation, but right now it can't discriminate between different kind of pixels (in the Bayer matrix). So, if you have notorious different channel weights, as in modified cameras, the result will still be a weird pixel.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com