PixInsight 1.6.1 - New RANSAC fitting rejection algorithm in ImageIntegration

[varmint]

especially now that I can save individually selected ProcessIcons.

How do you do this?  Copy/paste the code in the History explorer?

Can you save Process Containers?

I tried looking for answers to this last night and couldn't...  I think I ran into a thread from a year ago that said this wasn't available, but I am not sure.

Thanks,

Jim

[georg.viehoever]

Hi Juan,

Enter linear fit clipping:

...
The remaining question is: is linear fit clipping the best rejection algorithm? I wouldn't say that. Linear fit rejection requires a relatively large set of images to work optimally. With less than 15 images or so its performance may be worse or similar to Winsorized sigma clipping. To reject pixels around a fitted line with low uncertainty, many points are required in order to optimize it in the least average deviation sense. The more images, the better linear fit is possible, and hence the new algorithm will also perform better. It's a matter of testing it and see if it can yield better results than the other rejection algorithms available, for each particular case.
...

I must admit that this method of identifying outliers and the "true value" of a pixel appeared extremely suspicious to me. After all, who says that a distribution without outliers would be well fitted by a linear function after sorting the samples? So I did an experiment with Excel, generating 50 samples with a poisson distribution (as a CCD would do), and adding some polluting signal (a sine) to this that was intended to simulate changing sky conditions such as haze. As can be seen in the attached screenshot or Excel 2007 file, your approach would come close the "true" value of 8000 in both the unpolluted and the polluted case. Quite a surprise for me.

Is there any literature that describes the method? What are known cases where this method produces bad estimates?

Kind regards,
Georg

[georg.viehoever]

Hi,

I did some additional research.

In essence, this image is a Q-Q plot http://en.wikipedia.org/wiki/Q-Q_plot for a uniform distribution http://en.wikipedia.org/wiki/Uniform_distribution_%28continuous%29, i.e. when all values generated by the random process (here: CCD values) are equally likely within a certain range, the plot will be similar to a straight line. For distributions where values at the extremes become less likely (e.g. Poisson or normal distribution), the curve will always look more or less like an inverted S.

Basically, finding "outliers" always means that you have some idea how the data should be distributed, and filtering out those that do niot fit under this assumption. I think for CCD data, assuming a Poisson distribution http://en.wikipedia.org/wiki/Poisson_distribution is a valid approach. So to further refine this approach, it may be a good idea to use a Q-Q plot for a poisson distribution instead of one for a uniform distribution.

Just some ideas, not practically tested....

Georg

[Niall Saunders]

Quote from: Niall Saunders on June 24, 2010, 15:08:21
especially now that I can save individually selected ProcessIcons.

How do you do this?  Copy/paste the code in the History explorer?

Can you save Process Containers?

I tried looking for answers to this last night and couldn't...  I think I ran into a thread from a year ago that said this wasn't available, but I am not sure.

Thanks,

Jim

Jim - I am moving the reply to your question into the <Tips and Tricks> section

Have a look at http://pixinsight.com/forum/index.php?topic=2025.0

Cheers,

[Emanuele]

How come I dont have RANSAC Fitting Rejection under ImageIntegration ?

I just have Linear Fit Clipping.....

E.

[Juan Conejero]

Hi Emanuele,

I decided to drop RANSAC fitting in favor of the new linear fit clipping rejection algorithm. Sorry for the confusion I've generated with this post; I'll add a note to inform about this at the top of the thread.

[Emanuele]

Oh, no worries juan, thank you fir clarifying that.