Author Topic: PixInsight 1.6.1 - New RANSAC fitting rejection algorithm in ImageIntegration  (Read 27769 times)

Offline varmint

  • Newcomer
  • Posts: 35
    • Varmint's Astrophoto's
especially now that I can save individually selected ProcessIcons.

How do you do this?  Copy/paste the code in the History explorer?

Can you save Process Containers?

I tried looking for answers to this last night and couldn't...  I think I ran into a thread from a year ago that said this wasn't available, but I am not sure.

Thanks,

Jim
Clear Skies,

Jim
--"Do or do not.  There is no Try" --Jedi Master Yoda

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Hi Juan,

Enter linear fit clipping:

...
The remaining question is: is linear fit clipping the best rejection algorithm? I wouldn't say that. Linear fit rejection requires a relatively large set of images to work optimally. With less than 15 images or so its performance may be worse or similar to Winsorized sigma clipping. To reject pixels around a fitted line with low uncertainty, many points are required in order to optimize it in the least average deviation sense. The more images, the better linear fit is possible, and hence the new algorithm will also perform better. It's a matter of testing it and see if it can yield better results than the other rejection algorithms available, for each particular case.
...

I must admit that this method of identifying outliers and the "true value" of a pixel appeared extremely suspicious to me. After all, who says that a distribution without outliers would be well fitted by a linear function after sorting the samples? So I did an experiment with Excel, generating 50 samples with a poisson distribution (as a CCD would do), and adding some polluting signal (a sine) to this that was intended to simulate changing sky conditions such as haze. As can be seen in the attached screenshot or Excel 2007 file, your approach would come close the "true" value of 8000 in both the unpolluted and the polluted case. Quite a surprise for me.

Is there any literature that describes the method? What are known cases where this method produces bad estimates?

Kind regards,
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Hi,

I did some additional research.


In essence, this image is a Q-Q plot http://en.wikipedia.org/wiki/Q-Q_plot for a uniform distribution http://en.wikipedia.org/wiki/Uniform_distribution_%28continuous%29, i.e. when all values generated by the random process (here: CCD values) are equally likely within a certain range, the plot will be similar to a straight line. For distributions where values at the extremes become less likely (e.g. Poisson or normal distribution), the curve will always look more or less like an inverted S.

Basically, finding "outliers" always means that you have some idea how the data should be distributed, and filtering out those that do niot fit under this assumption. I think for CCD data, assuming a Poisson distribution http://en.wikipedia.org/wiki/Poisson_distribution is a valid approach. So to further refine this approach, it may be a good idea to use a Q-Q plot for a poisson distribution instead of one for a uniform distribution.

Just some ideas, not practically tested....

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Quote
Quote from: Niall Saunders on June 24, 2010, 15:08:21
especially now that I can save individually selected ProcessIcons.

How do you do this?  Copy/paste the code in the History explorer?

Can you save Process Containers?

I tried looking for answers to this last night and couldn't...  I think I ran into a thread from a year ago that said this wasn't available, but I am not sure.

Thanks,

Jim

Jim - I am moving the reply to your question into the <Tips and Tricks> section

Have a look at http://pixinsight.com/forum/index.php?topic=2025.0

Cheers,
« Last Edit: 2010 June 27 16:14:06 by Niall Saunders »
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline Emanuele

  • PixInsight Addict
  • ***
  • Posts: 270
How come I dont have RANSAC Fitting Rejection under ImageIntegration ?

I just have Linear Fit Clipping.....

E.

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Hi Emanuele,

I decided to drop RANSAC fitting in favor of the new linear fit clipping rejection algorithm. Sorry for the confusion I've generated with this post; I'll add a note to inform about this at the top of the thread.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Emanuele

  • PixInsight Addict
  • ***
  • Posts: 270
Oh, no worries juan, thank you fir clarifying that. :)