Author Topic: Rejecting data...which algorithm?  (Read 2195 times)

Offline Emanuele

  • PixInsight Addict
  • ***
  • Posts: 270
    • View Profile
Rejecting data...which algorithm?
« on: 2010 November 06 15:10:13 »
Pictures this: you have been imaging with 20 min subs and imaging time is precious.
3-4 of those 20 min subs are are bad: wind causing elongation of stars, a bump in the mount etc...

Is there an algorithm in PI that rejects the data outside of 2std deviation from a fixed value, when combining?
I asked this following a thread I started on Cloudy Nights, where someone said to use CCDStack because it can reject the trails on the stars caused by wind bumps, etc etc. and thus allowing to use even bad frames for the integration.



Online Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 6838
    • View Profile
Re: Rejecting data...which algorithm?
« Reply #1 on: 2010 November 07 00:17:16 »
Hi Emanuele,

There are two different problems here:

- Bad data are bad data. They won't get better by rejecting portions of elongated stars due to tracking defects or accidents. A sporadic bump can probably be rejected well during image integration, but in general, images with wind bumps will be of inferior quality and you should discard them because they will degrade your final result. Bad frames always degrade the result to some extent; they will improve the SNR apparently on smooth background regions for example, but will always hurt the PSF.

- Any pixel rejection algorithm can be used. The algorithm to use depends on the number of images. I will copy this answer that I wrote in another thread a couple months ago:

For very small data sets, say from 3 to 5 images, use percentile clipping or averaged (Poisson based) sigma clipping rejection. For intermediate sets (from 5 to 10 images) sigma clipping and averaged sigma clipping are normally the best options. For large sets (from 10 to 20 images) use Winsorized sigma clipping or linear fit clipping. For very large sets (more than 15-20 images) linear fit clipping should perform better than Winsorized s.c. This is a sort of 'rule of thumb'; don't take it as written on stone and carry out your own tests with your data.

Other pixel rejection algorithms have been implemented for completion/research purposes and for special applications. An example is min/max clipping, which can be useful in special cases but should be avoided for normal applications, as it performs an unconditional rejection of a fixed number of pixels without any statistical basis.

where someone said to use CCDStack because it can reject the trails on the stars caused by wind bumps, etc etc.

Everyone recommends what they know or simply what they have and/or have tested (good marketing also plays a role here). Any application with a reasonable implementation of image integration with pixel rejection can carry out that simple task very easily.
Juan Conejero
PixInsight Development Team