New in PixInsight 1.8.5: new large-scale pixel rejection

vicent_peris

PTeam Member
Sep 3, 2004
995
12
Valencia (Spain)
www.astrofoto.es
Hi,

People coming to my workshops already know that we were working on an important update to the ImageIntegration tool to allow a better rejection of meteors and other "track-like" outliers. Well, I'm happy to announce that we finally finished the development of this improvement to this critical tool. I post here some examples.

Below you can see a comparison between the regular integration and the integration with the new improved tool. It's a 15 frame integration using Linear Fit Clipping with the same settings in both cases. This data set has originally 75 frames, but I manually selected all the frames where there was a meteor:




Image provided by Frank Willburn.

This update is able to handle the rejection of big planes:



And perfectly reject RBI artifacts:



Below you can see the comparison between the current version of ImageIntegration and the new one coming in a few weeks:




Image provided by Frank Willburn.
This new feature also helps rejecting stars in a comet-aligned image set:




Of course, the rejection of the stars depend greatly on the speed of the comet and the optical system, but will be always better than in the current version of the tool.

One advantage of this new feature is that it is a completely automated solution, so it doesn't require any manual rejection of image bands. We hope you'll like this tool and the new ones that are coming in PixInsight 1.8.5. As always, our goal is to provide you a state-of-the-art solution for your imaging.


Best regards,
Vicent.
 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,129
334
57
Valencia, Spain
pixinsight.com
Here is a side-by-side comparison to show the typical performance of the new large-scale pixel rejection algorithm:


On the left, the result of a normal integration of 10 calibrated, demosaiced and registered DSLR raw frames, using Winsorized sigma clipping rejection with clipping points set to 5 and 4 sigmas, respectively for low and high clipping. On the right, the same integration with the new large-scale pixel rejection algorithm enabled. The plane trail has been completely rejected. It can be rejected equally well even if the high clipping point is raised up to 5 sigmas in this example, which I have selected because I think it's quite representative of this kind of problems in typical data sets.

With large-scale pixel rejection, now we can overcome virtually all problems caused by large and bright spurious objects with high uncertainty borders, such as plane and satellite trails and flashes, meteors, stars on sky flat frames, RBI artifacts, etc. As Vicent has shown, integrated comet images can also benefit from a much better rejection of trailed stars. The algorithm works the same way for high and low outlier structures, so it can also reject large dark artifacts as well. Since large-scale rejection works in a completely automatic way (basically, you only have to enable it), getting the most out of your data sets can now be easier than ever.
 
  • Like
Reactions: polare70

andreasmax

Active member
Jan 26, 2014
31
1
very good news! cannot wait to check out this feature on my comet frames.

thx for this!

andreas
 

Juan Conejero

PixInsight Staff
Sep 2, 2004
8,129
334
57
Valencia, Spain
pixinsight.com
No cost at all. Actually, it's just the opposite. Since large-scale pixel rejection is so efficient to reject large artifacts, one can use considerably higher clipping points, just as if the plane and satellite trails, meteors, etc. were not present. The end result is higher SNR, much better rejection of outliers, and easier use of the ImageIntegration tool.

I am considering the possibility to enable large-scale rejection by default, at least for bright structures.
 

mmirot

Well-known member
Mar 27, 2005
886
0
Do you think the RBI removal is as good as using the flash camera feature?

There is the obvious smear of bright pixels around stars etc. That seams to correct well.

What about the smearing other parts of the image,  not so bright structures?

You think it will recover some signal to noise in these regions? 



Max
 

vicent_peris

PTeam Member
Sep 3, 2004
995
12
Valencia (Spain)
www.astrofoto.es
Hi!

I just wanted to show one experiment we did today with data by Rick Stevenson. We are really happy with the results, so I thought it's maybe worth to share it here. It's a very difficult example because there's literally a "highway" of satellites in the image. With the new large-scale pixel rejection we were able to completely reject all those trails with minimal noise increase.

Attached you'll find an animation without any rejection and with the large-scale pixel rejection.

Hope you like it!
Best regards,
Vicent.
 

Attachments

ChoJin

Well-known member
Jul 17, 2016
106
1
that's really impressive. Please release 1.8.5 asap, I'm currently processing some data which would greatly benefit from this :)
 

mmirot

Well-known member
Mar 27, 2005
886
0
Yes Juan,

Get this one out. Too cool.

" Sound of fingers tapping on desk"

Max
 

joelkuiper

Member
Mar 14, 2016
12
0
Looking forward to this! One thing I noticed is that sometimes the clipping params were decent for normal ImageIntegration, but not when subsequently doing upsampling with DrizzleIntegration, will this new setting integrate with the Drizzle workflow?
 

KenM

Well-known member
Dec 25, 2016
59
1
48
Belgium
Congrats on the new update!

Does this automatically work in the current BatchPreprocessing Script?

Thanks
 

dougggg

Member
Sep 3, 2016
20
0
I just downloaded 1.8.5. I'm trying to remove a satellite trail from an image.

Is there a quick start user guide for large scale rejection? I'm trying to use it in ImageIntegration but it doesn't seem to have any effect on an image stack containing 4 images with 1 image with a satellite trail. Layers/Growth are set to 3 and 4, reject high large-scale structures is clicked.

Doug
 

javajunkie2121

Active member
May 25, 2014
43
0
dougggg said:
I just downloaded 1.8.5. I'm trying to remove a satellite trail from an image.

Is there a quick start user guide for large scale rejection? I'm trying to use it in ImageIntegration but it doesn't seem to have any effect on an image stack containing 4 images with 1 image with a satellite trail. Layers/Growth are set to 3 and 4, reject high large-scale structures is clicked.

Doug
I second Doug's request..this is a VERY useful update and will be VERY popular..if someone would please give some guidance on settings that would be wonderful

I'm seeing a better identification of multiple trails in the high_rejection map and a better end integrated image using 12 images (2 with satellite trails) and  using the default 2/2 for layers growth.  Not sure if number of subs is reason it's working better.  I did not change the pixel rejection sig low (4) and high (3) default settings.  I tried it on a 30 sub integration with 4 known bad subs with trails..rejection map looks awesome..but end result looks same..maybe original algorithm was smart enough when there are 30 subs to eliminate trails.

I am unclear though on the role of the "reject low large-scale structures" setting when trying to eliminate bright satellite trails?  I've run the integration with and without it and I'm not detecting a difference?

jeff
 

chris.bailey

Well-known member
Jan 6, 2010
235
0
Jeff

I did a bit of playing with this yesterday. With reject large scale structures enabled you can crank up the sigma low and high (say 6 and 5) and still get the same level of rejection and get some SNR improvement as a result, in fact, with clean individual frames (Cosmetic Correction) you can dispense with the normal rejection altogether. As I tend to image the faint SNR and Sharpless objects this is a significant advantage.

Chris
 

javajunkie2121

Active member
May 25, 2014
43
0
Hi Chris:  I tried this with 12 blue subs of M51, 2 of which had star trails. I left the layers/growth values for high large scale structure pixel rejection at defaults of 2

..as soon as I start increasing the pixel rejection sigma low/high values from default 4/3(windsorized sigma clip), I get star trails returning?

also I tried adding "low large scale structure" rejection to high large scale structure rejection with sigma low/high back at defaults...compared to high structure rejection alone I think I'm seeing some loss of detail in the galaxy?  Is the low large scale structure intended for other targets?

jeff
 

pfile

PTeam Member
Nov 23, 2009
6,726
263
indeed it should adam, and i am remiss because i had trouble rejecting dust spots during the beta period and i flaked out on providing the sample data to juan. i can't tell if my trouble was due to misconfiguration of large scale rejection or not. i need to follow up with juan on that.

rob
 

chris.bailey

Well-known member
Jan 6, 2010
235
0
javajunkie2121 said:
..as soon as I start increasing the pixel rejection sigma low/high values from default 4/3(windsorized sigma clip), I get star trails returning?
Jeff

Is you data dithered in acquisition?

Chris