New in PixInsight 1.8.5: new large-scale pixel rejection

[vicent_peris]

Hi,

People coming to my workshops already know that we were working on an important update to the ImageIntegration tool to allow a better rejection of meteors and other "track-like" outliers. Well, I'm happy to announce that we finally finished the development of this improvement to this critical tool. I post here some examples.

Below you can see a comparison between the regular integration and the integration with the new improved tool. It's a 15 frame integration using Linear Fit Clipping with the same settings in both cases. This data set has originally 75 frames, but I manually selected all the frames where there was a meteor:




Image provided by Frank Willburn.

This update is able to handle the rejection of big planes:



And perfectly reject RBI artifacts:



Below you can see the comparison between the current version of ImageIntegration and the new one coming in a few weeks:




Image provided by Frank Willburn.
This new feature also helps rejecting stars in a comet-aligned image set:




Of course, the rejection of the stars depend greatly on the speed of the comet and the optical system, but will be always better than in the current version of the tool.

One advantage of this new feature is that it is a completely automated solution, so it doesn't require any manual rejection of image bands. We hope you'll like this tool and the new ones that are coming in PixInsight 1.8.5. As always, our goal is to provide you a state-of-the-art solution for your imaging.


Best regards,
Vicent.

[Juan Conejero]

Here is a side-by-side comparison to show the typical performance of the new large-scale pixel rejection algorithm:

Click on the image for a full size version

On the left, the result of a normal integration of 10 calibrated, demosaiced and registered DSLR raw frames, using Winsorized sigma clipping rejection with clipping points set to 5 and 4 sigmas, respectively for low and high clipping. On the right, the same integration with the new large-scale pixel rejection algorithm enabled. The plane trail has been completely rejected. It can be rejected equally well even if the high clipping point is raised up to 5 sigmas in this example, which I have selected because I think it's quite representative of this kind of problems in typical data sets.

With large-scale pixel rejection, now we can overcome virtually all problems caused by large and bright spurious objects with high uncertainty borders, such as plane and satellite trails and flashes, meteors, stars on sky flat frames, RBI artifacts, etc. As Vicent has shown, integrated comet images can also benefit from a much better rejection of trailed stars. The algorithm works the same way for high and low outlier structures, so it can also reject large dark artifacts as well. Since large-scale rejection works in a completely automatic way (basically, you only have to enable it), getting the most out of your data sets can now be easier than ever.

[andreasmax]

very good news! cannot wait to check out this feature on my comet frames.

thx for this!

andreas

[Ignacio]

Great news, thanks!

Is there a cost to this implementation? Say, in terms of final general noise? I ask because of the optionality of enabling this feature.

Ignacio

[vicent_peris]

Hi Ignacio,

You should expect a signal to noise increase since you'll always need a more aggressive rejection to reject those outliers without large-scale rejection.

Best regards,
Vicent.

[Juan Conejero]

No cost at all. Actually, it's just the opposite. Since large-scale pixel rejection is so efficient to reject large artifacts, one can use considerably higher clipping points, just as if the plane and satellite trails, meteors, etc. were not present. The end result is higher SNR, much better rejection of outliers, and easier use of the ImageIntegration tool.

I am considering the possibility to enable large-scale rejection by default, at least for bright structures.

[Ignacio]

Understood, thxs!

[mmirot]

Do you think the RBI removal is as good as using the flash camera feature?

There is the obvious smear of bright pixels around stars etc. That seams to correct well.

What about the smearing other parts of the image,  not so bright structures?

You think it will recover some signal to noise in these regions? 



Max

[vicent_peris]

Hi!

I just wanted to show one experiment we did today with data by Rick Stevenson. We are really happy with the results, so I thought it's maybe worth to share it here. It's a very difficult example because there's literally a "highway" of satellites in the image. With the new large-scale pixel rejection we were able to completely reject all those trails with minimal noise increase.

Attached you'll find an animation without any rejection and with the large-scale pixel rejection.

Hope you like it!
Best regards,
Vicent.

[ChoJin]

that's really impressive. Please release 1.8.5 asap, I'm currently processing some data which would greatly benefit from this

[mmirot]

Yes Juan,

Get this one out. Too cool.

" Sound of fingers tapping on desk"

Max

[joelkuiper]

Looking forward to this! One thing I noticed is that sometimes the clipping params were decent for normal ImageIntegration, but not when subsequently doing upsampling with DrizzleIntegration, will this new setting integrate with the Drizzle workflow?

[OldSkyEyes]

Congrats on the new update!

Does this automatically work in the current BatchPreprocessing Script?

Thanks

[dougggg]

I just downloaded 1.8.5. I'm trying to remove a satellite trail from an image.

Is there a quick start user guide for large scale rejection? I'm trying to use it in ImageIntegration but it doesn't seem to have any effect on an image stack containing 4 images with 1 image with a satellite trail. Layers/Growth are set to 3 and 4, reject high large-scale structures is clicked.

Doug

[javajunkie2121]

I just downloaded 1.8.5. I'm trying to remove a satellite trail from an image.

Is there a quick start user guide for large scale rejection? I'm trying to use it in ImageIntegration but it doesn't seem to have any effect on an image stack containing 4 images with 1 image with a satellite trail. Layers/Growth are set to 3 and 4, reject high large-scale structures is clicked.

Doug

I second Doug's request..this is a VERY useful update and will be VERY popular..if someone would please give some guidance on settings that would be wonderful

I'm seeing a better identification of multiple trails in the high_rejection map and a better end integrated image using 12 images (2 with satellite trails) and  using the default 2/2 for layers growth.  Not sure if number of subs is reason it's working better.  I did not change the pixel rejection sig low (4) and high (3) default settings.  I tried it on a 30 sub integration with 4 known bad subs with trails..rejection map looks awesome..but end result looks same..maybe original algorithm was smart enough when there are 30 subs to eliminate trails.

I am unclear though on the role of the "reject low large-scale structures" setting when trying to eliminate bright satellite trails?  I've run the integration with and without it and I'm not detecting a difference?

jeff