New in PixInsight 1.8.5: new large-scale pixel rejection

vicent_peris

PixInsight Staff
Staff member
Hi,

People coming to my workshops already know that we were working on an important update to the ImageIntegration tool to allow a better rejection of meteors and other "track-like" outliers. Well, I'm happy to announce that we finally finished the development of this improvement to this critical tool. I post here some examples.

Below you can see a comparison between the regular integration and the integration with the new improved tool. It's a 15 frame integration using Linear Fit Clipping with the same settings in both cases. This data set has originally 75 frames, but I manually selected all the frames where there was a meteor:

M81-1.jpg


M81-2.jpg

Image provided by Frank Willburn.

This update is able to handle the rejection of big planes:

M51-4.jpg


And perfectly reject RBI artifacts:

M51-3.jpg


Below you can see the comparison between the current version of ImageIntegration and the new one coming in a few weeks:

M51-1.jpg


M51-2.jpg

Image provided by Frank Willburn.
This new feature also helps rejecting stars in a comet-aligned image set:

ISON-1.jpg

ISON-2.jpg


Of course, the rejection of the stars depend greatly on the speed of the comet and the optical system, but will be always better than in the current version of the tool.

One advantage of this new feature is that it is a completely automated solution, so it doesn't require any manual rejection of image bands. We hope you'll like this tool and the new ones that are coming in PixInsight 1.8.5. As always, our goal is to provide you a state-of-the-art solution for your imaging.


Best regards,
Vicent.
 

Juan Conejero

PixInsight Staff
Here is a side-by-side comparison to show the typical performance of the new large-scale pixel rejection algorithm:


On the left, the result of a normal integration of 10 calibrated, demosaiced and registered DSLR raw frames, using Winsorized sigma clipping rejection with clipping points set to 5 and 4 sigmas, respectively for low and high clipping. On the right, the same integration with the new large-scale pixel rejection algorithm enabled. The plane trail has been completely rejected. It can be rejected equally well even if the high clipping point is raised up to 5 sigmas in this example, which I have selected because I think it's quite representative of this kind of problems in typical data sets.

With large-scale pixel rejection, now we can overcome virtually all problems caused by large and bright spurious objects with high uncertainty borders, such as plane and satellite trails and flashes, meteors, stars on sky flat frames, RBI artifacts, etc. As Vicent has shown, integrated comet images can also benefit from a much better rejection of trailed stars. The algorithm works the same way for high and low outlier structures, so it can also reject large dark artifacts as well. Since large-scale rejection works in a completely automatic way (basically, you only have to enable it), getting the most out of your data sets can now be easier than ever.
 

idiaz

Well-known member
Great news, thanks!

Is there a cost to this implementation? Say, in terms of final general noise? I ask because of the optionality of enabling this feature.

Ignacio
 

vicent_peris

PixInsight Staff
Staff member
Hi Ignacio,

You should expect a signal to noise increase since you'll always need a more aggressive rejection to reject those outliers without large-scale rejection.

Best regards,
Vicent.
 

Juan Conejero

PixInsight Staff
No cost at all. Actually, it's just the opposite. Since large-scale pixel rejection is so efficient to reject large artifacts, one can use considerably higher clipping points, just as if the plane and satellite trails, meteors, etc. were not present. The end result is higher SNR, much better rejection of outliers, and easier use of the ImageIntegration tool.

I am considering the possibility to enable large-scale rejection by default, at least for bright structures.
 

mmirot

Well-known member
Do you think the RBI removal is as good as using the flash camera feature?

There is the obvious smear of bright pixels around stars etc. That seams to correct well.

What about the smearing other parts of the image,  not so bright structures?

You think it will recover some signal to noise in these regions? 



Max
 

vicent_peris

PixInsight Staff
Staff member
Hi!

I just wanted to show one experiment we did today with data by Rick Stevenson. We are really happy with the results, so I thought it's maybe worth to share it here. It's a very difficult example because there's literally a "highway" of satellites in the image. With the new large-scale pixel rejection we were able to completely reject all those trails with minimal noise increase.

Attached you'll find an animation without any rejection and with the large-scale pixel rejection.

Hope you like it!
Best regards,
Vicent.
 

Attachments

  • NGC2170_animation.gif
    NGC2170_animation.gif
    308.3 KB · Views: 437

ChoJin

Well-known member
that's really impressive. Please release 1.8.5 asap, I'm currently processing some data which would greatly benefit from this :)
 
Looking forward to this! One thing I noticed is that sometimes the clipping params were decent for normal ImageIntegration, but not when subsequently doing upsampling with DrizzleIntegration, will this new setting integrate with the Drizzle workflow?
 

KenM

Well-known member
Congrats on the new update!

Does this automatically work in the current BatchPreprocessing Script?

Thanks
 

dougggg

Member
I just downloaded 1.8.5. I'm trying to remove a satellite trail from an image.

Is there a quick start user guide for large scale rejection? I'm trying to use it in ImageIntegration but it doesn't seem to have any effect on an image stack containing 4 images with 1 image with a satellite trail. Layers/Growth are set to 3 and 4, reject high large-scale structures is clicked.

Doug
 

javajunkie2121

Active member
dougggg said:
I just downloaded 1.8.5. I'm trying to remove a satellite trail from an image.

Is there a quick start user guide for large scale rejection? I'm trying to use it in ImageIntegration but it doesn't seem to have any effect on an image stack containing 4 images with 1 image with a satellite trail. Layers/Growth are set to 3 and 4, reject high large-scale structures is clicked.

Doug

I second Doug's request..this is a VERY useful update and will be VERY popular..if someone would please give some guidance on settings that would be wonderful

I'm seeing a better identification of multiple trails in the high_rejection map and a better end integrated image using 12 images (2 with satellite trails) and  using the default 2/2 for layers growth.  Not sure if number of subs is reason it's working better.  I did not change the pixel rejection sig low (4) and high (3) default settings.  I tried it on a 30 sub integration with 4 known bad subs with trails..rejection map looks awesome..but end result looks same..maybe original algorithm was smart enough when there are 30 subs to eliminate trails.

I am unclear though on the role of the "reject low large-scale structures" setting when trying to eliminate bright satellite trails?  I've run the integration with and without it and I'm not detecting a difference?

jeff
 

chris.bailey

Well-known member
Jeff

I did a bit of playing with this yesterday. With reject large scale structures enabled you can crank up the sigma low and high (say 6 and 5) and still get the same level of rejection and get some SNR improvement as a result, in fact, with clean individual frames (Cosmetic Correction) you can dispense with the normal rejection altogether. As I tend to image the faint SNR and Sharpless objects this is a significant advantage.

Chris
 

javajunkie2121

Active member
Hi Chris:  I tried this with 12 blue subs of M51, 2 of which had star trails. I left the layers/growth values for high large scale structure pixel rejection at defaults of 2

..as soon as I start increasing the pixel rejection sigma low/high values from default 4/3(windsorized sigma clip), I get star trails returning?

also I tried adding "low large scale structure" rejection to high large scale structure rejection with sigma low/high back at defaults...compared to high structure rejection alone I think I'm seeing some loss of detail in the galaxy?  Is the low large scale structure intended for other targets?

jeff
 

ngc1535

PixInsight Ambassador
JavaJunkie....

Could this be a method of rejecting dust donuts (due to flat errors) just like what I was looking for?

-adam
 

pfile

PTeam Member
indeed it should adam, and i am remiss because i had trouble rejecting dust spots during the beta period and i flaked out on providing the sample data to juan. i can't tell if my trouble was due to misconfiguration of large scale rejection or not. i need to follow up with juan on that.

rob
 

chris.bailey

Well-known member
javajunkie2121 said:
..as soon as I start increasing the pixel rejection sigma low/high values from default 4/3(windsorized sigma clip), I get star trails returning?

Jeff

Is you data dithered in acquisition?

Chris
 
Top