Selective Rejection (Example)

[ngc1535]

Hi,

It is likely this has been shown before- but based on the response I received at AIC when I demonstrated this technique, it may be deserving of attention here (perhaps it will prove useful to someone). This demonstrates selectively rejecting problems in images that would otherwise not be rejected by statistical means. See below my pictures for a few comments. The basic steps are:

1. Identify problems in a set of images. The problem to be rejected should be a small subset of the total number of measurements/images made.
2. Create a black image of zeros (a paint bucket) and using the CloneStamp tool paint zeros on problems. Generally do this to image before registration if the problem is in many images, see below.
3. In the case shown below, the problem were dust donuts not flatted out properly. They exist in precisely the same spot for a given subset (night) of data...
4. If applying the painted area to many images, create a CloneStamp process and then apply to many images using an ImageContainer.
5. Register images in the normal way. Nearest Neighbor may give better results.
6. In ImageIntegration set up pixel rejection in the normal way. Make certain the Range Low is set to "0" and Reject Low Large Scale Structures is checked.

Comments: Obviously the S/N in rejected areas will be different- however the price may be small if the area isn't "important" or if the area was already a high signal (bright) region. I have not experimented with the effects of registration. I suspect all will work out well even with smoothing due to interpolation. However, performing this step on data after registering images will certainly work. With a large dither- the problem areas may dance around a bit.

In the last image, I show the difference of with and without the large scale rejection which takes care of the boundaries very well and makes selective rejection a reality.
Thanks.

-Adam Block

 

[sharkmelley]

Nice trick!  Thanks for sharing it.

Mark

 

[RickS]

Nice technique, Adam.

I do something similar for troublesome satellite trails using PixelMath, but with those it is better to write 1s instead of 0s as they reject out more cleanly.  Hopefully large scale rejection will make this unnecessary from now on 

Cheers,
Rick.

 

[ngc1535]

Hi Rick,

I am not certain why a clipping/threshold value of 0 or 1 would matter. The reason I choose the low end was to make certain that the number of high outliers are rejected strictly by statistical means so that "1's" are not included in count of rejections. Using the low end for rejection then seemed better. However, it could be it doesn't matter at all.

-adam

 

[RickS]

Hi Adam,

I had hoped to find time to run some tests but haven't managed it yet.  With clip low range enabled writing 0s should work just as well, so maybe I had it turned off (though it seems to be the default...)

Cheers,
Rick.

 

[astrovienna]

I've been trying to use this technique to salvage some data with a dust donut, but I'm not having any luck.  I can' t get ImInt  to reject the pixels that I've painted black.  The stacked image shows a very dark area where I painted out the dust donut in those frames, suggesting that the pixels from the blacked-out areas weren't rejected but simply included in the stack.  Below is a Dropbox file with twenty images of NGC 660, ten good images and ten where I've painted out the dust donut.  I've also included the ImInt icon with my settings, the resulting integration, and the rejection_low map from that integration.  The map suggests that Large Scale Pixel Rejection worked as expected, but Clip Low Range had no effect.

The technique seems pretty straightforward (brilliant, but straightforward!) and I watched Adam's explanation on the AstroImaging Channel, and I think I've got the settings right.  The data was collected on different nights, so the light pollution gradients are different, but I wouldn't think that would matter.  Any ideas?

https://www.dropbox.com/s/unkod6pwnwaoadq/NGC%20660%20Sample.zip?dl=0

Kevin

Edit:  NM, I figured it out. I thought I was using the hardest brush to paint out the pixels, but in fact I was using the softest brush. It's working now. I have a question, but I'll post that in the General forum.

 

[ngc1535]

Hi,

This is an update. Harmut Bornnemann has graciously updated his GAME script at my bequest. It is now possible to do this Selective Rejection in seconds. With GAME you can draw shapes on an image that represents the locations of artifacts and then output *pure* zeros (black pixels) to all images (writing to views or images on disk). Then you use low range rejection with "0" in order to reject as part of ImageIntegration. It is all explained in this new video. You can fast forward to minute 9:00 to see it in action without the explanation.
P.S. If you have a DRAMATIC example in a data set of something this technique would be good for...please let me know The example I chose here is just something I happened to be working on... not a super dramatic example:

Sincerely,
Adam

 

[txastro]

This update to the GAME script is very helpful. One thing I found is using this technique with subs having the default SubframeSelector SSWEIGHT FITS keyword will result in errors during ImageIntegration. Turns out the FITS keywords are not copied to the '_sr' files. I took care of that by using FITSHeader to copy the data from the corresponding images to the '_sr' images. Maybe it's better if I run SubframeSelect after the GAME script, but then the other variables would still not be there. Either way, this technique is wonderful.

Xavier

 

[ngc1535]

Xavier,

Harmut fixed this issue a while ago- all Keywords should be maintained.
Are you certain you have the latest version of his script?
I have 1.7.3.
I sent him an e-mail about this when it came up a few months ago.
That being said, I have not tested it recently.
-adam

 

[txastro]

Adam,

Turns out I had not checked updates for a while on my desktop. I get the notifications, but sometimes I don't apply them at the time. My laptop was up to date though. I ran the updates on the desktop and did a quick check. The keywords are being maintained with this version.

Thanks,
Xavier