Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - Salvatore Giambruno

Pages: [1] 2 3
1
Great presentation(s) Adam.  I've watched the first installment, going to watch the second one.  I'll mention it here (as I did on YoutTube) for the benefit of other PI users following your tutorial: I think in the first video you forgot to binarize the second star mask.  Not sure if that will make a difference in this process further down the line . . . we'll see :)

2
Mike,

That came out well.  It's all about getting the starmask and binarizing tweaked just right.  Works great on small to medium stars - it's always the large stars that are difficult to remove/replace.  Thanks for posting your effort.

Just for fun I took my 45mm shot of Cygnus and decided to see what the DefectMap could do. So first this is a simple image processing. ABE and then extract the red channel from the RGB. I use STF with HistogramTransformation for the stretch. Now the decided to try the ClassicStarMask script to generate the star mask. I then inverted the mask. Next I used Binarize with a very high setting of 0.97. This is the mask for the DefectMap to use.

I think it turned out very well for such a low resolution image. On the left is the full image and on the right is a 3:1 scaling of the section with Sadr.

Mike

3
No worries about hijacking, Adam - except for the explosive keyword itself, lol.

Looking forward to seeing the video.

-Sal


Hi Salvatore,

Thanks for writing. By the way, I felt a little bad about bringing up Fabian's example in your thread. I somewhat hijacked your post. Sorry about that.
Concerning the background, yes you are correct. The dark background was just my non-linear stretch that I actually used when I blended in the data. The background is unchanged in process of removing stars. I just didn't have the non-linear image of the stars removed with a lighter background that is all.

To everyone that might read this- I connected with Fabian...and I am working on making the video. However, he is in the process of reviewing my work (it is his implementation of this technique in PI after all)- and he is a busy guy! So it is still a work in progress... but very much in play.

-adam


Thanks for sharing, adam. 

Fabian's sounds like an interesting technique - I'll be sure to check it out as soon as I get a chance. 

I think you tried to explain why the sky background levels are different in the before/after images you posted as an example, but I'm not sure I understand the explanation.  Correct me if I am mistaken, but I believe you were trying to explain that the difference was not a shortcoming of Fabian's process, but that it turned out that way as a result of you rushing to prepare the example.

Hi Salvatore,

For the past few years Fabian Neyer has been demonstrating a technique of star removal that is based on practices that professionals use. He cites this paper of showing the intragalactic glow between members of the virgo cluster:

http://iopscience.iop.org/article/10.1088/0004-637X/720/1/569/pdf

The technique you demonstrate is something like a first order approximation of the idea. The full technique employs some other clever steps including using MLT to calculate the local sky value of stars at different scales. When you use the defect map, you are only using those pixels that are immediate neighbors to your regions. But varying the scale of MLT you can properly get better sky values (and therefore substitution) for the stars. In addition, and this is really clever- you can create a sky subtracted star images that you blur and then subtract from the real data in order to get rid of stellar halos and the wings of PSFs. This takes care of OldWexi's question.

(on a note of computing, Fabian's technique does not require the computational overhead of what the defect map does by looking at each neighbor pixel.)

I just completed an image that uses Fabian's technique. Below is a version of SH2-68 before and after star removal. (I did not see the black levels the same...just quickly grabbed). If there is interest, I can post my version of this tutorial (I will ask Fabian... but I think it isn't a problem). I added a few variations to Fabian's technique. It is a longer process, but the results are commensurate with the effort.


-adam

4
Bug Reports / Shared Image Reference
« on: 2017 November 23 16:41:39 »
This happened when I tried to close a separate image that I created as a CLONE OF A PREVIEW on a main image.  I use this technique to speed up previewing my MMT settings because using a preview that is still attached to the main image to preview MMT settings seems to want to process the entire main image (in cache) and seems slower than creating a separate preview image.  The problem with using this workaround now seems to be that Mask preview enabling/disabling (on the separate-image preview) is not working consistently when previewing MMT changes, and when I go to close the clone of the preview, I get this error message.

Restarting the application does not solve the problem.

5
Thanks for sharing, adam. 

Fabian's sounds like an interesting technique - I'll be sure to check it out as soon as I get a chance. 

I think you tried to explain why the sky background levels are different in the before/after images you posted as an example, but I'm not sure I understand the explanation.  Correct me if I am mistaken, but I believe you were trying to explain that the difference was not a shortcoming of Fabian's process, but that it turned out that way as a result of you rushing to prepare the example.

Hi Salvatore,

For the past few years Fabian Neyer has been demonstrating a technique of star removal that is based on practices that professionals use. He cites this paper of showing the intragalactic glow between members of the virgo cluster:

http://iopscience.iop.org/article/10.1088/0004-637X/720/1/569/pdf

The technique you demonstrate is something like a first order approximation of the idea. The full technique employs some other clever steps including using MLT to calculate the local sky value of stars at different scales. When you use the defect map, you are only using those pixels that are immediate neighbors to your regions. But varying the scale of MLT you can properly get better sky values (and therefore substitution) for the stars. In addition, and this is really clever- you can create a sky subtracted star images that you blur and then subtract from the real data in order to get rid of stellar halos and the wings of PSFs. This takes care of OldWexi's question.

(on a note of computing, Fabian's technique does not require the computational overhead of what the defect map does by looking at each neighbor pixel.)

I just completed an image that uses Fabian's technique. Below is a version of SH2-68 before and after star removal. (I did not see the black levels the same...just quickly grabbed). If there is interest, I can post my version of this tutorial (I will ask Fabian... but I think it isn't a problem). I added a few variations to Fabian's technique. It is a longer process, but the results are commensurate with the effort.


-adam

6
Rob,

If I recall correctly, I recently tried to apply the DefectMap to an image that had an active mask and got an error message.  I think the DefectMap process can only be applied to "maskless" images.

i wonder if it makes sense to apply the DefectMask process thru a mask (perhaps a different one with smoother star edges) since all DM can do is treat pixels as good or bad with no "nuance". this might mitigate some of the harsh edges around large stars. need to try.

rob

7
Gerhard,

It sounds like you forgot to INVERT the star mask.  The mask stars should be black, the background white.

Sal

Hi,

by the way, Defect Map. It takes hours to execute this tool, although I have an I7 PC. What´s my fault?

Regards   Gerhard

8
Hi Rob,

That's a great idea.  It's anyone's guess as to how it will work as far as filling in the large space for removed large stars, but I imagine that, at the very least, it will soften the edges of those removals . . .

9
I'm happy to see that this technique is proving useful to some.  I've never been so clever as this, and it feels great to have come up with something original that even seasoned pros find useful.  ;D

I did mention in my original post that the trick to getting this technique to work correctly is in getting the mask just right.  And that means getting the stars to be the right size, and BINARIZING your values as Marc has stated.  DefectMap only works with BINARIZED pixels.  A pixel is either BAD (black, zero) or it is is not bad (anything other than zero).

Star masks can be tweaked using whatever techniques you like.  I've used intensity transformations, convolution and morphology tricks to get my masks to work best.  But remember, whatever processes you use, the last process should be a Binarize process.

If the problem with star removal is one of scale, you can always generate multiple masks at different scales and apply the DefectMap several times (I've done this).

Unfortunately, big stars cover a wide area and therefore a DefectMap fix will not be perfect.  Therefore, I believe it is best to remove your stars in the linear state, when stars are as small as possible.

Last but not least, CloneStamp is always a last resort . . .

10
Hi Juan,

Sorry, but I have no recollection of the circumstances.  I can say however, that the PC I was using to process is a little light on RAM (12GB), so I suspect you are correct to suspect that a shortage of memory caused the error.  No big deal for now . . . I plan to upgrade my RAM.

Thanks for your help.

Sal

11
I'm not sure if this technique has been covered by anyone before or if I am a the first, but a brief search of this forum and Google turn up nothing as far as I can tell, so I thought I'd share it here in any case.

Briefly, you create a good star mask from your final image (or any of its derivations) using the StarMask process (or by whatever method you prefer), invert the resulting star mask (stars to black), then use that star mask as a map in the DefectMap process to and apply it to the image. 

This technique works VERY well and results in very little degradation to image detail.  The larger stars usually require additional touch-up because as a large "defect" they are replaced with a wide average of the background which is apparent even after the application of the star removal process.

The key to using this technique effectively is to create an accurate star mask to begin with.  If one mask doesn't remove all the stars, simply create another star mask at the scale required to get all the stars you need at a particular scale.






12
I was asked by the program to report this, so here it is:

Internal error: MainView::~MainView(): Unexpected shared image reference:
Image112_clone
This denotes corruption of internal data structures. To stay on the safe side, save your data, exit the application, and start again. Please report this problem to our development team.

13
No Problem, Juan. 

I'm not sure what image you are referring to.  I can't do a screenshot because the crashed PI session has long-since been terminated.

At the risk of exposing my ignorance on the use of Dynamic PSF, I was gathering PSF samples from a heavily-processed, non-linear subframe, NOT A SUBFRAME.   I'm not sure if this is relevant or if it explains why the process crashed.

I've since been using Dynamic PSF and am not experiencing a recurrence of the problem - so I'm happy for now.  However, let me know if there is anything else I can do to assist you if you wish to troubleshoot this problem further.

Thanks for your help.

14
Bug Reports / Dynamic PSF Window Causing Errors and Disappearing
« on: 2017 October 16 23:24:35 »
Dynamic PSF caused an error to appear (I think access violation) when I tried to close the interface.  I just ignored it.  When I tried to open another dynamic interface, I got an error saying itcouldn't be opened becuase PSF was already open but the window was nowhere to be found.  I tried closing the project (after saving), but that process merely aborted.  I closed all the image windows hoping it would disconnect the ghost dynamic PSF interface but then I got this pop-up:

"Internal error: MainView::~MainView(): Unexpected shared image reference:
Image821_clone
This denotes corruption of internal data structures. To stay on the safe side, save your data, exit the application, and start again. Please report this problem to our development team."


After this, tried to exit PI and got this pop-up:

"At address 00007FF6C4CAC8DB with exception code C0000005 :
Access violation: invalid memory read operation at address 0000000000000008"


Tried opening another project and got this in the console:

Loading project:
F:/Astronomy/PI/PROJECTS/SH_2_162 Bubble Nebula.xosm
* Project integrity verified, SHA1 checksum = 25b84683ba5a5e0d7fda8504d3fc185de832dfaa
*** PCL Win32 System Exception: At address 00007FF6C4CAC8D7 with exception code C0000005 :
Access violation: invalid memory read operation at address 0000000000000028

<* abort *>



Sal

15
Bug Reports / Re: Missing PI Icon
« on: 2017 October 01 15:31:50 »
Got it.  Thanks!

Pages: [1] 2 3