Star Shaping

[Simon Hicks]

Hi Jack,
              Yes, perfect guiding, no wind, no Murphy.....a man can dream

I have had another go with another image that had stars slightly extended towards the 2 O'clock and 8 O'clock directions. Again I tried drawing a structure that mimicked this, and hey presto it worked quite well. All the other directions just made things worse.

So I am starting to convince myself that this is how you can use the element shape facility in the morphological feature. But I await confirmation from the PI guys that this fits with the theory...or whether Murphy is just doing a double bluff.

I'll put together a few examples of odd shaped stars tomorrow.

Cheers
             Simon

[Carlos Milovic]

Hi Simon

1)
Indeed, morphological filter is a way to go to make the stars rounder, if the deformation is not too big. I mean, it will work very well for slight tracking errors, and some kinds of coma, but it is not perfect. The way to go is simply to draw a shape that resembles the star, or the deformation, just like you should do with a deconvolution.

For example, if you have those rugby balls (vertically), you may try a structuring element like this one:

B W B
B W B
B W B

B = black, W = white

(or 0 and 1...)

One quick way to understand how the morphological filters work, is just making the question: Which is the "%" pixel defined by this neighbourhood? So, the answer, is that the algorithm looks around every pixel, only for those defined as white in the structuring element, and writes the result in a new (temporal) image. By "%", I'm talking about the minimum, maximum, median or any other "statistical" operator (and combinations of them, like closing, aperture, and the not yet implemented skeleton).
The morphological filter has other meanings when working with binary images (pure black and white), from where comes the name "structuring element" for the neighbourhood definition.


So, more than just simply copying the shape of the star, we must know exactly what we want to do, and how to build the structuring element.
When we use multiple ways (more than one structuring element) the partial results are averaged, and the final result is copied to that temporal image. This kind of procedure works well with small structuring elements, and where the effect we want to emulate cannot be builded with the rigid definition performed by the binary structuring element. For example, for the star shaping routine, I'm using these days one way with a square 3x3 kernel (another word for the structuring element) and a 3x3 cross one.


Having said of all these, there is another kind of morphological filters, which I plan to implement this year, that makes uses of grayscale structuring elements. Here the interpretation is not as straight as looking at a neighbourhood, and the filter has some drawbacks (like the overall "bias" is changed"), but may prove to be a good alternative for this procedure.


Of course, you may always try a deconvolution too. The best way to apply it is to the raw data.
If you are not that "orthodox", you may try mixing mild amounts of the morphological filter and a deconvolution. I've used several iterations of a minimum filter and the Wienner deconvolution to make slight out of focus stars brighter in the center, and smaller in size.

Of course, make use of masks to avoid artifacts in other zones of the image. Another tip, using the minimum filter is not always a good idea... you may end with "black holes" in your image. It is better to use a selection, something in the 0.1 to 0.3 range.



2)
This sounds like a "brute force" solution
I'll think on something that dramatic to fix star shapes, but it will be a challenge

In most cases, the secret is just to build a good star mask, either with the StarMask process, or manually from the luminance. Since in my workflow I almost always perform a separation between large scale and small scale features, modifying the last is a quick way to build the mask. I just have to raise the black point (with histograms) to leave only stars in the image, and then I blur it a little bit with wavelets. If needed, a closing morphological operator (minimum + maximum) may be used as a intermediate step at the end, to select a minimum size of the stars to be modified. Since I apply a star shaping routine (like the one described by Matt Bendaniels in his web, using several iterations of the minimum filter, at 0.1 amount, and increasing the saturation with curves) just to the bigger stars, a 7x7 or bigger, circular, structuring element is my choise.


Hope this helps... if not, please show me the way to go

[Simon Hicks]

Hi Carlos,
                  Many thanks for the thorough response. A few points....I agree, everything depends on getting the masks right. I think I've read a number of times in the tutorials and other places in the forum the importance of being able to mask at every stage, and this is something I am now doing with great effect. That's really one of the things that makes PixInsight so powerful in my view.

Regarding your explaination of how the Morphological filter actually works...many thanks for this. This really confirms what I had suspected, but its really nice to have it spelt out by someone who really knows.

Would it make sense to be able to somehow put a square around a single (typical) star and make that the kernel for the filter?  And the same would apply to the deconvolution process. Being able to just sample a star from the the actual image might be a very handy addition.

I've done a lot of experimenting with the deconvolution method....and I like the "elegance" of this approach. The problems I seem to have are;

1. The centres become very bright very quickly and can lead to millions of equally blinding pinpoint stars very quickly. And dynamic range extension seems to leave dark rings around the base. I'm using a starmask, so dynamic range extension might be great for the centre of the star, but what is it doing to the background?....is this being decreased during the dynamic range extension process? If so then this would lead to dark rings around the stars. Is there a way of avoiding this, i.e. of getting the background within the starmask to remain the same after deconvolution and dynamic range extension as the background it started with?

2. Again, with an odd shaped star (maybe some guiding error, or maybe a slight flaring) the deconvolution can actually enhance the oddness of shape by packing more intensity into the odd bits rather than evolving towards a nice circle. I can increase the standard deviation and change a few other parameters to give a nice round shape, but these make the stars bigger rather than tighter.  Making stars bigger is not a problem, I can do that easily!  :lol:

I've tried combinations of morphological filters and deconvolutions, but again without too much sucess. If you start with a nice circular star that is slightly out of focus then I am sure that these will work very well. However, when you start with a slightly odd shaped star then the deconvolution only exaggerates the oddness.

Regarding the "brute force" solution.....I agree wholeheartedly that it is as subtle as a flying brick and about as elegant as a pig in high heels. But it really does seem to do a good job. Here's an example I threw together.....

So here's some crappy stars...the sort I would never be guilty of creating, but other users might  :lol: I've really used an agressive colour saturation for clarity....



Here's really tight starmask



created using the following settings



Note the small smoothness, the zero growth and zero comp values....I want smaller stars, not bigger.

I then ustilise the colour from the original stars simply by using PixelMath and multiplying the badstars image by the starmask image to give



I then want to replace the bad stars in the first image with these stars. So I create another starmask with more typical default settings (i.e. larger, softer openings) and then apply many iterations (maybe 10!) of an erosion mask to try and obliterate the stars out of the image. There are probably more elgant ways of doing this with wavelets, but this will illustrate the point. This leaves the following image...



Note that this could be done much better with wavelets and would provide some much needed background smoothing as well....but carrying on.....the next step is to insert the nice fabricated stars onto this image.

I did this with PixelMath and used a simple iff statement of the form;

iff(nicestars>badstars,nicestars,badstars)

and this gave the following image



Now most of the brighter stars have been replaced with nice small, round, perfectly coloured stars and they have roughly the correct intensity relative to each other....and the background has not been effected in anyway.....except maybe during the erosion process to obliterate the badstars....but this could be done more elegantly. And these stars should be ideal for subsequent stretching or other manipulation.

I might try this subtle brick approach in a bigger way when the clouds finally lift from the Glasgow area and I can get some actual images to play with.

But I just wonder if there is a way of using something from all of this in a more elegant way?

Just a thought.

Cheers
              Simon

[Simon Hicks]

Hi Jack and Carlos,
                                    I have been playing with the star masking module and pushing it in extreme ways for the above purpose...and realised what is probably obvious to you guys...that this module is relying heavily on wavelets. The PI software really uses these in powerful ways.

So I was wondering if the star shaping could be done better without the starmask module...but just using the underlying wavelets processing. You might still use the starmask module to section off the following processing from the other regions, but not use it to directly create false stars...if you see what I mean.

I first found some bad shaped stars....again nothing to do with me  :wink:



I then applied some ATrousWaveletTransform. I used a 5x5 guassian, put an X by layers 1 to 4 and put layer 5 to +1 with Deringing to maximum. This gave....



So the above made the bad star shapes into nice big round stars. I then applied 2 iterations of the morphological erosion with a 9x9 circular kernel to give....



This results in nice small circular (ish) stars. But these are a bit dim, so I then applied 20 iterations of a regularised Richardson-Lucy deconvolution to give.....



So the final image has massaged the bad star shapes into much nicer shapes. I'm sure that with a bit more effort they could be tightened further, but this illustrates the point.

I don't actually claim to have invented this technique as I am pretty sure I have seen reference to it somewhere in the forum, and you guys have probably already been doing this for years....I've just discovered it for myself.

Its certainly much quicker and simpler than the previous technique I described, and its also much less 'brute force' like. I think the stars have lost some of their colour though.

Cheers
             Simon

[Carlos Milovic]

Hi Simon

Sorry for the delay. I've been quite busy these days, so I had not the time to sit down and write a complete answer. Please have a little more patience.

[Simon Hicks]

Sorry Carlos....I can be known to race off at a tangent sometimes. Its exciting learning all this new stuff and I'm keen to try out new processes.

I shall now shut up, back away from the keyboard and be patient.  8)

[Carlos Milovic]

Hi Simon :)

I think I've read a number of times in the tutorials and other places in the forum the importance of being able to mask at every stage, and this is something I am now doing with great effect.

Well, I'm not of the opinion that every stage should be masked... but indeed if properly used it will show a significant improvement. The key point here, as always, is to know exactly what you are doing, and what do you want as a result. Image processing is not like cooking a cake, following instructions.  It is more likely to the work of a sculptor. You have to uncover the David from the stone :);  follow the path the image shows.

Would it make sense to be able to somehow put a square around a single (typical) star and make that the kernel for the filter? And the same would apply to the deconvolution process. Being able to just sample a star from the the actual image might be a very handy addition.

You may try something like this... make a preview over a star, create a new image with it, and use the Binarize process to make it black and white (convert it to grayscale before it). Now manually copy the shape into the MorphologicalFilter's Structuring Element.
As I said earlier, a more refined way to do this, with a "grayscale" structuring element is not possible with the current algorithm. We may find a way to solve this... or just implement the "standard" grayscale morphological transforms I described to you in the previous post. To put it in simple words, one adds or substracts the structuring element to the image, and then looks for the statistical value that is desired. The problem, as is quickly shown, this leads to a change in the overall brightness, it is not just a change in the size of the structures. I have some ideas in mind to partially solve this... but since this process is not a priority right now, it must wait :)

Now, about deconvolution. What you said has been implemented with the first release of both processes: Deconvolution and WienerDeconvolution.
Just look, under the PSF section, for the External PSF tab. There you may include a view as a PSF. The way this works is slighty different for both processes. Deconvolution needs square images, grayscale, while Wiener works with any kind of images.
A word of caution: using a star as the PSF theoreticaly is the best option, but in the practice the results may not be good. Experiment modifying it with morphological filters, curves and the histogram. Also, it is important that the star has been very well centered on the image to be used as PSF, or the deconvolved image will be shifted.

1. The centres become very bright very quickly and can lead to millions of equally blinding pinpoint stars very quickly. And dynamic range extension seems to leave dark rings around the base. I'm using a starmask, so dynamic range extension might be great for the centre of the star, but what is it doing to the background?....is this being decreased during the dynamic range extension process? If so then this would lead to dark rings around the stars. Is there a way of avoiding this, i.e. of getting the background within the starmask to remain the same after deconvolution and dynamic range extension as the background it started with?

To protect star centers use a inverted luminance mask (and the star mask as the deringing support).

Here is a quick explanation on how the dynamic range extension works. Internally, all pixels are handled as floating point numbers, or in other words, as real numbers. It doesn't matter if they go out of range, because they are just numbers, and not interpreted as pixel values. If you set it the lower boundary to "S", and the upper to "H", that "real valued" image will be rescaled so S will be zero, and H 1 (mathematically, [img-S]/[H-S]). Then, every pixel out of the new normalized range values will be clipped (will become 0 or 1). So, if you just increase the extension for the highlights, the image as a whole will be darkened. Imagine that we set it in a way that 2.0 will be the point for white values... when rescaled, the image will be divided by 2, so the brightness will be the half for every pixel. If you are using a star mask, and their sizes are larger than the star itself, you'll see that "ringing" or donut effect. This is just a problem with the mask (or it's combination with the extension). You may try to increase the extension for the shadows, to compensate the effect... but this will leave artifacts if the stars are surrounded with nebulosities or other features that vary the "background" brightness.
A best solution is to fine tune the mask... or don't use it at all (I'm talking about a Star Mask... a mask based on the luminance may prove to be much better in this case). As I said above, use StarMask to build deringing supports for the Deconvolution process, mainly, but I would not trust in it too much as a suitable mask (if you are not capable of fine tune it... I'm not :D, Juan is the expert with that tool).

2. Again, with an odd shaped star (maybe some guiding error, or maybe a slight flaring) the deconvolution can actually enhance the oddness of shape by packing more intensity into the odd bits rather than evolving towards a nice circle. I can increase the standard deviation and change a few other parameters to give a nice round shape, but these make the stars bigger rather than tighter. Making stars bigger is not a problem, I can do that easily!

The PSF is not the ideal image you want to generate. It is the model of the deformation the image underwent to reach the current state. So, this is why it is said that the stars are the "perfect" choise to build PSFs... they are supposed to be (ideally) point sources, so their shape is exactly how the image have been degraded. In the real world, matching exactly the stars with the PSF does not give the optimal results, but indeed is a very good starting point. In your case, instead of using a star as external psf, try to reproduce that shape with the gaussian options (change the ratio and angle, and keep the standard deviation low. Change the function order if needed). Another option, that works well if you have traking errors parallel to the x or y axis of the chip, is to use a motion blur psf. Because of pixel discretization, this doesn't work very well for other angles.





Now about your method. Seems to work very well, but there are lots of faint stars that have not been reshaped. What I'd to is to use morphological filters (with and without masks) to erase all stars, and even other small scale features (also, blur it a little disabling some wavelet layers). Do it to a duplicate of the image. Now, using PixelMath, apply: "img-deletedstars", without rescaling, and create the result as a new image. Close the "blured" image.

That new image will contain all the bright, high frecuency features. Now, again with PixelMath, apply "img-stars", againg without rescalling. The result will be all the large scale features, and the dark high frecuency (small scale) ones.

Process the small scale image to get rounder stars, and then add the result to the large scale image, rescaling the result. This should work ;)



About your second approach, a note: you may create star masks (with the exact current shape) with wavelets disabling the residual layer, and setting all other deringing parameters to the maximum. You'll have to use a trial/error process to find how many layers to discard (or, in other words, where the residual layer is), and which is the best scaling function.

Now, your method works well, but has some side effects... I would try to blur the star just a little bit, and make use of morphological filters to make them rounder before deconvolution. Also, you may try several iterations of this procedure, instead of a crude single processing. BTW, this is exactly the idea behind noise reduction. It works better by slowly approaching the solution, than getting it straight with a strong blur. The same happens with iterative deconvolution algorithms, like Richardson-Lucy. ;)


Thank you very much for your motivation to find another ways to perform these tasks. This is exactly what motivated us from our very beggining: beeing a "source" for innovations and imagination. Please keep those ideas coming. Who knows? maybe we'll end with a new process automating all of this, thanks to your ideas :D


PS: I'll work a with your sample images, and see what happens.

[Simon Hicks]

Hi Carlos,
                  Wow, you have really explained a lot there and clarified a lot of different aspects for me. I really appreciate the time and effort you have gone to to explain this. I will read your comments very carefully a number of times and do some experimenting with your suggestions. I'm excited about what the next image will look like!

Many thanks
                       Simon