Author Topic: Carlos' development modules collection  (Read 32422 times)

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2169
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Carlos' development modules collection
« on: 2010 September 04 22:57:20 »
This is the new repository for the modules that I'm working on, published here as beta products:
http://www.astrophoto.cl/Research.html
Please keep in mind that all of these are development processes, not final products. Bug reporting, or enhancing suggestions are highly welcome.

Now compiled for PCL 02.01.00.0749 (PI Core 1.8.4.1171 and above). Only 64bit releases will be available. Windows, and Linux only, for the moment.


Compilations for PCL 1.x are considered obsolete. Latest versions can be found here:
http://pteam.pixinsight.com/pcldev/carlos/PCL1
-----------------------------------------------------------------------------------------------------------------
For a description of the modules, enter the first link above.
« Last Edit: 2015 August 25 16:45:57 by Carlos Milovic »
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PTeam Member
  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • View Profile
    • http://www.carpephoton.com
Re: Carlos' development modules collection
« Reply #1 on: 2010 September 05 04:29:39 »
Good grief!

Excellent work Carlos! I'll be taking a look at all of them but I'm particularly interested in your Hough Transform as I looked at that some time ago.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1438
    • View Profile
    • http://www.harrysastroshed.com
Re: Carlos' development modules collection
« Reply #2 on: 2010 September 05 07:47:40 »
Hi

Excellent work young man  :-*


Selective blend oh how I have missed you  ;D


Harry
Harry Page

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1341
  • We have cookies? Where ?
    • View Profile
Re: Carlos' development modules collection
« Reply #3 on: 2010 September 05 12:13:23 »
Fantastic work Carlos,

Definitely a pageful of useful information that needs to be added to my 'PixInsight Forum Favourites' folder - and then it is just a case of 'playing with' each of the modules as time allows.

Thanks a lot for all the hard work.
Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chr├ętien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline sleshin

  • PixInsight Old Hand
  • ****
  • Posts: 431
    • View Profile
Re: Carlos' development modules collection
« Reply #4 on: 2010 September 05 19:16:42 »
Thanks, Carlos, for your hard work and for posting these. Now to figure out how to use them. :D

Steve
Steve Leshin

Stargazer Observatory
Sedona, Arizona

Offline sleshin

  • PixInsight Old Hand
  • ****
  • Posts: 431
    • View Profile
Re: Carlos' development modules collection
« Reply #5 on: 2010 September 06 18:14:14 »
Hi Carlos,

I'm starting to try these new modules and in fact used the "Sharpen" module on an image I'm currently working on. I liked the result on small scale structures, hope to have it finished soon and will post it.

I wonder if, in your spare time ;D, you could provide some expanded documentation for these modules. In the Sharpen module, for example, there are 4 filter choices with rather unfamiliar names to me, so some explanation of what they do and under what circumstances each might be preferred would be very helpful. Some, even minimal, documentation for all the modules would be appreciated. Thanks in advance.

Steve
Steve Leshin

Stargazer Observatory
Sedona, Arizona

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2169
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: Carlos' development modules collection
« Reply #6 on: 2010 September 06 19:44:56 »
Yes, I plan to write the documentation for all the modules, but time is a very limited resourse, so most likelly this will be delayed until my summer (dic-feb).

In the meanwhile, this is a quick guide to the filters:

In the image processing field, there are some kind of filters called "high pass filters", because they attenuate the low frecuencies and enhance the high ones (small scales). This kind of filters are implemented as convolutions (see wikipedia for this operation), where you define a kernel operator that goes over the image. The laplacian operators are a family of these kernels that emulates (for discrete samples) the effect of the second derivative of a function. So, if you define a first derivative as the simple difference between a pixel and the next (for each direction), and you apply it two times, both directions, you'll end with a kernel with the Laplacian 4 shape:

[0  -1 0
-1 4 -1
0 -1 0]

You may interpret this as a comparison between the central pixel, and the 4 orthogonal neighbors.
So, if you combine the effect of this filter to the original image, in a certain percentage, you'll end with a sharpened result (because the filter emphatizes all edges).

The Laplacian 8 is like a generalization of the previous filter, where you include the 4 diagonal neighbors:

[-1 -1 -1
-1 8 -1
-1 -1 -1]

This tends to give smoother results, and is less sensible to noise.

The Weighted Laplacian 8 is a "combination" of the previous filters, adding more weight to the orthogonal pixels, making them more important in the calculation:

[ -1 -2 -1
-2 12 -2
-1 -2 -1]

Finally, comes Laplace of Gaussian, or LoG. As the name may suggest, this is in fact two operations. First you blur the image with a gaussian filter, and then you sharpen it with a laplacian. The idea is to get rid of noise and other unhomogenities before sharpening the image. This gives the smoother results of the above.
Also, as you increase the size of the gaussian blur, the image is flatter, i.e. it is harder to sharpen it, so the amount value has a different meaning/effect here, not as direct as with previous filters.

On a side note, note that the size parameter also works different for the LoG filter than the others. Here it sets the size of the blur operation. With the other filters, it is the distance between the central pixel and the ones you are comparing it to. This is kind of inspired in the ATrous wavelets, in the sence that the filter grows with holes, not a continuos function. This may generate artifacts, but also is faster and for visually enhance some structures, may be good enough (by the way, the Larson-Sekannina, or rotational gradient, works just with the same assumption).
« Last Edit: 2010 September 07 13:26:36 by Carlos Milovic »
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PTeam Member
  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • View Profile
    • http://www.carpephoton.com
Re: Carlos' development modules collection
« Reply #7 on: 2010 September 09 01:52:25 »
Hi Carlos,

I tried the CHT module on a cropped image with an out of focus star. This resulted in an empty, transparent image. I'm probably missing something about how to use this module. I'm a bit familiar with the HT in general but not how it was implemented here.

Thanks!
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2169
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: Carlos' development modules collection
« Reply #8 on: 2010 September 09 03:10:12 »
Hi Sander

First, a disclaimer: CHT should be applied to grayscale images only.
The result is a multiple channel image, where the channel index goes with the radius size. The CHT generates a three dimensional space, where X and Y are the usual space, and Z is the radius.
Now, to get it work in optimal way, very sharp edges should be defined in the image. It can't find the radius of a disc, it should be a circle, like the letter O :) (white data, black background).

So, please check each individual channel for a propper inspection. There should be several alpha layers. Also, try to improve the input image. If you have something like a ring or donut, the gradient of the image, or extracting a single wavelet layer may do the trick.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PTeam Member
  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • View Profile
    • http://www.carpephoton.com
Re: Carlos' development modules collection
« Reply #9 on: 2010 September 09 03:37:17 »
Hi Carlos,

ok, I extracted the L and re-applied CHT. I now have an image with 50 alpha channels. I'll have to check them all to see if I got anything :) I supose it would be helpful if the process could have a filter that rejects circle sizes that have too few hits. I know CHT starts with a circle size and then tries to find it but for CHT to be useful it should be able to find *any* circle within a range and highlight those :) Also the max of 100 is a bit low. The process is very very fast so I see no reason not to allow sizes up to min(width, height). Oh and it's 'differential' :)

After mucking with the image a little, using HT to remove the background noise and brightening the donut I'm seeing a clear CHT appear. I'm not sure why it's detecting the outer circle when the range doesn't allow that but this is pretty cool. Thanks very much for these!

PS probably a good idea to stick all modules for a platform into a single zip next time. Much easier to download and install that way. Easier to upload too :)
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Nocturnal

  • PTeam Member
  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • View Profile
    • http://www.carpephoton.com
Re: Carlos' development modules collection
« Reply #10 on: 2010 September 09 05:23:22 »
Hi Carlos,

are all your modules multi threading where possible?
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2169
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: Carlos' development modules collection
« Reply #11 on: 2010 September 09 05:31:08 »
Quote
PS probably a good idea to stick all modules for a platform into a single zip next time. Much easier to download and install that way. Easier to upload too smile

I was even thinking on grouping them in two or three modules... sorted by category. Well, that should be the next step, for the next holidays :D (luckily, we are celebrating our bicentennial anniversary of independence next week... 4 days). Anyway, I'm not convinced 100%, as some people just may want to try one or two of them, and may not want to install many buggy processes :D

I'll comment on the CHT things tomorrow... it is just too late now :P

PS: No multi threading in the tasks that I specifically write. Of course I use a lot functions that are implemented internally at the PCL, and most of them are already optimized/multithreaded.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2169
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: Carlos' development modules collection
« Reply #12 on: 2010 September 09 13:58:05 »
Quote
ok, I extracted the L and re-applied CHT. I now have an image with 50 alpha channels. I'll have to check them all to see if I got anything smile I supose it would be helpful if the process could have a filter that rejects circle sizes that have too few hits.

Well, let's start from some background: The CHT of a perfect, ideal circle, is a double cone, each one "opening" to different orientation (up and down). What you are seeing in this process are a series of horizontal cuts of these cones (for ***every*** circle). Since the cone is pretty regular, with a "clear" convergence to the optimal circle radius, you don't have to analize every single radius value in a wide range. You may get there with a multiscale approach :) First try a large differential ;) Let's say, 10px. Then, you may find a very coarse range where the solution may be found, then, try 1px on it, or 2px, and repeat the process if needed.

Indeed such kind of rejection may be doable. I sort of normalized each plane, so the sum of the perimeter of the cone at each plane is always 1. This means, the pixel value gives something like the probability of being the optimal circle radius. Of course, if you have noise, adjacent circles, overlaping circles, etc., this may be screwed a bit, and you may end with values over 1, or give false positives.
Quote
I know CHT starts with a circle size and then tries to find it but for CHT to be useful it should be able to find *any* circle within a range and highlight those smile

If you want to plot the CHT, then you have to quantize the radius. Since the CHT is a three dimensional space, the only elegant way to manage that I found is to create alpha channels. An array of single channel images is just messy for plotting purposes.
Anyway, If you want just to detect the circle (one circle, for example), without plotting the CHT space, a more sophisticated algorithm may be written, where the maximum value of each layer is calculated, and then compared them all to find the most probable circle. It may be even a recursive algorithm, using subpixel accuracy. But I was too lazy at the time ;) Jokes apart, this is not hard to implement. Just needs time, and when I wrote this module, I needed the results quickly for a cooperation with researches of the Physics faculty.

Quote
Also the max of 100 is a bit low. The process is very very fast so I see no reason not to allow sizes up to min(width, height). Oh and it's 'differential' smile
Thanks for the correction :) I need a spell checking machine :D
I can't remember the reason... I'll increase it next time I upgrade the code (for the "module compression task" :P). The workaround: downsample the image.

Quote
After mucking with the image a little, using HT to remove the background noise and brightening the donut I'm seeing a clear CHT appear. I'm not sure why it's detecting the outer circle when the range doesn't allow that but this is pretty cool. Thanks very much for these!

As I said, I normalized the output so the values gives a sense of the probability of having a circle. For displaying purposes a STF is needed to see how pretty is this space :D

I don't think that it is detecting the outer "circle". If you analyze the image from the CHT point of view, there are not just 2 circles, at the boundaries of the donut. There are almost infinite circles between them, for every intermediate radius. That's why I suggested to calculate the modulus of the gradient, and use that as the input image. Also, if you look at very small radius values, the CHT will "find" all the circles that are located inside the donut. Remember, all the bright pixels are signals, and the CHT looks for every circle that fits the data.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nocturnal

  • PTeam Member
  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • View Profile
    • http://www.carpephoton.com
Re: Carlos' development modules collection
« Reply #13 on: 2010 September 10 14:07:12 »
Hi Carlos,

I ran the ReadPSF module on one of my images. Not quite clear what it does or how to operate it. The problem is that a) ReadPSF gets added to the process history even though I'd think it's a read-only process that measures and reports. I don't think it should go in the process history in that case. Open for debate. b) the real problem is that when loading a process history with the ReadPSF module I get an error 'unable to lock boolean parameter isSimmetric'.

More general it seems one of your modules can't deal with broadcast readout data. See the thread in the bugs forum about that. The problem went away when I disabled all your modules. Frankly I'm not sure what the broadcast feature is for and if I should simply disable it but it's probably a bug you should take a look at.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2169
  • Join the dark side... we have cookies
    • View Profile
    • http://www.astrophoto.cl
Re: Carlos' development modules collection
« Reply #14 on: 2010 September 10 14:13:52 »
Hi Sander

First of all, thank you very much for trying out all the processes ;) Your input is highly appreciated.

a) Yes, 100% right. This is something I always forget to modify. I'll see this weekend how to disable that.
b) Ooops. Yes, I saw that bug once, and I forgot to correct that. Count on that solved ASAP (most likely this sunday).

Yes, that bug is in ColorRange. I posted a "disclaimer" below the download links last week, with the workaround in the case the errors are shown. To avoid any problems, just disable it.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com