PixInsight Forum (historical)
Software Development => New Scripts and Modules => Topic started by: Carlos Milovic on 2010 September 04 15:57:20
-
This is the new repository for the modules that I'm working on, published here as beta products:
http://www.astrophoto.cl/Research.html
Please keep in mind that all of these are development processes, not final products. Bug reporting, or enhancing suggestions are highly welcome.
Now compiled for PCL 02.01.00.0749 (PI Core 1.8.4.1171 and above). Only 64bit releases will be available. Windows, and Linux only, for the moment.
Compilations for PCL 1.x are considered obsolete. Latest versions can be found here:
http://pteam.pixinsight.com/pcldev/carlos/PCL1
-----------------------------------------------------------------------------------------------------------------
For a description of the modules, enter the first link above.
-
Good grief!
Excellent work Carlos! I'll be taking a look at all of them but I'm particularly interested in your Hough Transform as I looked at that some time ago.
-
Hi
Excellent work young man :-*
Selective blend oh how I have missed you ;D
Harry
-
Fantastic work Carlos,
Definitely a pageful of useful information that needs to be added to my 'PixInsight Forum Favourites' folder - and then it is just a case of 'playing with' each of the modules as time allows.
Thanks a lot for all the hard work.
-
Thanks, Carlos, for your hard work and for posting these. Now to figure out how to use them. :D
Steve
-
Hi Carlos,
I'm starting to try these new modules and in fact used the "Sharpen" module on an image I'm currently working on. I liked the result on small scale structures, hope to have it finished soon and will post it.
I wonder if, in your spare time ;D, you could provide some expanded documentation for these modules. In the Sharpen module, for example, there are 4 filter choices with rather unfamiliar names to me, so some explanation of what they do and under what circumstances each might be preferred would be very helpful. Some, even minimal, documentation for all the modules would be appreciated. Thanks in advance.
Steve
-
Yes, I plan to write the documentation for all the modules, but time is a very limited resourse, so most likelly this will be delayed until my summer (dic-feb).
In the meanwhile, this is a quick guide to the filters:
In the image processing field, there are some kind of filters called "high pass filters", because they attenuate the low frecuencies and enhance the high ones (small scales). This kind of filters are implemented as convolutions (see wikipedia for this operation), where you define a kernel operator that goes over the image. The laplacian operators are a family of these kernels that emulates (for discrete samples) the effect of the second derivative of a function. So, if you define a first derivative as the simple difference between a pixel and the next (for each direction), and you apply it two times, both directions, you'll end with a kernel with the Laplacian 4 shape:
[0 -1 0
-1 4 -1
0 -1 0]
You may interpret this as a comparison between the central pixel, and the 4 orthogonal neighbors.
So, if you combine the effect of this filter to the original image, in a certain percentage, you'll end with a sharpened result (because the filter emphatizes all edges).
The Laplacian 8 is like a generalization of the previous filter, where you include the 4 diagonal neighbors:
[-1 -1 -1
-1 8 -1
-1 -1 -1]
This tends to give smoother results, and is less sensible to noise.
The Weighted Laplacian 8 is a "combination" of the previous filters, adding more weight to the orthogonal pixels, making them more important in the calculation:
[ -1 -2 -1
-2 12 -2
-1 -2 -1]
Finally, comes Laplace of Gaussian, or LoG. As the name may suggest, this is in fact two operations. First you blur the image with a gaussian filter, and then you sharpen it with a laplacian. The idea is to get rid of noise and other unhomogenities before sharpening the image. This gives the smoother results of the above.
Also, as you increase the size of the gaussian blur, the image is flatter, i.e. it is harder to sharpen it, so the amount value has a different meaning/effect here, not as direct as with previous filters.
On a side note, note that the size parameter also works different for the LoG filter than the others. Here it sets the size of the blur operation. With the other filters, it is the distance between the central pixel and the ones you are comparing it to. This is kind of inspired in the ATrous wavelets, in the sence that the filter grows with holes, not a continuos function. This may generate artifacts, but also is faster and for visually enhance some structures, may be good enough (by the way, the Larson-Sekannina, or rotational gradient, works just with the same assumption).
-
Hi Carlos,
I tried the CHT module on a cropped image with an out of focus star. This resulted in an empty, transparent image. I'm probably missing something about how to use this module. I'm a bit familiar with the HT in general but not how it was implemented here.
Thanks!
-
Hi Sander
First, a disclaimer: CHT should be applied to grayscale images only.
The result is a multiple channel image, where the channel index goes with the radius size. The CHT generates a three dimensional space, where X and Y are the usual space, and Z is the radius.
Now, to get it work in optimal way, very sharp edges should be defined in the image. It can't find the radius of a disc, it should be a circle, like the letter O :) (white data, black background).
So, please check each individual channel for a propper inspection. There should be several alpha layers. Also, try to improve the input image. If you have something like a ring or donut, the gradient of the image, or extracting a single wavelet layer may do the trick.
-
Hi Carlos,
ok, I extracted the L and re-applied CHT. I now have an image with 50 alpha channels. I'll have to check them all to see if I got anything :) I supose it would be helpful if the process could have a filter that rejects circle sizes that have too few hits. I know CHT starts with a circle size and then tries to find it but for CHT to be useful it should be able to find *any* circle within a range and highlight those :) Also the max of 100 is a bit low. The process is very very fast so I see no reason not to allow sizes up to min(width, height). Oh and it's 'differential' :)
After mucking with the image a little, using HT to remove the background noise and brightening the donut I'm seeing a clear CHT appear. I'm not sure why it's detecting the outer circle when the range doesn't allow that but this is pretty cool. Thanks very much for these!
PS probably a good idea to stick all modules for a platform into a single zip next time. Much easier to download and install that way. Easier to upload too :)
-
Hi Carlos,
are all your modules multi threading where possible?
-
PS probably a good idea to stick all modules for a platform into a single zip next time. Much easier to download and install that way. Easier to upload too smile
I was even thinking on grouping them in two or three modules... sorted by category. Well, that should be the next step, for the next holidays :D (luckily, we are celebrating our bicentennial anniversary of independence next week... 4 days). Anyway, I'm not convinced 100%, as some people just may want to try one or two of them, and may not want to install many buggy processes :D
I'll comment on the CHT things tomorrow... it is just too late now :P
PS: No multi threading in the tasks that I specifically write. Of course I use a lot functions that are implemented internally at the PCL, and most of them are already optimized/multithreaded.
-
ok, I extracted the L and re-applied CHT. I now have an image with 50 alpha channels. I'll have to check them all to see if I got anything smile I supose it would be helpful if the process could have a filter that rejects circle sizes that have too few hits.
Well, let's start from some background: The CHT of a perfect, ideal circle, is a double cone, each one "opening" to different orientation (up and down). What you are seeing in this process are a series of horizontal cuts of these cones (for ***every*** circle). Since the cone is pretty regular, with a "clear" convergence to the optimal circle radius, you don't have to analize every single radius value in a wide range. You may get there with a multiscale approach :) First try a large differential ;) Let's say, 10px. Then, you may find a very coarse range where the solution may be found, then, try 1px on it, or 2px, and repeat the process if needed.
Indeed such kind of rejection may be doable. I sort of normalized each plane, so the sum of the perimeter of the cone at each plane is always 1. This means, the pixel value gives something like the probability of being the optimal circle radius. Of course, if you have noise, adjacent circles, overlaping circles, etc., this may be screwed a bit, and you may end with values over 1, or give false positives.
I know CHT starts with a circle size and then tries to find it but for CHT to be useful it should be able to find *any* circle within a range and highlight those smile
If you want to plot the CHT, then you have to quantize the radius. Since the CHT is a three dimensional space, the only elegant way to manage that I found is to create alpha channels. An array of single channel images is just messy for plotting purposes.
Anyway, If you want just to detect the circle (one circle, for example), without plotting the CHT space, a more sophisticated algorithm may be written, where the maximum value of each layer is calculated, and then compared them all to find the most probable circle. It may be even a recursive algorithm, using subpixel accuracy. But I was too lazy at the time ;) Jokes apart, this is not hard to implement. Just needs time, and when I wrote this module, I needed the results quickly for a cooperation with researches of the Physics faculty.
Also the max of 100 is a bit low. The process is very very fast so I see no reason not to allow sizes up to min(width, height). Oh and it's 'differential' smile
Thanks for the correction :) I need a spell checking machine :D
I can't remember the reason... I'll increase it next time I upgrade the code (for the "module compression task" :P). The workaround: downsample the image.
After mucking with the image a little, using HT to remove the background noise and brightening the donut I'm seeing a clear CHT appear. I'm not sure why it's detecting the outer circle when the range doesn't allow that but this is pretty cool. Thanks very much for these!
As I said, I normalized the output so the values gives a sense of the probability of having a circle. For displaying purposes a STF is needed to see how pretty is this space :D
I don't think that it is detecting the outer "circle". If you analyze the image from the CHT point of view, there are not just 2 circles, at the boundaries of the donut. There are almost infinite circles between them, for every intermediate radius. That's why I suggested to calculate the modulus of the gradient, and use that as the input image. Also, if you look at very small radius values, the CHT will "find" all the circles that are located inside the donut. Remember, all the bright pixels are signals, and the CHT looks for every circle that fits the data.
-
Hi Carlos,
I ran the ReadPSF module on one of my images. Not quite clear what it does or how to operate it. The problem is that a) ReadPSF gets added to the process history even though I'd think it's a read-only process that measures and reports. I don't think it should go in the process history in that case. Open for debate. b) the real problem is that when loading a process history with the ReadPSF module I get an error 'unable to lock boolean parameter isSimmetric'.
More general it seems one of your modules can't deal with broadcast readout data. See the thread in the bugs forum about that. The problem went away when I disabled all your modules. Frankly I'm not sure what the broadcast feature is for and if I should simply disable it but it's probably a bug you should take a look at.
-
Hi Sander
First of all, thank you very much for trying out all the processes ;) Your input is highly appreciated.
a) Yes, 100% right. This is something I always forget to modify. I'll see this weekend how to disable that.
b) Ooops. Yes, I saw that bug once, and I forgot to correct that. Count on that solved ASAP (most likely this sunday).
Yes, that bug is in ColorRange. I posted a "disclaimer" below the download links last week, with the workaround in the case the errors are shown. To avoid any problems, just disable it.
-
Hi Carlos,
thanks again for providing all these interesting modules! I'll have to spend some time to learn how to use them.
-
Hi guys
I uploaded the ReadPSF and the FFTRegister modules. In the first one, I fixed the bugs with the History update, and the icon's error. FFTRegister, on the other side, was slighty modified on the wavelets extraction. Should work better now.
-
Don't know how exatly, but I fixed the bug in ColorRange. No error message is shown when clicking on the image (or passing over it).
Links replaced.
-
Thanks Carlos, will give them another go. Could you please put them in a single archive next time for easier downloading and install? In fact the upload would be easier too :)
-
Yeah, I'll compress them into less modules, and zip them... I just did not have the time these days (now I have to finish the code for the research I'm on at the university...).
-
PS: upload is quite easy right now :P I just use gFTP and drop the files on the server, as they are already in a single folder on my HDD. :D
-
Quick STF and new mask tools are great.
Max
-
Carlos...
wonderful work and thank you very much! BUT...
We OSX users feel left out ;D
-
Sorry :( I don't have a mac... and I don't have a clue how to mount virtual machines, so indeed this is going to be left out from my side. As soon as I compact the processes, I'll send the code to Juan, to see if he may compile it for you OSX users.
-
Hello,
I would also like to try these new modules...
Am i the only one on "old" 32 bit machine ???
Sharpen module would be nice for a start ;)
Thanks,
Jean Guy Moreau
Quebec
-
Am i the only one on "old" 32 bit machine ???
One more here (W32) ;)
-
Hi guys
I reduced the amount of installable modules to four, grouping the processes by their category. For the moment, I compiled Win32 and Win64 versions. Linux ones will come later (I have to figure out how to use the makefiles, instead of creating eclipse projects, what at this point is getting kinda boring :D )
Enjoy!
-
Hi Carlos,
the linux makefiles I got from Juan for the profile module were pretty close. I assume they're identical to the ones generated by the makefile script. I made some modifications to make the 32 and 64b builds work properly on a 64b machine. Let me know if things don't work for you and I'll attach the makefile here. I haven't yet tried to update the makefile generator script with my fixes.
-
Hi Sander
The fact, is that I don't now how to use them :D I tried executing the makefile file from the console, and I only got error messages... I'll create a new thread on this. Surely I'm not the only one hitting the wall with my head :D
-
Hi,
Links show a "403 Forbidden" error page when I tray to download the files.
Regards,
Enzo.
-
Hi Enzo
Thanks. The folders had incorrect permissions. I fixed that. Also, I uploaded the X11_64 (linux) files. X11_32 will be uploaded per request, since I was unable to compile them right now.
-
Cool. Thanks Carlos!
-
Just a quick note to let you know that I recompiled all the modules for the new PCL. This actualization also included some error/bug fixing, some minor enhancing (iterations parameter in the Blur process, and new readout mode in ReadPSF), and a few new processes (ButterworthFilter, LogStretch, HDRC and HomomorphicFilter).
Other new processes are in the way. Most of them are simple implementations of algorithm I've being revising for a image processing course, where I'm the T.A. Also expect more news in the High Dynamic Range field, and maybe other Poisson solver related issues.
Have fun, and feedback is highly welcome, specially with the new module!
-
A Win32 compilation of the libraries would be very useful. Although it is a declining platform, it is still widely used. From the (unscientific) poll in http://pixinsight.com/forum/index.php?topic=1128.0 (http://pixinsight.com/forum/index.php?topic=1128.0) 30% of Pixinsight users still use Win32. Juan could provide better statistics from the download page and the update system.
I have PI installed in two computers: a desktop with Win7 x64 and a notebook with Win7 x86 that I use for controlling the telescope and cameras.
-
Hi Andres
Yes, the problem right now is that I have no 32bits computers working... And I have no clue on how to compile that from a 64bits architecture. Anyway, I uploaded the source code, so you may compule them yourself, if needed.
-
Hi
Sorry for being stupid , but where do we download these from ;)
Harry
-
Sorry for being stupid , but where do we download these from ;)
Download Links (also to source code) at the beginning of this thread (message http://pixinsight.com/forum/index.php?topic=2275.msg15008#msg15008).
Georg
-
Hi
Thanks for that , I suppose you should always start at the beginning :D
Harry
-
Hi
Tried your HDR compresion and all my computer did was hang ???
Harry
-
It is VERY slow :) try it with small images, no more than 1000x1000. These kind of images take nearly 1 minute in my computer. This process works at an O(width^2) order, so try to keep it down. The problem is on the poisson solver. This is covered in much more depth in Georg's thread. We are working on better solvers.
-
Hi
Yes i tried a smaller image about 1500 x 1000 and still nothing after 5 mins , will reboot computer as it seems to have caused a memory problem ;)
Harry
-
Quick note to let you know that I uploaded new source code for the HDR module, and a new Linux x64 compilation. Win x64 compilation will be ready sometime through this week.
See first post for the links.
-
New HDR module and InterChannelCurves uploaded, Linux and Windows, 64bit versions.
-
Hi Carlos,
Do you have the modules for 1.8 RC3? :'( I can't process without some of them!!!, InterChannel and HE at least
Saludos.
Alejandro.
-
Alejandro, estoy en la hacienda ahora. Vuelvo en una semana. Ahi los compilare.
-
Excelente Carlos, los espero. Que envidia que estes disfrutando del mejor de los cielos. Aquí no para de llover.
Saludos a todos por allá.
Alejandro.
-----
Excellent Carlos, I'll wait for them. I envy you're enjoying the best of heaven. Here does not stop raining.
Regards,
Alejandro.
-
Repository updated for PCL 2.0 (PI Core 1.8RC3). All development processes in beta phase were included, plus a couple of alpha releases. See the list for descriptions.
Windows' compilations will come soon.
-
Thank you Carlos, fantastic! 11 modules downloaded and installed.
Saludos. Alejandro.
-
Hi Carlos,
I tried to install some of your modules (dll's) in a windows 7 64-bit PI 1.8 RC3 platform, but it complains about not resolving PCL/API variables. Any hints?
best
Ignacio
-
Hi Ignacio, Carlos's modules are available for linux, not yet for windows.
Saludos, Alejandro.
-
I've being unable to install the x64 development tools at my office, the installer fails... I'll try to install them at home tomorrow.
-
Any luck?
-
Hi Carlos,
Any progress on getting these modules compiled for use with PI 1.8 and Windows 7 64 bit?
Steve
-
Sorry, I've being busy with a lot of things. I'll try to set the 64bit environment at home this weekend.
-
Thanks, Carlos. Please know that your contributions to PI and this forum are priceless and sincerely appreciated. So an extra thanks for all your hard work.
Steve
-
New repository location, inside my homepage. See the first post for more details. Now Windows and Linux compilations avalaible (except windows compilation for TaylorRegister).
-
if the source is available i can probably build them for macosx
rob
-
Carlos,
The download links for the modules isn't working. Get an "Error 403 - Forbidden" message.
Thanks for getting these updated.
Steve
-
There was a problem with the atributes, for the dll files. Linux files were fine.
Rob, I'll contact you.
-
New OSX compilations, thanks to Rob (pfile).
-
Excellent new Carlos, thank you!
Saludos, Alejandro
-
Thanks Carlos and Rob. ICCurves now on PC and Mac...brilliant!
Chris
-
no problem and i'm happy to do this whenever there are new modules or updates.
one thing though - we ran into a very strange compatibility problem on macosx in NotchFilter and HDRCompression. we found a workaround but be on the lookout for any strange behavior of these new processes on macosx… so far they look okay to me, but just sayin'
rob
-
Hello Carlos
I upgraded to the latest PI version, and it does not recognice your modules collection.....
I you have the time, could you please recompile them wit the new PCL library?
Geert
-
Thanks! I'll recompile them tomorrow for linux. I'll upload the windows releases this monday (I have no internet these days at home, where my windows machine is).
-
Linux files are ready. Tonight I'm going to do the window's ones. I'm going to contact Rob to see if he can help me with the OSX releases.
-
:) :)
-
Forgot to say that I'll upload the new modules (linux and windows) tomorrow... it was a note to let you know that I was working on this. :P
-
All modules updated to the latest PCL release, Win, Linux and OSX.
-
Thank You Carlos. :)
-
All the modules have been updated to the current PCL release. Linux and Win only, for the moment.
-
Carlos
Thank you very much!
Mike
-
Good morning friends, sorry for my silly question, but how do I install the Carlos commands in my PixInsight version 1.8 X64 with Windows 7 64-bit?
I have looked in the forum and in the PixInsight page but I did not find this information.
Rômulo.
www.astronomiaemvix.blogspot.com.br
-
Hi Rômulo,
See this short video:
http://pixinsight.com.ar/en/docs/36/pixinsight-install-modules.html (http://pixinsight.com.ar/en/docs/36/pixinsight-install-modules.html)
(Linux example, but is similar in Windows)
Greetings!,
Enzo.
-
Thank you Guy!
Hi Rômulo,
See this short video:
http://pixinsight.com.ar/en/docs/36/pixinsight-install-modules.html (http://pixinsight.com.ar/en/docs/36/pixinsight-install-modules.html)
(Linux example, but is similar in Windows)
Greetings!,
Enzo.
-
so with the full release of the latest PI on OSX, can these development modules be compiled for OSX now?
thanks
rob
-
Sorry to ask a low level question.
Can I use your scripts on my MacBook Pro?
Bob
-
Hi Carlos:
will you planning to update the PCL Processing Modules for 1.8.5 PI new Version?.
Many <thank by your coments
Federico
-
Carlos,
I would like to use enter channel curves with the new PI version 1.8.5.
However, when I tried to install it, I get memory errors and the file is not installed.
Are the old modules still compatible with the new version of PI?
Are you planning to update the modules?
Thanks,
Eric
-
Hi Carlos !! I hope everithing is ok
Do you have any news about the update for yours modules to 1.8.5 PI version?
I´m waiting anxious for this !!
Thanks And Regards !!
Federico