PixInsight Forum (historical)
Software Development => New Scripts and Modules => Topic started by: georg.viehoever on 2011 July 31 07:02:41
-
Hi,
here is the latest release of the GradientDomain module. News:
- The feathering process has been accelerated quite considerably, thanks to hints by Juan. A feathering with radius 50 has been reduced from 20 minutes to just 5 seconds 8) .
- The problems at the border to the global background are gone, see attached screenshot. This was achieved by changing the handling of gradients at the border of images, using the gradients at the actual border where there are no other images. While it still is possible that problems happen (e.g. in corners between images), it is quite unlikely.
- I added some timing output to be able to identify performance bottlenecks in larger mosaics that testers may try.
- And last but not least, the module comes with decent PIDoc documentation, as should any slightly complex tool do O:) .
Please note that the module is still beta. Possible areas of improvement:
- No process icon
- No way to cancel the process
- incomplete/non existent progress reporting
- Code review: The whole source would certainly benefit from a thorough code review. I am a believer in code reviews :angel: .
- speed: the current solver is 3x slower than an FFTW solver. Still, for Harry's NA example, the process runs for only 55 seconds, which is quite acceptable, so we may decide to stick with the current solver. With FFTW, it needs only 25 seconds. Other processes elements also may benefit from some performance aware coding.
And of course, we need much more experience with application to real mosaics. The tests that some of you have done on previous incarnations of this process have been most helpful. Please do not hesitate to report test results and suggest improvments.
Enjoy,
Georg
PS: Juan, feel free to publish compiled versions. The documentation is not contained in the attached tar.gz file due to size issus. Please let me know where to mail it.
-
Hi Georg,
Compiles nicely on Linux 32/64; starting VMware's Windows virtual machine right now.
A few things:
- You've put a text line to describe the tool on each interface (e.g.: 'allows to merge registered images without any visible seams''). It is customary to avoid this kind of 'inline descriptions' in PixInsight tools. If you really like them then there's no problem, but I recommend removing them for the sake of coherence.
- I cannot revise the code in depth right now, although a quick inspection shows me many areas where we can improve things. The main improvement has already been done with separable convolutions, though. The solver is the next bottleneck; I hope we'll manage to reach the same level you've seen with FFTW --but without FFTW of course. Maybe not 'the fastest of the west' but quite close. This is a matter of time and some work to integrate a good cosine transform in PCL. For now, the tool works very well as it is.
- If you don't mind I'm going to announce availability of this module to all PixInsight users through a massive email. We need a lot of people trying it out.
- Documentation: if the whole thing (compressed PIDoc source code and figure images) is less than five megabytes or so, you can send it to me via email (where I am now I have a quite slow Internet connection). Otherwise feel free to upload it to your account on our server (If you don't have your access data let me know).
- Nice work :)
-
An additional thing I forgot to mention:
- The default combination mode should be 'overlay' in my opinion. This is the only correct way to merge mosaic images. By averaging, you are altering (improving by an average factor of sqrt(2) for each pair of adjacent mosaic frames) the SNR of the final mosaic only on the overlapped regions. After aggressive stretching and further processing the seams may easily become visible for this reason.
-
Documentation is on its way via email.
Georg
-
The module is now compiled on Linux 32/64 and Windows 32/64, and I am preparing the update-devel release.
I cannot have access to a Mac machine until September. So we need a volunteer with a Mac machine ready to compile Georg's source code for Mac OS X. Then if he/she sends me the -pxm.dylib files (32 and 64-bit versions), I'll publish them also through update-devel.
So we need a Mac OS X PI/PCL developer. ???
-
Hi
Brilliant stuff :police:
Can't wait to have a go :D
All PI users owe you a huge Bottle of wine ;D
Many Thanks Harry
-
The module is now compiled on Linux 32/64 and Windows 32/64, and I am preparing the update-devel release.
I cannot have access to a Mac machine until September. So we need a volunteer with a Mac machine ready to compile Georg's source code for Mac OS X. Then if he/she sends me the -pxm.dylib files (32 and 64-bit versions), I'll publish them also through update-devel.
So we need a Mac OS X PI/PCL developer. ???
i might be able to do this. where is the source? i have not been following the threads that carefully.
-
The source as attached to the first message in this thread http://pixinsight.com/forum/index.php?topic=3295.msg22678#msg22678 .
Georg
-
Hi
Had a quick go and works very well ;D
The average setting seems to give a slightley better result than the overlay ? when the star is at a edge
Regards Harry
-
The average setting seems to give a slightley better result than the overlay ? when the star is at a edge
Harry
That's right. It's because the averaging reduces the differences at both sides of the seam. Feathering (->Feather radius parameter) should be even more effective to reduce those issues.
Georg
-
The source as attached to the first message in this thread http://pixinsight.com/forum/index.php?topic=3295.msg22678#msg22678 .
Georg
okay, thanks. there have been a lot of threads for this module and i wanted to be absolutely sure i got the right one.
sorry i did not see this sooner, i don't know why your replies did not show up in my "unread posts since last visit" until now.
-
http://pixinsight.com/forum/index.php?topic=3304.msg22739#msg22739
-
Hi
I have had a good play with this wonderful tool :-* and can't throw it yet
It would be nice if I could select " views " rather than files , just a bit more friendly
Harry
-
Well, it is a good sign when people begin asking for creature comforts, isn't it?
Georg
-
Hi
A video of the masters work 8)
http://www.harrysastroshed.com/Mosaic.html (http://www.harrysastroshed.com/Mosaic.html)
Harry
-
Wow 8)!
-
http://www.harrysastroshed.com/Mosaic.html (http://www.harrysastroshed.com/Mosaic.html)
Terrific! :)
Do you mind if I send an email to all PI users announcing this video?
Georg: time to integrate this with StarAlignment? Unless you have something new or something important to change in the current module, I'm going to start revising the code.
-
Hi Juan,
I am currently very busy, so no progress with the integration, and no progress expected in the next few weeks. Go ahead with any changes you would like to make.
From a functional point of view, I have the following in my wishlist:
- Integration with StarAlignment
- I think that even after integration with StarAlignment, we should still offer a separate GradientMergeMosaic process for those that obtain aligned images by other means (e.g. via astrometry.net or DynamicAlignment)
- Offer a selection of views in addition to a list of files - just what Harry described and which already is offered by StarAlignment.
Georg
-
Hi
Of course you can announce the video as you see fit ;D and I will redo it when you have integrated it all together
Again many thanks to george for his excellent work
Harry
-
...Again many thanks to george for his excellent work
There are many more that contributed to this work. Here are the "credits" as I put them into the code:
Process_Label.SetToolTip("(c) 2011 Georg Viehoever, published unter LGPL 2.1. "
"With important contributions in terms of tests, test data, code and ideas "
"by Juan Conejero, Carlos Milovic, Harry Page, David Raphael, Geert Vanhauwaert, zvrastil, and others.");
Georg
-
fantastic work guys, however i found a bug, it doesnt like different sized images form the same camera (cropped the stacked images)
-
Hi,
after doing StarAlignment as shown in Harry's video, you should end up with identically sized images.
The code explicitly checks that images are the same size. It is not clear to me how the tool should handle differently sized images: At which position of image 1 should image 2 appear? Or should one of them be rescaled? Which one? For the moment, I would prefer to stick with equally sized images (just as ImageIntegration does).
Georg
-
Hi,
after doing StarAlignment as shown in Harry's video, you should end up with identically sized images.
The code explicitly checks that images are the same size. It is not clear to me how the tool should handle differently sized images: At which position of image 1 should image 2 appear? Or should one of them be rescaled? Which one? For the moment, I would prefer to stick with equally sized images (just as ImageIntegration does).
Georg
to be fair i havent watched the video yet, but i did assume it would be similar to doing a mosaic the old way where the orientation or size isnt that a problem.
-
to be fair i havent watched the video yet, but i did assume it would be similar to doing a mosaic the old way where the orientation or size isnt that a problem.
With this version of the tool, GradientMergeMosaic leaves the task of finding the correct positions for the mosaic tiles to the PI StarAlignment tool. The aligned tiles can then be merged using GradientMergeMosaic. Harry's video (and the online documentation, try the help button on the bottom right of the GUI) should be helpful to understand the workflow.
Georg
-
to be fair i havent watched the video yet, but i did assume it would be similar to doing a mosaic the old way where the orientation or size isnt that a problem.
With this version of the tool, GradientMergeMosaic leaves the task of finding the correct positions for the mosaic tiles to the PI StarAlignment tool. The aligned tiles can then be merged using GradientMergeMosaic. Harry's video (and the online documentation, try the help button on the bottom right of the GUI) should be helpful to understand the workflow.
Georg
thanks Georg, will read up fully
-
wow!!! gracias gracias gracias!!!
parece que sabian que tenia que usar esta herramienta.
un gran trabajo que debemos agradecer los usuarios, gracias. en mi primer mosaico problemas, ahora solucionado.
solo reportar, que en 3 imagenes de DSLR hubo fallo al unir la tercera "out of memory".
es fallo de 32 BITS de sistema? solucionaré trabajando en 64 bits? mi mujer tiene uno, podria unir con ese PC y luego retornar al mio.
para hacer la prueba, reducí resolucion al 50% y unió bien
-----------------------
(Bad google-translate)
wow! thans thanks thanks!
think they knew I had to use this tool.
a great job , thanks. in my first mosaic problems, now resolved.
single a report, which was 3 DSLR, images failed to unite the third "out of memory".
failure is 32-bit system? trick working in 64 bits? my wife has one that could unite in that PC and then back to mine.
to test, reduced to 50% resolution and join.
can give me solution to "out of memory" into three images? Work bits?
my job thanks to you, I'm new at this and am very happy
http://imageshack.us/f/808/mosaicop1.jpg/
Thaaaanks!!!
-
Thanks for your kind words!
Working on a 64 bit OS is certainly recommended for PI. You will see this again and again if you search the forum for "64 bits" or memory problems. I mostly work on Win7-x64 with 4 GBytes RAM. Same machine had lots of memory trouble with Vista-x86 (32 bits).
Georg
-
Hi,
Rogelio noted in http://pixinsight.com/forum/index.php?topic=3283.msg23368#msg23368 some strange artefacts ("a couple of those funny dark-looking "smudges"" using the words of Rogelio) at the margins of two merged images. An analysis revealed that GradientMergeMosaic sometimes produces artefacts when part of the images coincides with the border, while the other parts are only a few pixels away from the border. Such situations occur for instance if StarAlignment applies a small rotation to one of the aligned images. An example is shown on the left hand side of the attached screen shot. Large rotations appear to be no problem.
Workaround: Add a black 1 pixel border to all images that you feed to GradientMergeMosaic. This can be done using the Crop process, see right hand side of the screenshot.
Future versions of GradientMergeMosaic will be enhanced to do this automatically.
Georg
-
A great module! I just started using it and the seams are really nice. I do experience the same smudging-issues though and look forward to the update. There are no "feather radius" nor "shrink radius" parameters in the version I've downloaded from the repository though (mac-version). Is the mac-version up to speed?
Ginge
-
The mac version should behave just like the others. Can you provide a screenshot of what you see.
Georg
-
Sure George, it looks like this.
ginge
-
Looks as if this has never been updated from the first version.
Juan, any comments? ???
Georg
-
Hi,
The Mac OS X port of this module (kindly provided by Rob(pfile) (http://pixinsight.com/forum/index.php?topic=3304.0)) was released through the update-devel repository on July 31st as 20110731-macosx-x86_64-module.zip and 20110731-macosx-x86-module.zip, respectively for the 64-bit and 32-bit versions. The reference documentation was also distributed on the same date.
Ginge, do you have the update-devel repository enabled in PixInsight? See this thread (http://pixinsight.com/forum/index.php?topic=3335.0) for complete information.
-
The Mac OS X port of this module (kindly provided by Rob(pfile) (http://pixinsight.com/forum/index.php?topic=3304.0)) was released through the update-devel repository on July 31st as 20110731-macosx-x86_64-module.zip and 20110731-macosx-x86-module.zip, respectively for the 64-bit and 32-bit versions. The reference documentation was also distributed on the same date.
Ginge, do you have the update-devel repository enabled in PixInsight? See this thread (http://pixinsight.com/forum/index.php?topic=3335.0) for complete information.
Hi Juan!
Yes, I do, and from the Installed Updates list I see that I have the abovementioned update installed already. However, an earlier update has been installed after the 20110731-macosx-x86_64-module.zip , namely 20110703-macosx-x86_64-module.tar.gz. Maybe this is causing the problem, in that case, how do I uninstall it?
Best regards,
Ginge
-
Ginge,
Yes, that's the problem. Hmm, that should never happen; maybe you've discovered a bug in the update system, or something very strange has happened to your PI installation.
Anyway, you can fix the problem following these steps:
- Exit PixInsight if you have it running.
- Select the PixInsight.app application bundle (in Finder) and right-click it. Then select 'Show package contents'.
- A new Finder window opens showing the contents of the application bundle. Look for a file named 'updates.xri' within the package's directory tree. Normally it should be located on the Contents/MacOS subfolder.
- Open updates.xri with a good code editor. Dashcode may suffice.
- Locate a line starting with "<package fileName="20110731". This is the beginning of an XML element. Locate the end of this element, which is a line with "</package>". For example:
<package fileName="20110731 ...
...
</package>
You have to delete all of the text starting with "<package" and up to "</package>".
- Save the file.
- Run PixInsight and the update should be available to your installation. It will replace your current GradientDomain module.
Alternatively, you can simply delete your updates.xri file, but then you'll get all updates available since the initial release of 1.7.0, which can be a hassle.
Let me know if this helps. I'm going to investigate how is it possible that an older update replaces a newer one. I designed the whole update system to rule out this possibility, so something is happening...
-
Thanks Juan, that did the trick. I erased the update.xri and installed all updates once more. The GradientDomain module now looks more up to date, but PI still wants to install an older 20110703-macosx-x86_64-module.tar.gz update... which might prove a tad annoying after a while if it persists >:D
Clear skies!
Ginge
-
Hi Ginge,
Oops! This was an error in the update-devel repository for Mac OS X. It is now fixed; the 20110703 update will no longer appear when you check for updates (or upon application startup). Sorry for the extra trouble!
-
No worries :)
Ginge
-
Hi,
this is a new source release of the Gradient Domain Module. Main features:
- fixing bug disovered by RBA
- improved bubble help
- improved documentation
- new documentation for the GradientsHdrCompression tool.
Details are listed in the changelist below. The documentation is too large to be attached here. I am mailing it to Juan in the hope that he will distribute compiled modules and documentation using the update system.
Kind regards,
Georg
CHANGES.txt:
Version 00.05.00.0003 (2011-10-15):
- integrated Juan's changes into documentation (mainly image sizes)
- added information about the contributions of others to the author tag of .pidoc
- removed head label in GUI, and set tooltip on ProcessInstance itself for
both GradientsMergeMosaicInterface and GradientsHdrInterface. This is for conformance
with other PI tools
- GradientsHdrCompression: reduced range of LogMin/LogMax parameter after Vicent's remark that the parameters are
extremely sensitive. Also, the minimum logMin value is now coerced to zero instead of just
being a small number. The defaults are all set to "neutral" now.
- GradientsHdrCompression: Improved bubble help and other help texts
- GradientsHdrCompression: Added help page.
- GradientsHdrCompression: Added module to HDR category (was Intensity Transform) to documentation
- GradientsMergeMosaic: Added reference to video tutorial to doucmentation
- GradientsMergeMosaic: Fixed issue identified by RBA in http://pixinsight.com/forum/index.php?topic=3295.msg23480#msg23480
- Known issues:
-- GradientsHdrCompression: Realtime preview sometimes does not follow settings correctly. Also
has problems with STF.
-- GradientsMergeMosaic with large Feather Radius values shows artefacts at the border of the merged images, if the image has a staircase structure there. This unfortunately cannot be easily avoided, and is discussed in the documentation.
Version 00.04.00.0002 (2011-07-24)
-First release that is fully functional and usually delivers results of the quality that I had in mind
-
George, that last update is working beautifully! What a great tool! Thanks a bunch!
Clear skies,
Ginge
-
Newer message in http://pixinsight.com/forum/index.php?topic=3572.0, older messages in http://pixinsight.com/forum/index.php?topic=3289.0.
Georg