Gradient Domain Operations, take 3 (was: Gradient Domain HDR Compression)

[georg.viehoever]

Hi,

here is the third installment of the Gradient Domain Module. The new feature this time is the Gradient Domain Mosaic Composition process, something I played with a while ago (http://pixinsight.com/forum/index.php?topic=2124.0). The general idea is that you use gradient domain operations to smoothly connect a number of pictures, with no small scale gradients.

The effect is best explained be looking at the attached screenshot:
- left: Some portrait embedded into a black background
- center: Juan's portrait is smoothly embedded into to gray background, as merged by the Gradient Domain Mosaic Composition process.
-- the differences are adjusted by generally brightening the backgorund
-- remaining differences are smoothed over a large area, very much in the same fashion a thin elastic membrane would respond to external forces.
-- There are no longer any hard gradients between the two regions - if they can be avoided at all.
- right: the GUI.
-- The top is a list of files, very much like the list of files in ImageIntegration. Below this is the parameter section
-- You can select how two images are merged: Overlay means that images are stacked on top of each other, with the large image "winning". "Average" computes the average brightness in regions where some images overlap.
-- Black Point allows to specify the brightness level that is considered Background/transparent. 0 (black) is the default - just what ImageRegistration will produce.
-- Finally, "Generate Mask" causes the process to generate an additional picture  showing the different image regions.

Other features of this release:
- Gradient Domain HDR compression is the second process in this module. No major changes here.
- By default, the module will now use the solver developed by Carlos (used with permission  )
-- That means that no FFTW library is needed any longer (FFTW would prevent inclusion into the PI product - FFTW charges for commercial use.).
-- For most images, this solver is fast enough.
-- However, it occasionally shows some artifacts. I will have to work with Carlos to find the bug.
- If you like, FFTW can be used as an alternative by using the suitable #define in GradientsBase.h and linking with libfftw3.so. It is faster, consumes less memory, and does not show the problems.
- Again, the whole module has been tested on Fedora14-x64. I don't see any reason why it should not compile and work on other PI platforms.

Attached are the sources. An application example will follow later.

Georg

[Harry page]

Hi

If I read this right could this be used with a deep sky mosaic to help merge the edges ?  how would you do this ?


Harry

[georg.viehoever]

Hi,

Harry recently showed in a video http://pixinsight.com/forum/index.php?topic=1167.msg21839#msg21839 how StarAlignment supports merging moaics. Linear fit brings images into a similar brightness scale. This works fine - if you were able to remove any remaining background gradients during the procesing of the separate images. This is not always possible, for example if you cannot find sufficient free space for DBE - just what happened to me some time ago.

The first attachment shows:
- top rows: two shots of M31
- bottom left: StarAlignment in Mosaic mode without Linear Fit activated. We clearly see the huge difference between the two parts
- bottom right: same with Linear Fit. The images are almost the same. But: image 2 has an internal gradient that leads to clearly visible seems in the center of the image.

How can GradientDomainMergeMosaic help here?
- it can adjust for the brightness differences. It will not help to adjust for differencs in texture.
- Screenshot 2 shows the result of the process on the M31 images:
-- top Left: merge result
-- top middle: mask displaying the different regions
-- top right: GUI
-- bottom: result with mask as overlay.
- We still see some texture differences, but the brightness jumps are gone.
- There are still some artifacts where stars are just on the border between two regions, for example in the region  marked as "Preview 1". I am not yet sure how to handle such cases, ideas are welcome.

Enjoy,

Georg

[georg.viehoever]

Hi Harry,

If I read this right could this be used with a deep sky mosaic to help merge the edges ?  how would you do this ?

You are right. General procedure:
- align your mosaic tiles using Star Alignment.
-- First, create a big reference frame using the "Register/Union-Mosaic" functionality
-- Then register all your tiles again in "Register/Match Images" mode, using the reference frame as the "Reference Image"
- The merge the registered frames files using the MergeMosaic process.

BTW: Not my invention. Gradient Domain Compositing has been around for quite a while, and MS apparently has used it to stitch the images in its "WorldWide Telecope" project http://research.microsoft.com/en-us/um/people/hoppe/proj/spherestitch/.

Georg

[Nocturnal]

Great work Georg!

[Harry page]

Hi

I was suggesting this sort of thing to Juan the other day   8)

The ultimate thing would be if this was implamented in the image reg / mosaic tool , too much to ask I know

But great work though should really help and look forward to seeing a windoz version so I can Play 

Harry

[Harry page]

Hi

I may be oversimplifying this but why cant the edge say 100 pixels feather in from 1 image to another without averaging all the overlapping pixels and avoiding the problems this causes 

Sorry if I am being dumb 

Harry

[zvrastil]

Hi Georg,

this is quite interesting module! Great job!
I tried to compile it on windows. After some minor changes to code (attached) to satisfy not-so-standard M$ compiler, I compiled it successfully.
However, I'm not able to install it into PixInsight. I'm getting following error:

Installing 1 module(s):
C:/Program Files/PixInsight/bin/GradientDomain-pxm.dll
*** PixInsight API Error: MetaProcess: GradientsMergeMosaic: Invalid LengthQueryParameter routine: path
<* failed *>
*** PixInsight API Error: Module: C:/Program Files/PixInsight/bin/GradientDomain-pxm.dll: Module initialization error: Exit code: 1
*** Warning: No modules were installed.


I'm not sure what does it mean. It never happened to me with my modules.

regards, Zbynek

[Juan Conejero]

Hi,

Georg, this is a great tool. Well done!

Later this afternoon I'll try to post versions for Windows and Mac OS X, 32/64.

Zbynek, you've been faster than me I'll fix those little problems too. The problem is simply that the ProcessImplementation::ParameterLength() reimplementation does not provide a valid length for the "path" process parameter (which must be a descendant of MetaString; I haven't seen the code yet). Are you using the last PCL version? Here it is:

http://pixinsight.com/developer/pcl/download/PCL-01.00.94.0367-20110529.tar.gz

Unfortunately, there is an incompatibility with previous versions, so using the latest PCL is necessary.

[zvrastil]

Hi Juan,

I have still PCL-01.00.91.0338-20101220 installed, so I'll update it immediately. To be honest, it's very difficult to even find the download link for PCL now, when it's no longer under Software Distribution link. I was never able to find the download link anywhere but in forum and in the storm of 1.7 posts, I missed the one about new PCL version. Would it be a problem to add PCL download link to Software Distribution? But let's not hijack Georg's module thread with this little bit off-topic theme .

regards, Zbynek

Update: with new PCL, it compiles and installs without problem .

[georg.viehoever]

Hi,

thanks for your kind words  .

- What the module now needs most is experience from other users, so we can find possible bugs or or kinks. Therefore I am very grateful to Juan for publishing compiled versions.
- PCL download: see http://pixinsight.com/forum/index.php?topic=3060.0
- Invalid path problem: The code is basically taken from ImageCalibration, so if there is an issue then it would be in ImageCalibration as well.

Keep up the good work.

Georg

[Juan Conejero]

Hi all,

The new GradientDomain module is now available as an update from a new development repository that we have created today. To add the new repository, please follow these steps:

- From the main menu, select: Resources > Updates > Manage Repositories

- On the Manage Update Repositories dialog, click the Add button.

- Enter the URL of the new repository: http://update-devel.pixinsight.com/

- Click OK.

- Select: Resources > Updates > Check for Updates

Now the new module should be available for your platform as an update from the new update-devel repo. Enjoy!

[Nocturnal]

That worked fine Juan thanks.

One minor bug: when I started PI it said I had an update. I ignored this, added the devel repository. PI then told me there were no updates. I then explicitly checked for updates and it showed me both. The update then succeeded. Not something to worry about but it could be confusing.

[Juan Conejero]

Hi Sander,

Thanks for the info. Perhaps the check for updates should be done automatically after modifying the update repositories?

[Nocturnal]

I think that would be good Juan. It seems there is a check but due to the updated repository it returns '0' or 'nil'. Just a previous un-tested code path