Author Topic: New Tool: HDRComposition  (Read 52449 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
New Tool: HDRComposition
« on: 2010 September 13 07:01:03 »
Hi all,

A new standard tool is now available for all supported platforms: HDRComposition.

HDRComposition allows you to integrate a series of images of the same subject with varying exposures into a single high dynamic range composite image. HDRComposition implements basically the same algorithm originally created by Vicent Peris, and already implemented as a script two years ago by Oriol Lehmkuhl. However, this new tool implements a completely different image scaling algorithm (also published as the new LinearFit tool), much more accurate and robust, and works in a much more automatic way. In fact, you normally will only need to adjust one parameter, and most times you'll get an excellent result just with the default parameter values.

To give you an idea of how HDRComposition works, I've prepared a small example with a set of five Images of the M42 region by Vicent Peris and José Luis Lamadrid. You can see them on the screenshot below.


These are raw linear images acquired with a modified Canon 300D DSLR camera through three different instruments: A Takahashi FC-100 refractor (10 cm @ f/8), a Celestron 80ED (8 cm @ f/7.5), and a Takahashi Epsilon 180ED telescope (18 cm @ f/2.8). The exposures are of 10 seconds with the 80ED; 2, 10 and 30 minutes with the FC-100, and 10 minutes with the Epsilon 180ED, all of them from a very dark location in Teruel, Spain.

Our goal is to integrate all of these exposures into a high dynamic range image. Here is the result after running HDRComposition with default parameters:


Note that the set of source images can be specified in any arbitrary order; HDRComposition automatically computes the relative exposure of each image and sorts the list accordingly. This routine is very robust, and works reliably without requiring any metadata information about exposure times, ISO speed, CCD parameters, etc. (HDRComposition ignores image metadata). In the rare case you need to specify the exposure order manually (which should not happen in any situation we can figure out), you can disable the Automatic exposure evaluation option.

The resulting HDR image is extremely dark, as expected. It is a linear image with a huge dynamic range (the whole range of the M42/M43 region) stored in a numeric range of at least 230 discrete sample values. Note that this HDR image does not fit into a 32-bit floating point image, where you can store a maximum of about 224 discrete values. This HDR image requires either the 32-bit unsigned integer format or the 64-bit floating point format, both available in PixInsight. The HDRComposition tool generates 64-bit floating point images by default, although 32-bit floating point can also be used as an option suitable for moderately large HDR compositions.

HDRComposition also provides, as an option, the set of composition masks used internally. Each composition mask is white where the pixels on a long exposure have been replaced with the corresponding pixels from the next exposure in decreasing exposure order. You can see the four masks generated in this example on the screenshot below.


Composition masks provide you full control over the process. Composition masks are initially generated as binary images (only 0 and 1 values). The Binarizing threshold parameter defines where to cut long exposures to replace their pixels with pixels from the next image in decreasing order of exposure. Normally the default value of 0.8 is quite appropriate, but you can change it if you see that the masks are not driving the composition correctly (which is rare). After binarization, the masks are smoothed to prevent artifacts generated by hard masking transitions. The degree of low-pass filtering applied to composition masks can be controlled with the Mask smoothness parameter, although the default value will usually give excellent results.

Here is the HDR image with an automatic STF applied. STFs are 16-bit look-up tables in PixInsight. Note that a 16-bit LUT causes posterization in the screen representation of this image. There is no surprise here, since as noted this is a 64-bit floating point image storing more than 230 discrete values.


Below is the image after BackgroundNeutralization. Note the tiny upper bound of the background sampling range: 0.0001.


To better evaluate the numerical depth of this linear HDR image, here is a zoomed area with automatic STF:


Note the strong posterization caused by the 16-bit rendition of the image (please click the image to see the full-size version). Now you can see and show a clear example of what happens when 16 bits aren't sufficient. Actually, 32-bit floating point doesn't suffice in this case, either. This is the same area after stretching the 64-bit image with HistogramTransformation:


And this is the whole stretched image:


After stretching, the standard procedure in PixInsight is applying HDRWaveletTransform to compress the dynamic range of the HDR image. This is the result:


And finally, the image greatly benefits from a generous increase in color saturation. Note the chromatic richness in the following screenshot.


Summarizing:

- The best results are obtained when HDRComposition is used with linear images.

- For astronomical purposes, all the images must be accurately calibrated. You normally will calibrate, register and integrate a set of individual frames for each exposure, then the results of these integrations will feed HDRComposition. However, you can also use single calibrated frames, as this example demonstrates.

- HDRComposition requires registered images. In other words, pixel-per-pixel correlation is required for all the images taking part in the composition; dissimilar images cannot be used.

- Additive gradients, if they are dissimilar across the set of integrated images, will hurt the accuracy of the HDR composition process. If possible, all additive gradients should be removed prior to HDR composition. This involves applying the DBE or ABE tool to the linear images.

- To speed up the process, you can define a sampling region of interest (ROI). This is a rectangular area used to restrict image scaling. This has the additional advantage that if there are slight flat fielding errors near the edges of the images (which is frequent), one can define a central ROI to improve accuracy of image scaling.

That's it. Happy HDR compositing!

« Last Edit: 2010 September 13 08:00:21 by Juan Conejero »
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Carlos Milovic

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2172
  • Join the dark side... we have cookies
    • http://www.astrophoto.cl
Re: New Tool: HDRComposition
« Reply #1 on: 2010 September 13 08:08:14 »
Looks great :) Good job with this one.
Regards,

Carlos Milovic F.
--------------------------------
PixInsight Project Developer
http://www.pixinsight.com

Offline Nigel Ball

  • PixInsight Addict
  • ***
  • Posts: 277
    • Astrophotography by Nigel
Re: New Tool: HDRComposition
« Reply #2 on: 2010 September 13 08:34:48 »
This product just gets better and better

VERY impressive   ;D

Nigel
Nigel Ball
Nantwich, Cheshire, United Kingdom

Takahashi FSQ-106 at f/8, f/5 and f/3.6 on AP900, Nikon 28 mm and 180mm f/2.8
SBIG STL-11000M, Astrodon LRGB, 5nm Ha
ST-10XME, Astrodon HaLRGB
www.nigelaball.com

Offline Enzo De Bernardini

  • PTeam Member
  • PixInsight Addict
  • ***
  • Posts: 274
  • Resistance is futile.
    • Astronomí­a Sur
Re: New Tool: HDRComposition
« Reply #3 on: 2010 September 13 08:40:07 »
Excellent!

And the button "From Preview" in both process...you read my mind  ;)

Thanks!

Enzo.

Offline sleshin

  • PixInsight Old Hand
  • ****
  • Posts: 431
Re: New Tool: HDRComposition
« Reply #4 on: 2010 September 13 10:48:48 »
Very cool Juan. Thanks or all your hard work.

Steve
Steve Leshin

Stargazer Observatory
Sedona, Arizona

Offline Harry page

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1458
    • http://www.harrysastroshed.com
Re: New Tool: HDRComposition
« Reply #5 on: 2010 September 13 11:16:08 »
oh Yes

 excellent tool all I need is something to work with  >:D


Harry
Harry Page

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: New Tool: HDRComposition
« Reply #6 on: 2010 September 13 11:18:23 »
Which target besides the Orion Nebula would be a candidate?
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Re: New Tool: HDRComposition
« Reply #7 on: 2010 September 13 11:25:05 »
Which target besides the Orion Nebula would be a candidate?

Andromeda, deep M51, moon, jupiter+moons, ...
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Jack Harvey

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 975
    • PegasusAstronomy.com & Starshadows.com
Re: New Tool: HDRComposition
« Reply #8 on: 2010 September 13 12:17:32 »
Iris Neb
Jack Harvey, PTeam Member
Team Leader, SSRO/PROMPT Imaging Team, CTIO

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: New Tool: HDRComposition
« Reply #9 on: 2010 September 13 12:56:30 »
There are lots of astronomical objects that pose high dynamic range problems. Along with the already mentioned objects, any relatively bright galaxy has a bright nucleus that can benefit from HDR processing. Also don't forget planetary nebulas, where a very bright object is frequently surrounded by dim structures; see this nice example with the Cat's Eye nebula (NGC 6543), with raw data from the Nordic Optical telescope processed by Vicent Peris with the old HDRComposition script.  
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline harist

  • Newcomer
  • Posts: 46
Re: New Tool: HDRComposition
« Reply #10 on: 2010 September 13 13:17:33 »
Excellent!

Tasos

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: New Tool: HDRComposition
« Reply #11 on: 2010 September 13 13:36:07 »
I think all this depends a bit on the optical system and camera used as well. Large scopes gather more light and will saturate more quickly. Shallow wells overflow more quickly.

I took subs of M31 with my C11/HS and even with 4m exposures I still see detail in the core with my QHY8. I suppose I should try some 1m exposures and see if that reveals extra detail in the core.

I used M31 to demonstrate HDRWT at MWAIC and I think the attendees will agree that detail was recovered from the bright areas without any problems :)
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline vicent_peris

  • PTeam Member
  • PixInsight Padawan
  • ****
  • Posts: 988
    • http://www.astrofoto.es/
Re: New Tool: HDRComposition
« Reply #12 on: 2010 September 14 01:10:04 »
Hi Sander,

As we learnt at the Adler, HDRC is not only to make HDR compositions of the main subject. It can be actually used as a true debloomer. :)

I showed that, instead of using a software "fake" debloomer, it's way better to make some short exposures of your FOV and recover the bloomed areas with *real* data. The output masks are also very useful for denoising this areas.


Regards,
V.

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • http://www.carpephoton.com
Re: New Tool: HDRComposition
« Reply #13 on: 2010 September 14 06:21:10 »
Hi Vicent,

I suppose with blooms you have no choice but to replace the bloom with short exposure background or truly making stuff up. In both cases though the pixels aren't comparable to their neighbors. If there are faint targets hidden by blooms they will not become visible with short exposures. The only real solution is to use ABG sensors :)
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline xatamec

  • Member
  • *
  • Posts: 52
Re: New Tool: HDRComposition
« Reply #14 on: 2010 September 14 08:25:44 »
The objects that pose more high dynamic range problems are the stars which appear in all our astrophotos  ;D
Specially shooting with a DSLR this is an issue. In the past I've used the HDRcomposition algorithm with long and short exposures to recover the saturated stars... but didn't get good results... after stretching the hdr image the stars got saturated again. One possibility to keep the stars under control is to do the initial stretch with MST... but I don't like it very much becauase the whole image gets flat and there is a considerable lost of constrast.

Any ideas about how to face successfully the problem of dynamic range of the stars?

Sergi

There are lots of astronomical objects that pose high dynamic range problems. Along with the already mentioned objects, any relatively bright galaxy has a bright nucleus that can benefit from HDR processing. Also don't forget planetary nebulas, where a very bright object is frequently surrounded by dim structures; see this nice example with the Cat's Eye nebula (NGC 6543), with raw data from the Nordic Optical telescope processed by Vicent Peris with the old HDRComposition script.