Hi Harry,
The HDRWaveletTransform (HDRWT) tool generates its own luminance mask, just as ACDNR does. However, in general a simple luminance mask is all you need with HDRWT, so you don't need to tweak it as tightly as an ACDNR mask; that's why there are no mask generation controls in HDRWT.
If you need more control, you must build and use your own mask. It's very easy:
1. Select your image.
2. If it's a grayscale image, simply duplicate it (Image > Duplicate). If it's a color image, extract its luminance (Image > Extract > Luminance).
3. Now select the luminance image you've extracted in step 2. To simplify things, you can rename it. For example, select Image > Identifier and type "Mask" (for example; you can use the identifier you prefer) in the Set Image Identifier dialog.
4. Open the HistogramTransform interface (on the Process Explorer, double click the corresponding item in the Favorites category). Select your Mask image in the view list of HistogramTransform (the combo box).
5. Activate the Real Time Preview option of HistogramTransform. It's the small blue button with an elliptic shape (an "eye" shape, actually) on the bottom bar of the interface. Now you can adjust your mask by changing histogram parameters. You can see the changes in real time on the Real Time Preview window.
6. When you're happy with the mask, close the Real Time Preview window and apply HistogramTransform to your Mask image (e.g., press F5 when the HistogramTransform window is active, or drag the small triangle (blue, at the lower left corner) to the Mask image).
7. Now select the image you want to process (you extracted its luminance at step 2). Select Mask > Select Mask and choose the Mask image.
8. The mask is visible by default. Select Mask > Show Mask to hide the mask.
9. Now you can apply any (maskable) process to the image, including HDRWT, and it will be masked automatically.
Note that there is no problem in combining the Luminance Mask option of HDRWT with a custom mask: both masks will be applied in sequence (the automatically generated mask first). Doing this is not a usual practice, but it's perfectly legal.
Hope this helps. Keep questions coming!