Hi Jeff,
Our implementation of the Richardson-Lucy and Van Cittert deconvolution algorithms includes a wavelet-based regularization algorithm. Regularization splits the data into significant structures and noise at each deconvolution iteration. Basically, significant structures and preserved and deconvolved, while the noise is attenuated or suppressed. This greatly improves the behavior of these deconvolution methods when applied to real world images.
The regularization parameters allow you to control which image structures will be considered as noise during the deconvolution process, and how they will be treated. The parameters define a wavelet decomposition. There are wavelet layers selectable up to five levels. Each wavelet layer isolates image structures within a typical range of sizes in pixels, ranging from one pixel (layer 1) to 16 pixels (layer 5). Normally most of the noise is supported by the first two or three layers (noisy structures from 1 to 2 or 4 pixels) in most images.
For each layer, you have noise threshold and noise reduction parameters. Noise threshold is expressed in sigma units. The larger the threshold value, the more structures will be filtered as noise. Typically, thresholds should decrease as layers increase, since uncertainty (dispersion) is smaller at larger scales. The default value of dispersion for the first layer is 3 sigma, which will catch more than a 99% of the pixels pertaining to small-scale structures (assuming a Gaussian distribution of wavelet coefficients). This is because the first wavelet layer does not support significant structures, other than high-contrast borders, in most deep-sky images. The noise reduction parameter is a simple attenuation factor: reduction=0 means no noise reduction at all; reduction=1 means complete suppression of noise structures.
Your goal with deconvolution regularization parameters is basically to prevent intensification of the noise, especially on low-SNR areas such as the sky background. Start with default parameters. If you see that the noise is getting boosted on the background, define stronger regularization parameters (more wavelet layers / higher thresholds / higher noise reductions) and try again. As always, this requires practice, especially to judge what the images "asks" you. If properly configured, regularization allows you to apply deconvolution without needing any mask in most cases.
As for the Gaussian/Poisson noise models for regularization, the difference is very small in practice... you can simply use a Gaussian model in most cases. If I find the time, I'll probably implement a mixed Gaussian/Poisson noise model and will remove this parameter. The whole Deconvolution tool needs a good revamp anyway.