Hi Andrea,
Building star masks is a complex task, but be sure we have the most powerful tools and techniques here; it's just that some practice (and a lot of patience!) is required.
In PixInsight, we have basically two different strategies to build a star mask:
- Use the StarMask tool. Optionally, in difficult cases you can use the HDRWaveletTransform tool before StarMask. This combo is extremely powerful. For example, we have managed to build a star mask to isolate all stars inside the core of M42/M43:
http://pixinsight.com/forum/viewtopic.php?p=2830#2830- Use the ATrousWaveletTransform tool. In this case, the basic idea is to isolate the scales that define the stars that we want to protect. This approach can work very well in relatively simple cases. Consider this image:
http://forum-images.pixinsight.com/legacy/star-masks/atw/01.jpgHere is an example of star mask built with wavelets:
http://forum-images.pixinsight.com/legacy/star-masks/atw/02.jpgAs you see, I have disabled large-scale wavelet layers where we have no object of interest, including the residual layer. The remaining layers, where stars are supported, have been biased to give them more relevance. The k-sigma noise thresholding parameters allow us to remove most nonstellar objects, as nebular features. This mask can be easily transformed with HistogramTransform and MorphologicalTransform to achieve the required star protection.
If you can upload an image where you are having trouble to build a star mask, I can try to build one as a small tutorial. This can help you (and others) in "getting the right direction" with these tools.