Hi Larry,
You can use either a grayscale image or an RGB color image to mask an RGB color image in PixInsight. It depends on what you want to do.
When a grayscale image acts as a mask for a color image, it is applied equally to each color channel to filter maskable processes. When the mask is an RGB color image, each mask channel is used separately for its corresponding channel of the masked image, that is, it works like three separate images, each one with its own mask. Note that a color image cannot be a mask of a grayscale image.
When masking color images, most of the time you want the same mask applied to all color channels, so in most cases you'll use a grayscale mask image. Masks can be generated in many ways and for many purposes. Extracting the lightness component of a color image is just one, and typical, way to obtain a general-purpose mask.