Personally, I use 32 bit as well. Totally agree with Sander. Everything is 32 bit FITS right up until final output when I convert to a 16 bit TIFF ProPhoto RGB master finished image (old habit from PhotoShop). Then for web display it's converted to sRGB JPG.
Keep the higher quality files up front, then once you've done all the heavy stretching and number-crunching, right at the end of the process you can start reducing the bits. Less chance of losing quality.
Keep in mind that just because you can't see a difference, doesn't mean it isn't there. When I first started astro, I wasn't even using PhotoShop, I was using GIMP. 8 bits. Stretching the histogram in that really highlighted the necessity of at least 16 bits that PS offered.
I know you're talking about master calibration files, not the actual images here. I think the philosophy still stands. As Sander mentioned, it's all about the numbers in between. Start with the highest quality that is reasonable up front, for all files, then right at the end you can start reducing it down when it's less likely to impact quality.