Author Topic: rounding of floating point to integers  (Read 2133 times)

Offline chrisvdberge

  • PixInsight Addict
  • ***
  • Posts: 104
rounding of floating point to integers
« on: 2015 August 22 07:42:32 »
I'm wondering how PI is converting floating point values to integers. If I understood correctly, when we do an Integration with Combination method 'Average' we are working with floating point. But at the end of the stacking process we need to convert to integers for the RGB values right?
So theoretically we loose signal here.

Is that statement correct? And if so, how is the conversion done, ie; might we also be gaining signal since 2.5 will be rounded up as 3?
 

Offline chrisvdberge

  • PixInsight Addict
  • ***
  • Posts: 104
Re: rounding of floating point to integers
« Reply #1 on: 2015 August 22 15:13:22 »
just to clarify my question;
I'm looking into the differences between stacking a subset versus stacking all subs at once.
To me it seems you lose signal and the noise wouldn't be as 'nicely' distributed as with more frames, but just trying to confirm some ideas on that here..

Offline NGC7789

  • PixInsight Old Hand
  • ****
  • Posts: 391
Re: rounding of floating point to integers
« Reply #2 on: 2015 August 22 19:14:25 »
But at the end of the stacking process we need to convert to integers for the RGB values right?

I don't think this is correct. I think you stay 32 bit until you save or explicitly covert to another bit depth.