Author Topic: Rounding and precision problem in NumericControl  (Read 2963 times)

Offline NKV

  • PTeam Member
  • PixInsight Guru
  • ****
  • Posts: 677
Rounding and precision problem in NumericControl
« on: 2011 October 08 06:42:55 »
Hi.
Edit SandboxParameters.cpp
Code: [Select]
double SandboxParameterOne::MaximumValue() const
{
   return 0.9998;
}

Move slider maximum to the left and then press Enter.

Also note:
"0.9999" no problem.
"0.9998" problem is.

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Rounding and precision problem in NumericControl
« Reply #1 on: 2011 October 13 23:26:23 »
Hi Nikolay,

These problems are always caused by roundoff errors. A Slider control uses an integer variable to represent its current position internally. Depending on the length of the sliding region, there can be problems to adapt the slider's internal range to a particular range of real numbers. Note that this not only depends on the length in pixels that you are forcing programmatically, but also on other conditions that you have no control over, such as the current GUI style and other platform-dependent items. It may also happen that a new version of the PixInsight Core application changes the way these controls are generated and drawn on the screen, invalidating existing code that depends on some kind of tight control over specific control dimensions and ranges. Finally, bear in mind that the user can customize all GUI controls with a modified .qss (Qt Style Sheet) file.

Always try to avoid using 'accurate' maximum and minimum values for process parameters. In other words, try to limit yourself to 0.0 and 1.0 instead of 0.001 or 0.9998, for example. If you really need more than one or two decimal digits for the limits of your parameters, better handle them directly in event handlers, but don't expect a Slider control being able to manage things such as 0.9998 for you.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline NKV

  • PTeam Member
  • PixInsight Guru
  • ****
  • Posts: 677
Re: Rounding and precision problem in NumericControl
« Reply #2 on: 2011 October 14 02:13:27 »
Hi Juan,
I want dynamically change NumericControl.SetRange according pixelsdata. To avoid the rounding problem I want use code like that:
Code: [Select]
//For example, there are process parameter in range 0.0 to 1.0
precision = 10; // Precision() { return 10; }

NumericControl.slider.SetRange( 0, 0x7fffffff ); //for maximum precision control in tiny range (+/- StdDev)
NumericControl.SetPrecision( precision );

minimum = Statistics.Mean() - Statistics.StdDev();
maximum = Statistics.Mean() + Statistics.StdDev();

min = Round( minimum, precision );
max = Round( maximum, precision );

NumericControl.SetRange( min, max );
It's OK?