Author Topic: Sumofsquares formula from statistics  (Read 533 times)

Offline Roberto Jimenez

  • Newcomer
  • Posts: 3
Sumofsquares formula from statistics
« on: 2019 November 10 14:43:17 »
Hi, I'm from Spain.
I'm calculating the statistics from an image (dark) and I don't understand how can I calculate de variance from de sumofsquares.

_120sec_1x1_amb_14_7C_frame1
               K
count (%)      99.92818
count (px)     1444211
mean           266.0
modulus        384099706.0
norm           384099706.0
sumOfSquares   1743754.5
meanOfSquares  1.2
median         265.0
variance       8393.9
stdDev         91.6
avgDev         10.9

I thought that de sumofsquares divided by the samples px was de variance. But isn't it in this case.
What's the formula of de sumofsquares that PI uses?
Thank you very much

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Sumofsquares formula from statistics
« Reply #1 on: 2019 November 13 12:10:02 »
Hi Roberto,

This is a bug in the Statistics process. Thank you for discovering it.

The sum of squares and mean of squares statistics must be scaled with the square of the integer range (e.g., 65535 for the 16-bit range) when the selected range is not the normalized [0,1] range. Not it is being scaled with the integer range.

I have just fixed this bug. The update will be released tomorrow for all platforms. Again thank you for reporting this, and sorry for the trouble.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Roberto Jimenez

  • Newcomer
  • Posts: 3
Re: Sumofsquares formula from statistics
« Reply #2 on: 2019 November 13 15:10:55 »
I was analizing flats and darks frames data when I saw it.
I found an old similar BUG in the forum (not scaled data Statistics process bug) but I thought that in this case I'll be wrong in my calculations.
Thank you very much Juan for your help.

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
Re: Sumofsquares formula from statistics
« Reply #3 on: 2019 November 14 04:38:35 »
Hi Roberto,

This bug is now fixed with an update for all platforms. The update is available after upgrading to the latest 1.8.8-1 core version. Thank you for your support.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Roberto Jimenez

  • Newcomer
  • Posts: 3
Re: Sumofsquares formula from statistics
« Reply #4 on: 2019 November 16 15:41:42 »
Hi Juan,

I tested the new core version yesterday. Good job. It's ok now.
It's important to say that the sumofsquares formula from pixi statistics isn't the sumofsquares of some websites, for example on a lot of websites said that the formula for sum of squares is

SUM( xi - mean )2

Where the mean refers to the sample mean, and the symbol x tells us to add up the squared differences (xi - mean) for all i.
Also said there is an equivalent, shortcut formula that does not require us to first calculate the sample mean. This shortcut formula for the sum of squares is

sum (xi2)-(sum xi)2/n

In pixinsight sumofsquares is ONLY the values for this formula: sum(xi2).
It's logical because is the value that we really need to calculate. But it's important to say to prevent confusions.
I have done some homework. I put on an excel shit the values from the histogram (a lot of work !!).
I missed a pixel in my calculation but it's almost equal than pixi statistics values.

Thank you very much Juan. I'm gladly surprised for your fantastic support. 
« Last Edit: 2019 November 18 07:10:35 by Roberto Jimenez »