Author Topic: PixInsight 1.8.0 Ripley: Redesigned Statistics Tool  (Read 5779 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 6782
    • View Profile
    • http://pixinsight.com/
PixInsight 1.8.0 Ripley: Redesigned Statistics Tool
« on: 2013 November 21 16:44:15 »
A small number of tools have seen few improvements since their initial implementations around 2005, when I made the transition from PixInsight LE to settle the foundations of the current PixInsight platform. So it's time for a change.

One of these tools is Statistics. In the latest version 1.8.0 of PixInsight, I have redesigned this essential tool almost completely. The new interface is shown on the following screenshot.


Along with necessary aesthetic changes (the previous HTML-based presentation was looking quite aged), the tool has been revamped in two main directions:

* All statistics are now computed with implicit rejection of zero (black) and one (saturated, white) pixels, which are simply ignored. In previous versions, when an image had a moderate amount of black and/or white pixels, the computed statistical values were mostly useless. This has been a serious limitation.

* A comprehensive set of statistics is now available. By clicking the wrench button, the Statistics Options dialog opens.


The following items can be selected:

- Count. This is the number of pixel samples used to compute statistics for each channel. The value is shown as a pixel count and as the percentage of the total pixels in the image.

- Arithmetic mean.

- Modulus, or the sum of absolute sample values.

- Norm, the sum of sample values. For normal images this is equivalent to the modulus since pixel samples can only be positive. However, for out-of-range images the norm can be different as a result of negative values.

- The sum of square sample values.

- The mean of square sample values.

- The median.

- The variance from the mean.

- Standard deviation from the mean.

- Average absolute deviation from the median.

- Median absolute deviation from the median (MAD).

- The square root of the biweight midvariance.

- The square root of the percentage bend midvariance.

- Sn and Qn scale estimators of Rousseeuw and Croux.

- Extreme sample values.

- Positions (in image coordinates) of the first occurrences of the extreme sample values.

For more information and further references on these estimators, see the latest reference documentation for the ImageIntegration tool (jump to the Image Normalization section, Location and Scale Estimators subsection).

On the Statistics Options dialog, click the Reset button to select a default set of statistics including count, mean, median, average deviation, MAD, minimum and maximum. This is a reduced but normally sufficient set.

Click the Save button to set the currently selected set as the default, which will be remembered across sessions. If you don't save the current set explicitly, it will only take effect during the current session.

Bear in mind that the more statistics you select, the more computation time will be necessary to produce them. This can be problematic with some particularly expensive estimators, such as Sn and Qn, especially if your images are large and your machine isn't on the fast side. For performance reasons, the statistics calculation process cannot be interrupted.

Returning to Statistics' main interface, you can select a numeric range for statistics representation, as well as the scientific or fixed point format. These options haven't changed with respect to previous versions. A new option allows you to obtain normalized or unnormalized scale estimates. Normalized scale estimates are made coherent with the standard deviation of a normal distribution by multiplication with the following constants:

Average deviation1.2533
MAD1.4826
Biweight midvariance0.9901
Percentage bend midvariance  0.9709
Sn1.1926
Qn2.2191

Finally, you can click the Text View button to open a window with all the statistical data represented as plain text. You can copy this text to the clipboard for later use. You can also edit it quickly on the same text view dialog.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • View Profile
    • http://www.carpephoton.com
Re: PixInsight 1.8.0 Ripley: Redesigned Statistics Tool
« Reply #1 on: 2013 November 21 18:38:29 »
Nice! Is the clipboard data easily grokked by Excel?
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 6782
    • View Profile
    • http://pixinsight.com/
Re: PixInsight 1.8.0 Ripley: Redesigned Statistics Tool
« Reply #2 on: 2013 November 22 03:47:48 »
Thanks Sander. The generated text isn't separated with tabulation characters or commas, so I don't know. I'm thinking on adding a CSV export routine, which would ease exporting to other applications.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline Nocturnal

  • PixInsight Jedi Council Member
  • *******
  • Posts: 2727
    • View Profile
    • http://www.carpephoton.com
Re: PixInsight 1.8.0 Ripley: Redesigned Statistics Tool
« Reply #3 on: 2013 November 22 07:42:34 »
If the columns are fixed width then the text import wizard should be able to deal with it. Of course one step copy/paste would be preferable. The CSV output would be useful for other purposes as well so it would be a nice option to add at some point.
Best,

    Sander
---
Edge HD 1100
QHY-8 for imaging, IMG0H mono for guiding, video cameras for occulations
ASI224, QHY5L-IIc
HyperStar3
WO-M110ED+FR-III/TRF-2008
Takahashi EM-400
PIxInsight, DeepSkyStacker, PHD, Nebulosity