BatchStatistics

By Ian Lauwerys (www.blackwaterskies.co.uk)

A script to measure statistics on batch of images and write them to a file. [more]

Keywords: batch statistics, batch, statistics, noise evaluation, noise, spreadsheet, excel, csv, text, delimited, export

Contents

[hide]

1 Description

[hide]

The BatchStatistics script enables measurement of statistics on a batch of images. Results are output to the console and/or a text file in a variety of delimited formats such as CSV, suitable for import to spreadsheets and other external data analysis tools. All statistics and measures in the Statistics process and NoiseEvaluation script are available for output.

Batchstatistics supports multi-image file formats and any number of channels per image.

1.1 Images to Analyse

  • Click the Add button to choose image files to analyse. Any image file format supported by PixInsight should be suitable. The script will support images with multiple channels (RGB, etc.) and it should also support multi-image file formats such as FITS. (Please note that his latter feature is untested by me so please report any problems via this thread). Note that the dialog can be resized vertically to show more list entries and horizontally to show long file names by dragging its edges as required.
    Important: When working with multi-channel images, it is best to try to work with sets of images that have the same number of channels and in the same order. BatchStatistics will process images with any number of channels, but will warn you via the PixInsight console if the current image contains a different number of channels than the previous one. Given that the objective is to output to a file format containing columns of data, changing the number of channels between images will mean that the column headers do not always correspond to the data rows. (Note that if you append results to an existing output file, BatchStatistics has no way to know how many columns of data it contains, so the operator is responsible for doing the right thing!)
    BatchStatistics does not currently support working with open image views, previews, etc.
  • The Clear, Invert Selection and Remove Selected buttons function in the same manner as other PixInsight scripts and processes should you need to amend the list.

1.2 Statistics Options

  • The Number Format combo box functions in the same manner as the existing Statistics process. You may output results using PixInsight's native Normalized Real format, or converted to Integer bit depth to match your camera's output format (typically 16 bit for CCD cameras or 12 / 14 bit for DSLR cameras).
  • The Precision spin box allows you to choose the number of digits in Normalized Real and Scientific formats between 0 and 17 places.
  • The Scientific Notation check box functions in the same manner as the existing Statistics process. It allows you to output Normalized Real format numbers in scientific notation.
  • The Normalize check box functions in the same manner as the existing Statistics process. It makes all scale estimates consistent with the standard deviation of a normal distribution.
  • The Unclipped check box functions in the same manner as the existing Statistics process. By default, statistics are computed by clipping (ignoring) pixels with values of 0 ("black") and 1 (fully saturated) when this box is unchecked. If you change the Clipping Low and Clipping High controls, then pixels with values outside the chosen clipping range will be ignored instead. By checking the Unclipped check box, statistics are computed for all pixels in the image ignoring any clipping range.
  • Clipping Low and Clipping High: If the Unclipped check box is unchecked, any pixels with values less than Clipping Low or greater than Clipping High will be excluded from statistics calculations.

1.3 Metadata and Statistics to Output

  • Select the appropriate check boxes to choose which statistics to output. Except as noted below, all options should function in the same manner as the existing Statistics process.
    Important: Selecting all statistics may lead to long pauses between images. The script may appear to have hung but be patient as some options (such as Qn) are computationally expensive and may take some time to complete. The status bar and console messages will only update once all channels in an image have been fully processed.
    The following options are output once per image. The corresponding column headers are also shown:
    • File Path: File_Path The path of the image being analysed.
    • File Name: File_Name The file name of the image being analysed.
    • Full File Name: File_Full_Name The combined path and file name of the image being analysed.
    • Image Number: Image_Number The image number within the file (useful for multi-image file formats only).
    • Clipping Low: Clipping_Low The low range clipping value for statistics calculations.
      Note: If Unclipped is selected, this column will be null.
    • Clipping High: Clipping_High The high range clipping value for statistics calculations.
      Note: If Unclipped is selected, this column will be null.
    • Image Width: Width The width of the image in pixels.
    • Image Height: Height The height of the image in pixels.
    • Number of Channels:
      • Channels_First_Analysed First channel number analysed.
      • Channels_Last_Analysed Last channel number analysed.
      • Channels_Analysed Number of channels analysed.
      Note: that monochrome images usually have one channel (0), and colour images have three, Red (0), Green (1) and Blue (2), but this may vary depending on the image format.
    The following options are output once per channel in the image. The corresponding column headers are also shown. Each column header will have the channel number to which it relates appended, e.g. Mean_0, Mean_1, Mean_2, etc.
    • Count Percent: Count_Pct_n Pixels used for statistics calculations (i.e. not clipped) as a percentage of total pixels in image.
    • Count Pixels: Count_Px_n Number of pixels used for statistics calculations (i.e. not clipped).
    • Mean: Mean_n The arithmetic mean, i.e. the average of sample values.
    • Median: Median_n The median of sample values.
    • Modulus: Modulus_n The sum of absolute sample values.
    • Norm: Norm_n The sum of sample values.
    • Sum of Squares: Sum_Of_Squares_n The sum of the squares of sample values.
    • Mean of Squares: Mean_Of_Squares_n The mean of the squares of sample values.
    • Variance: Variance_n The variance from the mean of sample values.
    • Standard Deviation: StdDev_n The standard deviation from the mean of sample values.
    • Average Absolute Deviation: AvgDev_n The average absolute deviation from the median of sample values.
    • Median Absolute Deviation (MAD): MAD_n The median absolute deviation from the median (MAD) of sample values.
    • Biweight Midvariance (BWMV): SQRT_BWMV_n The square root of the biweight midvariance of sample values.
    • Percentage Bend Midvariance (PBMV): SQRT_PBMV_n The square root of the percentage bend midvariance of sample values.
    • Sn: Sn_n The Sn scale estimator of Rousseeuw and Croux of sample values.
    • Qn: Qn_n The Qn scale estimator of Rousseeuw and Croux of sample values.
    • Minimum: Min_n The minimum sample value.
    • Maximum: Max_n The maximum sample value.
    • Minimum Position:
      • Min_Pos_X_n The X coordinate of the first occurrence of the minimum sample value.
      • Min_Pos_Y_n The Y Coordinate of the first occurrence of the minimum sample value.
    • Maximum Position:
      • Max_Pos_X_n The X coordinate of the first occurrence of the maximum sample value.
      • Max_Pos_Y_n The Y Coordinate of the first occurrence of the maximum sample value.
    The following additional measures are available over and above those in the existing Statistics process:
    • Noise Evaluation:
      • Sigma_MRS_n The noise standard deviation calculated using the Multi-resolution Support method.
      • Count_MRS_n The count of noise pixels calculated using the Multi-resolution Support method.
      • Layers_MRS_n The number of layers used by the Multi-resolution Support method.
        Note: If the MRS method does not converge on a solution, these three columns will be null.
      • Sigma_K_Sigma_n The noise standard deviation calculated using the K-Sigma method.
      • Count_K_Sigma_n The count of noise pixels calculated using the K-Sigma method.
        Note: Noise Evaluation will produce the same results as the existing NoiseEvaluation script, except that the K-Sigma values will always be calculated regardless of whether the MRS method converges on a solution.
    • The Select All button checks all available statistics and metadata checkboxes.
      Caution: Computing all statistics for a large number of images will take a significant amount of time, so only select those that you actually require.
    • The Select None button unchecks all available statistics and metadata checkboxes.

1.4 Output Options

  • The File Format combo box allows you to select various delimited output formats from Tab, Pipe, Colon, Space, Comma or CSV.
    Note: Any data value which contains the delimiter value will be enclosed in double quotes (" "). In the case of the CSV format, the delimiter is the comma character and all data values are enclosed in double quotes. Line endings are the MS-DOS CR LF format as per RFC 4180.
  • The To Console checkbox outputs header and result rows to the PixInsight Console. Due to informational messages (e.g. loading of images) this is not as useful as it could be as it is not possible to cut and paste a block of results directly from the console to your application, but it may be of use for single images (vs. cutting and pasting individual data values from the Statistics process).
    Note: MS Excel's Text to Columns option is your friend when cutting and pasting from the console.
  • The To File checkbox outputs header and result rows to a text file.
  • The Overwrite checkbox overwrites any existing text file of the same name as that specified in Output File.
    Important: No warning will be given when overwriting (this is by design for future developments in re-using script instances) so please be careful!
  • The Append checkbox appends results to any existing text file of the same name as that specified in Output File or creates a new file if one does not exist.
  • The Include Header checkbox outputs a row of column headers appropriate to the first image in the file list. Please see the note above about working with images containing varying numbers of channels.
    Note: Headers are not written to existing files when appending, only to new ones if created (this is by design to ensure that each output file only contains a single header row at the top).
  • The Output File... button allows you to select a folder and select/specify a file name for output of results. The chosen file name is displayed in the Output File text box.

1.5 Control Buttons

  • The Analyse button analyses all images in the file list and produces the selected statistics and metadata.
  • The Exit button exits the BatchStatistics script.
    Note: No warning is given upon exit so please take care to create a process icon if required.
  • The blue Process triangle can be dragged to the PixInsight workspace to create a process icon. All file entries in the list and the options are saved as part of the process icon. The process may be re-instantiated by right clicking the script icon, choosing Launch Script Instance and then clicking the round Apply Global icon at the bottom of the Script dialog.
    Note: Dragging the "Process" triangle on to an image window has no effect at present. It is planned to allow processing of image statistics from open images using this method in a future version.
  • The Help button displays this help documentation, or hover over it for a brief summary in the tooltip.

Related Tools

Related Scripts