Javascript: kappa-sigma clipping

[David Serrano]

You're making a good job. Keep it running.

Same to you, of course .

Why are you saving/disabling/restoring the onNodeUpdated event handler function? For the reason we've seen in the previous point, changing the checked property of TreeBoxNode programmatically won't trigger the TreeBox.onNodeUpdated event, so there's no need to care.

Or are you experiencing spurious events triggered in this case? If so, then you've found a bug that must be fixed immediately. Please let me know.

I did it for performance reasons. It's useless to trigger that event while we're in the loop.

And yes, the event was actually triggered, with recursion too, but the engine was smart enough to notice it before entering an infinite loop, something like the engine saying "Oops, I've been here already, and following this same object... time to backtrack!" (the "following the same object" part is important). I think I'll make a little example with debug statements and post it as a bug report to explain exactly what I mean.

Code: [Select]

     //engine.kappa = value.toInt;  // maybe better!!

What's this?

An idea I had. After some googling I discovered that it doesn't exist in javascript, although there are some third party libs that provide it. Performing type casting by doing "0 + string" looks just horrible to me. The fact that the + operator in javascript concatenates strings is one of its poorest features.

On a side note, I still think that a version of the script to integrate disk image files would be great, not excluding (but complementing) what you're doing here.

Probably but, since no example script (that I'm aware of) works like this, it's difficult to copy&paste :^P.

Thank you for your input!

[Juan Conejero]

Code: [Select]

  this.kappa_NC.onValueUpdated = function (value) {
      engine.kappa = 0 + value;   // javascript nuisances
      //engine.kappa = value.toInt;  // maybe better!!
   }

I think there is a bit of confusion here. The onValueUpdated event handler will always receive a numeric value as its unique argument, so value can be directly assigned to engine.kappa in the code above without problems. Or am I missing something here?

Code: [Select]

  this.ok_Button.onClick = function() {
      // couldn't achieve this using this.dialog.target_List.selectedNodes, its length is always 1
      for (n = 0; n < this.dialog.target_List.numberOfChildren; n++)
         if (this.dialog.target_List.child (n).checked)
            engine.imgIDs.push (this.dialog.target_List.child (n).text (0));
      this.dialog.ok();
   };

Ok, there's absolutely no bug here, but a misunderstanding instead. TreeBox.selectedNodes will return an array of TreeBoxNode objects with the currently selected nodes (in the version of PI that you have, multiple tree node selections are not allowed, so the returned array cannot have more than one element; this has been fixed and will be available in the next release). However, this is not what you want here. What you want is the set of nodes that are checked, and the only way to collect them is with a loop as you've done. Note that checked != selected

Isn't it wonderful when I can demonstrate that you have not discovered one more bug? (hehehe  :mrgreen: )

[David Serrano]

Or am I missing something here?

I thought you was, but now the code runs as expected without the "0 +" :?:. One thing is sure: I needed that to make it run, it just may be that, since then, I moved the line from one place to another, and now value is always a number as you say.

Note that checked != selected

Good catch there .

Isn't it wonderful when I can demonstrate that you have not discovered one more bug? (hehehe  :mrgreen: )

In any case, multiple selections were not possible until now, which could be considered as a bug :^P.

[David Serrano]

v0.5. Varios cambios internos y dos cambios visibles: una checkbox que estaba un poco loca, y la granularidad del slider. Disfruten!

v0.5. Several internal changes and two visible ones: a checkbox that was a bit weird, and the slider granularity. Enjoy!

Code: [Select]

/*
    KSIntegration.js v0.5 - Image integration with kappa-sigma clipping.
    Copyright (C) 2007  David Serrano, Juan Conejero

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, version 3 of the License.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.
*/

// TODO: write a tutorial

#include <pjsr/Sizer.jsh>
#include <pjsr/FrameStyle.jsh>
#include <pjsr/NumericControl.jsh>
#include <pjsr/TextAlign.jsh>
#include <pjsr/StdButton.jsh>
#include <pjsr/StdIcon.jsh>
#include <pjsr/ColorSpace.jsh>
#include <pjsr/SampleType.jsh>
#include <pjsr/UndoFlag.jsh>

#feature-id KSIntegration
#feature-info Implementation of image combination using kappa-sigma \
  method for rejecting hotpixels. Needs that the images to be combined \
  are previously loaded in the application. They must have the same size \
  and bit depth. Generates a new image with the result of the combination.
// #feature-icon KSIntegration.xpm

#define VERSION "0.5"

function kappa_sigma_engine() {
   this.kappa = 2.0;
   this.imgIDs = new Array;
   this.views = new Array;

   // Populates the views array.
   this.load = function() {
      var N = this.imgIDs.length;
      this.views.length = 0;
      for (var i = 0; i < N; i++) {
         this.views.push (new View (this.imgIDs[i]));
      }
   };

   // Performs an iterative kappa-sigma clipping of a pixel set.
   this.clip = function (set) {
      var stddev;
      var mean;
      var delta;
      var low;
      var high;
      var result;
      // Fill a temporary array with the values that are inside the range
      // specified by kappa. If it ends up the same size as the original
      // one (or if it gets too small), exit. If not, them some values
      // were deleted, so reset the original to not include the deleted
      // ones, and start over again.
      while (1) {
         stddev = Math.stddev (set);
         mean = Math.avg (set);
         delta = this.kappa * stddev;
         low  = mean - delta;
         high = mean + delta;
         result = new Array;  // temp array
         for (var i = 0, n = set.length; i < n; ++i) {
            var v = set[i];
            if (v >= low && v <= high)
               result.push (v);
         }
         // Need at least 3 items to calculate a meaningful standard deviation
         if (result.length < 3)
            return set;
         // if lengths of both sets are equal, all elements are ok: all done
         if (result.length == set.length)
            return result;
         set = result;   // reset the work set with the elements that survived
      }
   }

   // Performs a kappa-sigma integration of the selected views.
   this.combine = function() {
      var N = this.views.length;
      if (N < 3)
         throw Error ("At least three images are required for kappa-sigma integration.");
      var imgs = new Array (N);
      var w = 0;
      var h = 0;
      var n = 0;
      var bps = 0;
      var real = false;
      // initialization of vars and error checking
      for (var i = 0; i < N; i++) {
         if (this.views[i].isNull)  // redundant but harmless
            throw Error ("Nonexistent view: " + this.views[i].fullId);
         imgs[i] = this.views[i].image;
         // first pass: initialize vars; next passes: check properties consistency
         if (i == 0) {
            w = imgs[i].width;
            h = imgs[i].height;
            n = imgs[i].numberOfChannels;
         } else {
            if (imgs[i].width != w || imgs[i].height != h || imgs[i].numberOfChannels != n)
               throw Error ("Incongruent image dimension(s): " + this.views[i].fullId);
         }
         bps = Math.max (bps, imgs[i].bitsPerSample);
         real = real || (imgs[i].sampleType != SampleType_Integer);
      }

      var dstimg = new Image (w, h, n,
         (n != 1) ? ColorSpace_RGB : ColorSpace_Gray,
         bps,
         real ? SampleType_Real : SampleType_Integer);
      dstimg.statusEnabled = true;
      dstimg.initializeStatus (format ("Integrating %u images", N), w*h);

      // now, after creating a new image, traverse the ones to be combined and
      // calculate the value for every pixel in every channel
      var clipped;
      var nclipped = 0;
      var pixels = new Array (N);
      for (var iy = 0; iy < h; iy++) {
         for (var ix = 0; ix < w; ix++) {
            for (var ic = 0; ic < n; ic++) {
               for (var i = 0; i < N; i++)
                  pixels[i] = imgs[i].sample (ix, iy, ic);
               clipped = this.clip (pixels);
               dstimg.setSample (Math.avg (clipped), ix, iy, ic);
               if (clipped.length < N)
                  nclipped += N - clipped.length;
            }
         }
         gc();
         dstimg.advanceStatus (w);
      }

      console.writeln (format ("Clipped %u pixels from a total of %u (%.2f%%).",
         nclipped, h*w*n*N, 100.0*nclipped / (h*w*n*N)));
      var window = new ImageWindow (1, 1, n, bps, real,
         dstimg.colorSpace != ColorSpace_Gray,
         format ("kappa_sigma_%.0f", 100.0 * this.kappa));
      var view = window.mainView;
      view.beginProcess (UndoFlag_NoSwapFile); // do not generate any swap file
      view.image.transfer (dstimg);
      view.endProcess();
      window.show();
      window.zoomToFit (false); // don't zoom in
   };
}

var engine = new kappa_sigma_engine();

function kappa_sigma_dialog() {
   this.__base__ = Dialog;
   this.__base__();

   // ----- HELP LABEL

   this.helpLabel = new Label (this);
   this.helpLabel.frameStyle = FrameStyle_Box;
   this.helpLabel.margin = 4;
   this.helpLabel.wordWrapping = true;
   this.helpLabel.useRichText = true;
   this.helpLabel.text = "<b>KSIntegration v"+VERSION+"</b> - A script to integrate " +
      "a set of images by the iterative kappa-sigma rejection method.<br>" +
      "Select the views to be combined and choose the kappa value desired. " +
      "Then click 'Ok' and wait for a new image to be created.";

   // ----- LIST OF TARGETS

   // if all images are checked, check the "All images" checkbox; else, uncheck it
   // ditto previews
   this.updateCBs = function() {
      // start with both marked to be checked and enter a loop
      // if an unckecked image/preview is found, mark the corresponding
      // checkbox to be unchecked.
      var imagesCBstatus = true;
      var previewsCBstatus = true;
      var nimages = 0;
      var npreviews = 0;
      for (i = 0; i < this.dialog.target_List.numberOfChildren; i++) {
         var n = this.dialog.target_List.child (i);
         if (-1 == n.text (0).indexOf ("->")) {   // it's an image
            nimages++;
            if (false == n.checked)
               imagesCBstatus = false;
         } else {                                 // it's a preview
            npreviews++;
            if (false == n.checked)
               previewsCBstatus = false;
         }
         // if both are already marked to be unchecked, there's no need to
         // keep walking the list: exit prematurely
         if (false == imagesCBstatus && false == previewsCBstatus)
            break;
      }
      // only update the CBs if some images/previews were seen.
      // In the case of images this is redundant (this function won't ever be
      // entered if there are no images)
      if (nimages)
         this.dialog.selectImages_CB.checked = imagesCBstatus;
      if (npreviews)
         this.dialog.selectPreviews_CB.checked = previewsCBstatus;
   }

   this.toggleSimilar_CB = new CheckBox (this);
   this.toggleSimilar_CB.text = "Toggle similar elements";
   this.toggleSimilar_CB.checked = true;
   this.toggleSimilar_CB.toolTip = "Tick to automatically (un)check " +
      "all images or previews that have<br>the same dimensions and bit depth " +
      "as the one that has been just (un)checked.";

   this.target_List = new TreeBox (this);
   this.target_List.setMinSize (400, 200);
   this.target_List.font = new Font ("monospace", 10); // best to show tabulated data
   this.target_List.numberOfColumns = 2;
   this.target_List.headerVisible = true;
   this.target_List.headerSorting = true;
   this.target_List.setHeaderText (0, "Views");
   this.target_List.setHeaderText (1, "Dimensions");
   this.target_List.setHeaderAlignment (0, Align_Left);
   this.target_List.setHeaderAlignment (1, Align_Left);
   // save this function in a visible place
   this.target_List.onNodeUpdated = function (node, column) {
      // if "Toggle similar" is on, walk the list and set the similar
      // images/previews to the state of the just-clicked one
      if (this.dialog.toggleSimilar_CB.checked)
         for (i = 0; i < this.dialog.target_List.numberOfChildren; i++)
            if (this.dialog.target_List.child (i).text (1) == node.text (1))
               this.dialog.target_List.child (i).checked = node.checked;
      this.dialog.updateCBs();
   }

   // Node creation helper
   function addViewNode (parent, view) {
      var node = new TreeBoxNode (parent);
      node.checkable = true;
      node.checked = false;

      node.setText (0, view.fullId);

      var image = view.image;
      var metadata = format ("%5d x %5d x %d", image.width, image.height, image.numberOfChannels);
      node.setText (1, metadata);

      return node;
   }

   // Build the view tree structure
   var windows = ImageWindow.windows;
   for (var i = 0; i < windows.length; ++i) {
      var node = addViewNode (this.target_List, windows[i].mainView);
      node.expanded = false; // or true to initially expand all preview lists

      var previews = windows[i].previews;
      for (var j = 0; j < previews.length; ++j)
         addViewNode (this.target_List, previews[j]);
   }
   this.target_List.sort();

   // Ensure that all columns are initially visible
   this.target_List.adjustColumnWidthToContents (0);
   this.target_List.adjustColumnWidthToContents (1);

   // ----- CHECKBOXES

   this.selectImages_CB = new CheckBox (this);
   this.selectImages_CB.text = "Select all images";
   this.selectImages_CB.checked = false;
   this.selectImages_CB.toolTip = "Tick to select all entire images.";
   this.selectImages_CB.onCheck = function (checked) {
      for (var i = 0; i < this.dialog.target_List.numberOfChildren; i++) {
         var str = this.dialog.target_List.child (i).text (0);
         if (-1 == str.indexOf ("->")) {   // this is an image
            this.dialog.target_List.child (i).checked = checked;
         }
      }
   };

   this.selectPreviews_CB = new CheckBox (this);
   this.selectPreviews_CB.text = "Select all previews";
   this.selectPreviews_CB.checked = false;
   this.selectPreviews_CB.toolTip = "Tick to select all previews.";
   this.selectPreviews_CB.onCheck = function (checked) {
      for (var i = 0; i < this.dialog.target_List.numberOfChildren; i++) {
         var str = this.dialog.target_List.child (i).text (0);
         if (-1 != str.indexOf ("->")) {   // this is a preview
            this.dialog.target_List.child (i).checked = checked;
         }
      }
   }

   this.checkBoxes_Sizer = new HorizontalSizer;
   this.checkBoxes_Sizer.spacing = 4;
   this.checkBoxes_Sizer.add (this.selectImages_CB);
   this.checkBoxes_Sizer.add (this.selectPreviews_CB);
   this.checkBoxes_Sizer.addStretch();
   this.checkBoxes_Sizer.add (this.toggleSimilar_CB);

   this.target_GroupBox = new GroupBox (this);
   this.target_GroupBox.title = "Images to integrate";
   this.target_GroupBox.sizer = new VerticalSizer;
   this.target_GroupBox.sizer.margin = 4;
   this.target_GroupBox.sizer.spacing = 4;
   this.target_GroupBox.sizer.add (this.checkBoxes_Sizer);
   this.target_GroupBox.sizer.add (this.target_List, 100);

   // ----- KAPPA NUMERIC CONTROL

   this.kappa_NC = new NumericControl (this);
   this.kappa_NC.label.text = "Kappa value:";
   this.kappa_NC.setRange (0.1, 10);
   this.kappa_NC.slider.setRange (0, 198);
   //this.kappa_NC.slider.minWidth = 150;
   this.kappa_NC.setPrecision (2);
   this.kappa_NC.setValue (engine.kappa);
   this.kappa_NC.toolTip = "Threshold for eliminating hotpixels."
   this.kappa_NC.onValueUpdated = function (value) {
      engine.kappa = value;
   }

   // ----- BUTTONS

   this.ok_Button = new PushButton (this);
   this.ok_Button.text = " OK ";
   // transfer the names of the selected images to the engine
   this.ok_Button.onClick = function() {
      // couldn't achieve this using this.dialog.target_List.selectedNodes, its length is always 1
      for (n = 0; n < this.dialog.target_List.numberOfChildren; n++)
         if (this.dialog.target_List.child (n).checked)
            engine.imgIDs.push (this.dialog.target_List.child (n).text (0));
      this.dialog.ok();
   };

   this.cancel_Button = new PushButton (this);
   this.cancel_Button.text = " Cancel ";
   this.cancel_Button.onClick = function() {
      this.dialog.cancel();
   };

   this.buttons_Sizer = new HorizontalSizer;
   this.buttons_Sizer.spacing = 4;
   this.buttons_Sizer.addStretch();
   this.buttons_Sizer.add (this.ok_Button);
   this.buttons_Sizer.add (this.cancel_Button);

   // ----- PACK EVERYTHING

   this.sizer = new VerticalSizer;
   this.sizer.margin = 6;
   this.sizer.spacing = 6;
   this.sizer.add (this.helpLabel);
   this.sizer.addSpacing (4);
   this.sizer.add (this.target_GroupBox, 100);
   this.sizer.add (this.kappa_NC);
   this.sizer.add (this.buttons_Sizer);

   this.windowTitle = "KSIntegration v" + VERSION;
   this.adjustToContents();
}

kappa_sigma_dialog.prototype = new Dialog;

function main() {
   // used for computing elapsed time
   var startts = new Date;

   // Hide the console while our dialog is active.
   console.hide();

   var dialog = new kappa_sigma_dialog();

   for (;;) {
      if (!dialog.execute())
         break;

      console.show();
      console.abortEnabled = true;

      engine.load();
      engine.combine();

      // Quit after successful execution.
      break;
   }

   var endts = new Date;
   console.writeln (format ("<end><cbr><br>%.2f s",
      (endts.getTime() - startts.getTime()) / 1000));
}

main();

[David Serrano]

I do not always necrobump…


…but when I do, it's in an 8 year old own thread 8)

I'm curious about any specific reason why this script hasn't made its way into the bundle. Half an hour ago I thought it had just become rotten and a decision had been made to drop it, however I tried running it and, lo and behold, still works like a charm! So I infer that it was never included in the first place. Is there anything better these days?

For anyone interested, at this point (PixInsight 1.8.3.1123) there are two fixes needed:

• At line 336 we have "for (n = 0; ..." but that gives a warning "assignment to undeclared variable". Turning it into "for (var n = 0" makes the error go bother someone else.
• At 154 there's a more serious one: "ImageWindow.zoomToFit(): Wrong number of arguments". The script calls "window.zoomToFit (false)", I guess there has been an API change here. "window.zoomToFit ()" works fine.