Author Topic: PixInsight 1.8.0 Ripley: Mozilla SpiderMonkey 24.2 JavaScript Engine  (Read 8843 times)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
The latest build 1065 of the PixInsight Core application integrates a new JavaScript engine: Mozilla's SpiderMonkey version 24.2. This version is essentially the same used by the Firefox 24 web browser.

The most important feature of SM 24 is the new IonMonkey just-in-time (JIT) compiler. IonMonkey is a powerful optimizing JIT that translates JavaScript interpreted code to machine code. It replaces completely the old JaegerMonkey JIT that was included in SM 17, and hence in previous versions of PixInsight 1.8.

Performance Improvements

So what performance improvements can we expect for script execution on PixInsight? In an embedded interpreter/compiler, this basically depends on the amount of native calls performed, as well as on how the scripts have been designed and implemented. If most of a script's running time is spent executing native PJSR routines implemented in the PixInsight Core application, the performance gains can be marginal. However, for scripts running mostly pure JavaScript code, the new JIT compiler can yield significant performance improvements.

The following script calculates the Gaussian efficiency of a statistical estimator of scale, as well as the corresponding scaling factor to make the estimator coherent with a normal distribution. The script implements a bootstrap estimation scheme.

Code: [Select]
/*
 * A bootstrap estimation of Gaussian efficiency of statistical estimators.
 */

/*
 * Returns a Vector object with a random sample of size n from a standard
 * normal distribution.
 *
 * Generates n random normal deviates using the Marsaglia polar method:
 *    http://en.wikipedia.org/wiki/Box%E2%80%93Muller_transform
 *    http://en.wikipedia.org/wiki/Marsaglia_polar_method
 */
function normalSample( n )
{
   var g = new Vector( n );
   for ( var i = 0; i < n; ++i )
   {
      var s, u, v; // I love SUVs :)
      do
      {
         u = 2*Math.random() - 1;
         v = 2*Math.random() - 1;
         s = u*u + v*v;
      }
      while ( s >= 1 || s == 0 );
      s = Math.sqrt( -2*Math.ln( s )/s );
      g.at( i, u*s );
      if ( ++i < n )
         g.at( i, v*s );
   }
   return g;
}

/*
 * Iterative k-sigma estimator of location and scale (IKSS).
 * Returns an Array object [location,scale].
 */
function IKSS( x, eps )
{
   if ( eps === undefined )
      eps = 1.0e-06;
   x.sort();
   var i = 0;
   var j = x.length;
   var s0 = 1;
   for ( ;; )
   {
      if ( j - i < 1 )
         return [0,0];
      var v = new Vector( x, i, j );
      var m = v.median();
      var s = Math.sqrt( v.BWMV( m ) );
      if ( 1 + s == 1 )
         return [m,0];
      if ( (s0 - s)/s < eps )
         return [m,/*0.991**/s];
      s0 = s;
      for ( var x0 = m - 4*s; x.at( i ) < x0; ++i ) {}
      for ( var x1 = m + 4*s; x.at( j-1 ) > x1; --j ) {}
   }
}

/*
 * Bootstrap estimation
 */

#define sampleSize      2000
#define numberOfSamples 1000
#define numberOfTests   25

console.show();
console.writeln( "<end><cbr>Performing bootstrap, please wait..." );
console.flush();

var t0 = new Date;

var sigma_a = new Vector( numberOfTests );
var sigma_m = new Vector( numberOfTests );
var k = 0;
for ( var j = 0; j < numberOfTests; ++j )
{
   var a = new Vector( numberOfSamples );
   var m = new Vector( numberOfSamples );
   var kj = 0;
   for ( var i = 0; i < numberOfSamples; ++i )
   {
     var x = normalSample( sampleSize );
     var s1 = x.stdDev();
     var s2 = IKSS( x )[1];
     //var s2 = Math.sqrt( x.BWMV() );
     //var s2 = x.MAD();
     //var s2 = x.avgDev();
     a.at( i, s1 );
     m.at( i, s2 );
     kj += s1/s2;
   }
   kj /= numberOfSamples;
   k += kj;
   m.mul( k/(j + 1) );
   sigma_a.at( j, a.variance() );
   sigma_m.at( j, m.variance() );
}
k /= numberOfTests;

var t1 = new Date;

var aa = sigma_a.mean();
var ma = sigma_m.mean();
console.writeln( format( "<end><cbr>Variance of stddev ...... %.4e", aa ) );
console.writeln( format(           "Variance of estimator ... %.4e", ma ) );
console.writeln( format(           "Gaussian efficiency ..... %.3f", aa/ma ) );
console.writeln( format(           "Normalization factor .... %.5f", k ) );

console.writeln( format( "%.2f s", (t1.getTime() - t0.getTime())/1000 ) );

I have used this script as an intermediate example between dependency on native routine calls and execution of pure JavaScript code. The following graph plots execution times in seconds on PixInsight Core 1.8.0.1054 (previous prerelease version using SpiderMonkey 17) and 1.8.0.1065 (current prerelease with SpiderMonkey 24). The machine is a 6-core Intel Core i7 990X with 24 GB of RAM running Fedora 16 Linux.


The first thing that becomes obvious is that execution times are more stable in the new JavaScript engine. SM 17 oscillates between 10.7 and 12.8 seconds approximately, while SM 24 remains quite stable around 9.8 seconds. The new version is about a 16% faster on average for this example.

New Language Features

The new JavaScript engine comes with several new language features. Among the most important ones are JavaScript typed arrays. This includes the following new standard objects:

ArrayBuffer
ArrayBufferView
DataView
Int8Array
Uint8Array
Uint8ClampedArray
Int16Array
Uint16Array
Int32Array
Uint32Array
Float32Array
Float64Array

These objects are now directly available in PJSR.

If you want to try out one of the latest and most exciting technologies, the new JavaScript engine also comes with asm.js support. asm.js is a strict subset of JavaScript that can be used as a low-level, efficient target language.

Future Development

Integration of SpiderMonkey 24 is just the beginning of a series of planned improvements for PixInsight's JavaScript development environment. During the next year we'll have an integrated JavaScript debugger in the PixInsight Core application. We'll work on a mixed native/JavaScript execution model that will allow developers to define critical script sections as C/C++ blocks that will be compiled and linked automatically by the core application (a very similar concept to kernel functions in GPU programming). Finally, we are working also to improve integration of scripts with the PixInsight platform. Ideally, a script should be allowed to run asynchronously (e.g., using nonmodal graphical interfaces) just as module-defined tools. With all of these new development lines our goal is to make PixInsight the most powerful and attractive development platform specialized in image processing.
« Last Edit: 2013 December 15 16:28:54 by Juan Conejero »
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4729
Finally, we are working also to improve integration of scripts with the PixInsight platform. Ideally, a script should be allowed to run asynchronously (e.g., using nonmodal graphical interfaces) just as module-defined tools. .

great! i was just going to ask for that pony!

rob

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
Wow, that's exciting stuff.

One question: There are many java script libraries (see http://en.wikipedia.org/wiki/List_of_JavaScript_libraries) or frameworks (http://en.wikipedia.org/wiki/Comparison_of_JavaScript_frameworks)  out there. Is any of these compatible with PJSR? Which ones would be useful and deserve a closer look?

Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline Juan Conejero

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 7111
    • http://pixinsight.com/
In principle, any reasonably well designed JavaScript framework should be compatible with PJSR. However, most of these frameworks rely on functionality provided by web browsers, which doesn't exist in PixInsight/PJSR for obvious reasons.

So for example, if a graphical visualization library relies on HTML 5's Canvas object, it won't work in PixInsight. The same is true for any framework requiring W3 DOM objects. PixInsight has its own object model, which is not document-oriented, and hence has nothing to do with a web browser's model. This limits the applicability of standard JavaScript frameworks considerably.

One possibility to improve on this front would be a new Canvas PJSR object to emulate all the functionality of HTML 5 Canvas. It would be relatively easy to write in pure JavaScript around our VectorGraphics core PJSR object. The same could be done with HTML 5 SVG.
Juan Conejero
PixInsight Development Team
http://pixinsight.com/

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
... So what performance improvements can we expect for script execution on PixInsight? In an embedded interpreter/compiler, this basically depends on the amount of native calls performed, as well as on how the scripts have been designed and implemented. If most of a script's running time is spent executing native PJSR routines implemented in the PixInsight Core application, the performance gains can be marginal. However, for scripts running mostly pure JavaScript code, the new JIT compiler can yield significant performance improvements. ...
To add one more data point to your experiments: I tried the CanonBandingReduction script with both 1054 (old engine) and 1070 (new engine). I added a few calls to time the most time consuming parts. It should be noted that 1054 is already a lot faster than the PI version that I used when I originally developed this script. Here the results:
1054:
Code: [Select]
BandingEngine.doConvertImage() required time [ms]=162...
BandingEngine.doStatistics() required time [ms]=7268...
BandingEngine.doResult() required time [ms]=7892

1070:
Code: [Select]
BandingEngine.doConvertImage() required time [ms]=181...
BandingEngine.doStatistics() required time [ms]=4859...
BandingEngine.doResult() required time [ms]=5622

This documents a speed advantage of roughly 40%!

It is a bit surprising since most of the operations largely happen on the C-level. For instance, ConvertImage() converts the data from integer to float format (one call to C). DoStatistics() does a ConvertImage(), then generates linewise image statistics and generates a correction image from this: Again calls that handle images in linewise chunks on the C-level. This is the part the benefits most from the new engine. Finally, doResult() does doStatistics() and then  applies the correction image via multiply-and-add to the original. All the benefit here comes from the call to doStatistics(), not the actual operations done here.

Amazing...
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)

Offline georg.viehoever

  • PTeam Member
  • PixInsight Jedi Master
  • ******
  • Posts: 2132
...If you want to try out one of the latest and most exciting technologies, the new JavaScript engine also comes with asm.js support. asm.js is a strict subset of JavaScript that can be used as a low-level, efficient target language....

Could you provide an asm.js example that does something useful on a PI images? After having a glance at the asm.js spec, I am not sure on how to do that.
Georg
Georg (6 inch Newton, unmodified Canon EOS40D+80D, unguided EQ5 mount)