Proposal: Optimizing scripts with special native modules

[Juan Conejero]

Hi scriptaholics

It is becoming increasingly evident that scripting support in PixInsight has tremendous possibilities that surpass by far my initial expectations, when I embedded the Mozilla JavaScript engine (SpiderMonkey) in late 2006.

As we devise more applications of scripts on the PixInsight platform, we see that scripts are now addressing problems that go far beyond simple batch processing, trying to solve complex image processing tasks.

However, scripts in their current form suffer from a severe bottleneck: when we have to process millions of pixel samples, interpreted code is too slow as to represent a feasible solution in real-world cases.

So, an idea came to my mind a few days ago. Why not allow scripts lo load and execute (in a reasonably secure way) small modules compiled to native code? The idea is quite simple:

1. Consider small shared objects (.so files under Linux/UNIX/OSX; .dll files under Windows) that implement relatively small routines written in C (or C++ with a C front-end). These routines implement the bottlenecks of our scripts. For example, the typical loop that iterates through every pixel in an image. Nothing prevents us from parallelizing these routines, of course.

2. Define a standard protocol to communicate these small shared objects with the JavaScript runtime and the calling script. More specifically, define a collection of C function prototypes to solve common pixel mass-processing tasks, so that the JS runtime can look for them into the shared objects.

3. Define a standard way to load and unload these shared objects, to know if a given standard routine is available (as defined in point 2), and to run standard routines from JavaScript code. Of course, the script would "call" these routines with parameters that specialize them for a particular image. Easy example. Let's say that one of these routines is:

Code: [Select]

void forEachSample( float* samples, int width, int height );

Then the script could call it with samples = the beginning of a channel of a particular image, and width, height = the dimensions of the image.

That's the idea, basically. Come on, expose your thoughts here. I think that if we manage to implement something like this well, then scripts can be real solutions to real problems in PixInsight. With purely interpreted code, we are very limited.

By the way, how these small native objects could be called? The words "script" and "gadget" come to my mind (because I tend to think that the name should end with the -let suffix), but I don't know how to link them. Any ideas as for the name of these little beasts?

[David Serrano]

So, an idea came to my mind a few days ago. Why not allow scripts lo load and execute (in a reasonably secure way) small modules compiled to native code?

Sorry, but I can't see the difference between this new mechanism and the existing Javascript objects. It seems clear to me that these modules aren't going to be distributed by JS scripts' authors (if this were the case, we'd end up with several implementations of the same algorithms floating around, and people would keep on searching for the "best" one), so there should be one or more reference modules that implement everything scripts will ever need. These reference modules could be integrated in the PixInsight distribution, or distributed separately. But in any case, the only option I see for making the routines visible from JS is via a new, say, "Externals" object. Doing otherwise would be a waste of time and effort.

This is so clear to me, that I think I'm missing something. Specifically, I'd like to have a couple of examples of those "common pixel mass-processing tasks" you talk about. For example, let's say one of those tasks is "add X to any pixel value". That could be done from JS with something like:

Code: [Select]

Externals.addToPixel (Image I, Float X, Boolean Rescale);

Code: [Select]

// usage example
var fooImage = new Image;
var value = 0.1;
Externals.addToPixel (fooImage, value, false);

And this would take advantage of the existing infrastructure by means of which scripts are already able to run PCL code and external processes.

[Juan Conejero]

Hi David,

Sorry, but I can't see the difference between this new mechanism and the existing Javascript objects.

The difference is that these new modules would provide a standardized mechanism to link scripts with small pieces of extremely efficient native code, to solve pixel mass-processing tasks.

Existing JavaScript objects and their methods are of course implemented as highly optimized, parallel native code, so there's no point in trying to repeat what they already do. These modules are just for the rest of the infinite set of tasks that can be solved with scripts

It seems clear to me that these modules aren't going to be distributed by JS scripts' authors (if this were the case, we'd end up with several implementations of the same algorithms floating around, and people would keep on searching for the "best" one)

That is precisely the idea: authors of high-performance scripts would write their own native modules containing specialized routines to fix the bottlenecks of their scripts.

Well, I don't see as a real problem the possibility of two or more authors writing similiar routines to solve similar tasks --after all, that's called software development :lol:

There is also an additional "nice" feature that native modules would add to the JavaScript universe on PixInsight: the possibility to hide some sensitive parts of a script's code, as proprietary algorithms and implementations. Heaven forbids, but yes there seem to be bad guys out there that want to hide code... :-S

Seriously now. I think a more elaborated example is necessary to clarify things at this point. Consider the following script code. It uses a fragment of the PolarCoordinates.js script that comes with the standard distribution:

Code: [Select]

/**
 * Transformation of an image from Cartesian to polar coordinates.
 */
function CartesianToPolar( image )
{
   // Gather working parameters
   var n = image.numberOfChannels;
   var w = image.width;
   var h = image.height;
   var w2 = 0.5*w;
   var h2 = 0.5*h;
   var r0 = Math.sqrt( w2*w2 + h2*h2 ); // semi-diagonal

   // Create a working image to store one channel of the target image
   var tmp = new Image( w, h, 1 );

   // Initialize the status monitoring system for this image.
   // The status monitor will provide progress information on the console.
   image.statusEnabled = true;
   image.initializeStatus( "Polar coordinate transform", 2*w*h*n );

   // Don't allow other routines to re-initialize the status monitor
   image.statusInitializationEnabled = false;

   // Reset the rectangular selection to the whole image boundaries
   image.resetRectSelection();

   // For each channel
   for ( var c = 0; c < n; ++c )
   {
      tmp.fill( 0 ); // initialize working data with zeros

/* *** BEGIN BOTTLENECK *** */

      // For each row
      for ( var i = 0; i < h; ++i )
      {
         // Polar angle for the current row
         var theta = 2*Math.PI*i/h;
         var stheta = Math.sin( theta );
         var ctheta = Math.cos( theta );

         // For each column
         for ( var j = 0; j < w; ++j )
         {
            // Radial distance for the current column
            var r = r0*j/w;

            // Horizontal coordinate on the source image
            var x = w2 + r*ctheta;

            if ( x >= 0 && x < w )
            {
               // Vertical coordinate on the source image
               var y = h2 - r*stheta;

               // Copy the source pixel to the polar-transformed location on
               // the working image.
               if ( y >= 0 && y < h )
                  tmp.setSample( image.interpolate( x, y, c ), j, i );
            }
         }

         // Update status monitoring (progress information)
         image.advanceStatus( w );
      }

/* *** END BOTTLENECK *** */

      // Copy our working data to the channel c of image
      image.selectedChannel = c;
      image.apply( tmp );
   }

   delete tmp;
}

The routine above performs a polar coordinate transform. The code isn't complicated at all; however, the routine is quite slow. This PixelMath expression does the same and runs nearly ten times faster on a dual core machine:

Code: [Select]

var p1 = new PixelMath;
with ( p1 )
{
   variables = "r, theta, x, y, z";
   expression = "r = Sqrt( Width()/2*Width()/2 + Height()/2*Height()/2 )*X(); " +
                "theta = 2*Pi()*Y(); " +
                "x = Round( Width()/2 + r*Cos( theta ) ); " +
                "y = Round( Height()/2 - r*Sin( theta ) ); " +
                "Pixel( $target, x, y )";
}

p1.executeOn( ImageWindow.activeWindow.mainView );

PixelMath uses also an interpreted language, however PixelMath is highly optimized for the very reduced set of tasks it performs and it is a completely static language, while JavaScript is a general-purpose, extremely dynamic language.

How about a change like this:

Code: [Select]

/**
 * Transformation of an image from Cartesian to polar coordinates.
 */
function CartesianToPolar( image )
{
   // Gather working parameters
   var n = image.numberOfChannels;
   var w = image.width;
   var h = image.height;
   var w2 = 0.5*w;
   var h2 = 0.5*h;
   var r0 = Math.sqrt( w2*w2 + h2*h2 ); // semi-diagonal

   // Create a working image to store one channel of the target image
   var tmp = new Image( w, h, 1 );

   // Initialize the status monitoring system for this image.
   // The status monitor will provide progress information on the console.
   image.statusEnabled = true;
   image.initializeStatus( "Polar coordinate transform", 2*w*h*n );

   // Don't allow other routines to re-initialize the status monitor
   image.statusInitializationEnabled = false;

   // Reset the rectangular selection to the whole image boundaries
   image.resetRectSelection();

   // Install the native module
   // ... This is the complex part, which we don't know how to implement yet ...

   // For each channel
   for ( var c = 0; c < n; ++c )
   {
      // Initialize working data with zeros
      tmp.fill( 0 );

      // Allow the native module to update status info through the tmp image
      tmp.assignStatus( image );

      // Call our native module. It is supposed to be installed in some way,
      // somewhere before calling it here.
      tmp.callNativeModule( "myPolarCoordinateTransform.so", image, c );

      // Update image's status monitor
      image.assignStatus( tmp );

      // Copy our working data to the channel c of image
      image.selectedChannel = c;
      image.apply( tmp );
   }

   // Uninstall the native module
   // ... We also don't know this part, but it should be very easy ...

   delete tmp;
}

The "myPolarCoordinateTransform.so" shared object would implement the two inner loops of the transform in native code. The (bogus) call to callNativeModule() is where our script would run the native routine. Now our script would be virtually indistinguishable from a pure native implementation in terms of execution speed.

Now the problem is how to standardize a mechanism to install a shared module callable from script code with a variable number and type of parameters. This job is complex and must be exquisitely done, or we may end with more problems than solutions. But I think the benefits are well worth the effort.

[David Serrano]

Sorry for the delay. My empty skull unconsciously forgot this.

Existing JavaScript objects and their methods are of course implemented as highly optimized, parallel native code, so there's no point in trying to repeat what they already do. These modules are just for the rest of the infinite set of tasks that can be solved with scripts

[es]Lo siento, soy duro de mollera[/es] Of course, what is already done shouldn't be repeated but for implementing these new tasks, the same mechanism could be used. Note that I'm talking about the mechanism, not the processes (in the usual meaning of the word ).

That is precisely the idea: authors of high-performance scripts would write their own native modules containing specialized routines to fix the bottlenecks of their scripts.

And that routines could be made available as PixInsight modules (*-pm32.* files) that would do nothing but create the needed object(s) in the JS object model. And implement their behaviour, of course . So the JS code you suggest could be like this:

Code: [Select]

  // Install the native module
   // ... This is the complex part, which we don't know how to implement yet ...

Code: [Select]

  my polar = new polar_coordinates;    // I hate CamelCase ;)

Code: [Select]

     // Call our native module. It is supposed to be installed in some way,
      // somewhere before calling it here.
      tmp.callNativeModule( "myPolarCoordinateTransform.so", image, c );

Code: [Select]

     polar.transform (image, c);

Code: [Select]

  // Uninstall the native module
   // ... We also don't know this part, but it should be very easy ...

Code: [Select]

  polar = null; gc();    // or something like that

Now the problem is how to standardize a mechanism to install a shared module callable from script code with a variable number and type of parameters.

I see no problem, other that the memory claimed by the module would be unavailable to the system for all the time PixInsight is running, but I think this is negligible. What does bother me is the average size of PixInsight modules... (NoOperation-pm32.so is 1.6 MiB).