Making ExternalProcess Run Synchronously?

[daveachuk]

Context: I'm using FFmpeg to handle lengthy timelapse sequences. Each sequence is kept together in an AVI container for easy file handling and previewing (with no compression). On Windows. The PJSR script reads in a directory of AVI files, unpacks them one at a time into the component images, does some processing steps (stuck pixel and noise reduction), then combines them back into an AVI file using FFmpeg again.

I'm having a bit of a problem though because the ExternalProcess function is executing asynchronously. It seems like the ExternalProcess.start() function creates a background thread and execution of the next statement of code happens before the ExternalProcess completes (eg in one case, the code that follows the ExternalProcess.start() call reads the directory of individual images that were just extracted from the AVI, but it says the directory is empty; as well when piecing the individual images back into an AVI at the end of processing, the next statements delete the temporary images, but then FFMPEG complains that there are no images in the directory to add to the AVI as they've already been deleted).

A very crude workaround is to add a sleep() call, which is a bit of a waste of time, but gives ffmpeg enough time to finish what it's doing before the main thread continues. The lost time is not a huge deal since this is many, many hours of processing involved, but I'm worried about running a task overnight and the sleep() delay not being long enough and then I come back in the morning to find the job hasn't completed.

Why not use the ExternalProcess.onFinished event handler? I tried, but it seems to not work:

Code: [Select]


   var ext = new ExternalProcess(); 
   ext.onStateChanged = function(state)   { console.writeln(' statechange: ' + state); }
   ext.onStarted  = function()            { console.writeln(' starting....');   }
   ext.onFinished = function(code,status) { console.writeln(' finished:' + code + ' / ' + status);       }   
   ext.onError    = function(code)        { console.writeln(' ERROR: ' + code); }
   ext.onStandardOutputDataAvailable = function() { console.writeln('     stdout = ' + ext.standardOutput); }
   ext.onStandardErrorDataAvailable  = function() { console.writeln('     stderr = ' + ext.standardError);  }

   ext.start(prog, args);

The output from the above is:

Code: [Select]


 statechange: 1
 statechange: 2
 starting....

so onStateChanged and onStarted work fine. onError also works if you supply an invalid command, but onFinished never gets called, nor do onStandardOutputDataAvailable or onStandardErrorDataAvailable. This was the same with multiple commands (eg running a batch file rather than FFMPEG).

Wondering if there's something I'm doing wrong or something I haven't understood from the API listing. Any help appreciated. Until then I will let PixInsight do some sleep() while I do the same!

[Juan Conejero]

Hi Dave,

I apologize for the extremely long delay in getting back to you to solve this issue. Unfortunately, I am alone implementing and testing a large software platform, which depletes almost all of my time.

After starting an external process, you must wait until the process starts and finishes execution. This is because an external process necessarily runs on a separate thread, so if your code doesn't wait, you don't allow it to receive events. The following test script encapsulates these ideas as a ProcessExecution convenience object, around the sample code you posted:

Code: [Select]

function ProcessExecution( program, args )
{
   this.program = program;

   if ( args === undefined )
      this.args = [];
   else
      this.args = args;

   this.process = new ExternalProcess;

   this.process.onStateChanged = function( state )
   {
      console.writeln( 'statechange: ' + state );
   };

   this.process.onStarted = function()
   {
      console.writeln( 'starting...' );
   };

   this.process.onFinished = function( code, status )
   {
      console.writeln( 'finished:' + code + ' / ' + status );
   };

   this.process.onError = function( code )
   {
      console.criticalln(' ERROR: ' + code);
   };

   this.process.onStandardOutputDataAvailable = function()
   {
      console.writeln( 'stdout = ' + this.standardOutput.toString() );
   };

   this.process.onStandardErrorDataAvailable  = function()
   {
      console.writeln( 'stderr = ' + this.standardError.toString() );
   };

   this.process.start( this.program, this.args );
   for ( ; this.process.isStarting; )
      processEvents();
   for ( ; this.process.isRunning; )
      processEvents();
}

let P = new ProcessExecution( "ls", ["-la"] );

Note that the calls to processEvents() are *absolutely* necessary, not just to keep the platform responsive (which may be unimportant for processes that terminate quickly), but because in order to receive events from the running process, you have to process them from your script.

Let me know if this is what you want.