batch converting loaded FITS images to JPEG?

[Nocturnal]

Hi,

I would like to start automating some image processing. Sometimes I have a stack of FITS images that I would like to animate. This is helpful to show drift of a target or in the case of observing an asteroid it actually shows the path. PCL does a nice job converting FITS images to JPEG and it's easy to drop a whole bunch onto the work surface to open them all. It seems that it should be easy to script looping through all open images and saving them as jpg.

I'm a proficient coder (Perl, Ruby, VB6, VB.NET, C/C++, a tad of Javascript) so all I need is some pointers to scripts to get me started. It would be *really* sweet if PCL can record a user interaction like Excel and other office applications do. Ok, I can dream, right? :-)

Maybe this can be done with simple command line commands instead of a script?

Thanks,

  Sander

[David Serrano]

Not a great answer, but sometimes just the tip of the iceberg is needed to discover the rest .

I think it can be done using an ImageContainer. Try with it.

[Carlos Milovic]

Indeed. All you need is to build a ImageContainer with the files, specify that the output should be a jpeg archive... then, apply the NoOperation process to the container.


We know that this is quite tricky... in the following weeks we'll write the oficial documentation for the core application, and surelly this will be an example of the power of image containers.

[Juan Conejero]

Hi Sander and all,

Here is a simple batch conversion script:

Code: [Select]

/*
 * A simple batch image file format conversion script.
 * STANDARD DISCLAIMER: Tested to work well, but use carefully.
 */

#include <pjsr/Sizer.jsh>
#include <pjsr/FrameStyle.jsh>
#include <pjsr/TextAlign.jsh>
#include <pjsr/StdButton.jsh>
#include <pjsr/StdIcon.jsh>

#define DEFAULT_EXTENSION  ".jpg"

#define WARN_ON_NO_OUTPUT_DIRECTORY 1

// ----------------------------------------------------------------------------
// Batch Conversion Engine
// ----------------------------------------------------------------------------

function BatchConversionEngine()
{
   this.inputFiles = new Array;
   this.outputDirectory = "";
   this.outputExtension = DEFAULT_EXTENSION;
   this.overwriteExisting = false;

   this.convertFiles = function()
   {
      for ( var i = 0; i < this.inputFiles.length; ++i )
      {
         var w = new ImageWindow( this.inputFiles[i] );

         if ( w.isNull && i+1 < this.inputFiles.length )
         {
            var msg = new MessageBox( "<p>Unable to load input image file:</p>" +
                                      "<p>" + this.inputFiles[i] + "</p>" +
                                      "<p><b>Continue batch format conversion?</b></p>",
                                      "BatchConversion Script",
                                      StdIcon_Error, StdButton_Yes, StdButton_No );
            if ( msg.execute() == StdButton_No )
               break;
         }

         // Build the output file path from its separate components:
         //    drive+directory+name+extension
         // Of course, drive is always an empty string under Linux/UNIX/OSX

         var fileDir = (this.outputDirectory.length != 0) ?
            this.outputDirectory : File.extractDrive( this.inputFiles[i] ) +
                                   File.extractDirectory( this.inputFiles[i] );

         // Ensure that our directory string ends with a slash separator.
         if ( fileDir.length != 0 && fileDir.charAt( fileDir.length-1 ) != '/' )
            fileDir += '/';

         var fileName = File.extractName( this.inputFiles[i] );

         var outputFilePath = fileDir + fileName + this.outputExtension;

         if ( !this.overwriteExisting && File.exists( outputFilePath ) )
         {
            // Obtain a nonexisting file name by appending an underscore and a
            // growing integer to the output file name.
            for ( var u = 1; ; ++u )
            {
               var tryFilePath = File.appendToName( outputFilePath, '_' + u.toString() );
               if ( !File.exists( tryFilePath ) )
               {
                  outputFilePath = tryFilePath;
                  break;
               }
            }
         }

         // Write the output image.
         // Force ImageWindow to disable all format and security features:
         // * Query format-specific options
         // * Warning messages on missing format features (icc profiles, etc)
         // * Strict image writing mode (ignore lossy image generation)
         // * Overwrite verification/protection
         w.saveAs( outputFilePath, false, false, false, false );

         // Close the image window.
         // Note that we haven't called w.show(), so it is hidden.
         w.close();
      }
   }
}

var engine = new BatchConversionEngine;

// ----------------------------------------------------------------------------
// Batch Conversion Dialog
// ----------------------------------------------------------------------------

function BatchConversionDialog()
{
   // Add all properties and methods of the core Dialog object to this object.
   this.__base__ = Dialog;
   this.__base__();

   //

   this.helpLabel = new Label( this );
   this.helpLabel.frameStyle = FrameStyle_Box;
   this.helpLabel.margin = 4;
   this.helpLabel.wordWrapping = true;
   this.helpLabel.useRichText = true;
   this.helpLabel.text = "<b>BatchConversion</b> - A batch image format conversion utility.";

   //

   this.files_TreeBox = new TreeBox( this );
   this.files_TreeBox.multipleSelection = true;
   this.files_TreeBox.setMinSize( 500, 200 );
   this.files_TreeBox.numberOfColumns = 1;
   this.files_TreeBox.headerVisible = false;

   for ( var i = 0; i < engine.inputFiles.length; ++i )
   {
      var node = new TreeBoxNode( this.files_TreeBox );
      node.setText( 0, engine.inputFiles[i] );
   }

   this.filesAdd_Button = new PushButton( this );
   this.filesAdd_Button.text = " Add ";
   this.filesAdd_Button.toolTip = "<p>Add image files to the input images list.</p>";

   this.filesAdd_Button.onClick = function()
   {
      var ofd = new OpenFileDialog;
      ofd.multipleSelections = true;
      ofd.caption = "Select Images";
      ofd.loadImageFilters();

      if ( ofd.execute() )
      {
         this.dialog.files_TreeBox.canUpdate = false;
         for ( var i = 0; i < ofd.fileNames.length; ++i )
         {
            var node = new TreeBoxNode( this.dialog.files_TreeBox );
            node.setText( 0, ofd.fileNames[i] );
            engine.inputFiles.push( ofd.fileNames[i] );
         }
         this.dialog.files_TreeBox.canUpdate = true;
      }
   };

   this.filesClear_Button = new PushButton( this );
   this.filesClear_Button.text = " Clear ";
   this.filesClear_Button.toolTip = "<p>Clear the list of input images.</p>";

   this.filesClear_Button.onClick = function()
   {
      this.dialog.files_TreeBox.clear();
      engine.inputFiles.length = 0;
   };

   this.filesInvert_Button = new PushButton( this );
   this.filesInvert_Button.text = " Invert Selection ";
   this.filesInvert_Button.toolTip = "<p>Invert the current selection of input images.</p>";

   this.filesInvert_Button.onClick = function()
   {
      for ( var i = 0; i < this.dialog.files_TreeBox.numberOfChildren; ++i )
         this.dialog.files_TreeBox.child( i ).selected =
               !this.dialog.files_TreeBox.child( i ).selected;
   };

   this.filesRemove_Button = new PushButton( this );
   this.filesRemove_Button.text = " Remove Selected ";
   this.filesRemove_Button.toolTip = "<p>Remove all selected images from the input images list.</p>";

   this.filesRemove_Button.onClick = function()
   {
      engine.inputFiles.length = 0;
      for ( var i = 0; i < this.dialog.files_TreeBox.numberOfChildren; ++i )
         if ( !this.dialog.files_TreeBox.child( i ).selected )
            engine.inputFiles.push( this.dialog.files_TreeBox.child( i ).text( 0 ) );
      for ( var i = this.dialog.files_TreeBox.numberOfChildren; --i >= 0; )
         if ( this.dialog.files_TreeBox.child( i ).selected )
            this.dialog.files_TreeBox.remove( i );
   };

   this.filesButtons_Sizer = new HorizontalSizer;
   this.filesButtons_Sizer.spacing = 4;
   this.filesButtons_Sizer.add( this.filesAdd_Button );
   this.filesButtons_Sizer.addStretch();
   this.filesButtons_Sizer.add( this.filesClear_Button );
   this.filesButtons_Sizer.addStretch();
   this.filesButtons_Sizer.add( this.filesInvert_Button );
   this.filesButtons_Sizer.add( this.filesRemove_Button );

   this.files_GroupBox = new GroupBox( this );
   this.files_GroupBox.title = "Input Images";
   this.files_GroupBox.sizer = new VerticalSizer;
   this.files_GroupBox.sizer.margin = 4;
   this.files_GroupBox.sizer.spacing = 4;
   this.files_GroupBox.sizer.add( this.files_TreeBox, 100 );
   this.files_GroupBox.sizer.add( this.filesButtons_Sizer );

   //

   this.outputDir_Edit = new Edit( this );
   this.outputDir_Edit.readOnly = true;
   this.outputDir_Edit.text = engine.outputDirectory;
   this.outputDir_Edit.toolTip =
      "<p>If specified, all converted images will be written to the output directory.</p>" +
      "<p>If not specified, converted images will be written to the same directories " +
      "of their corresponding input images.</p>";

   this.outputDirSelect_Button = new PushButton( this );
   this.outputDirSelect_Button.text = " Select ";
   this.outputDirSelect_Button.toolTip = "<p>Select the output directory.</p>";

   this.outputDirSelect_Button.onClick = function()
   {
      var gdd = new GetDirectoryDialog;
      gdd.initialPath = engine.outputDirectory;
      gdd.caption = "Select Output Directory";

      if ( gdd.execute() )
      {
         engine.outputDirectory = gdd.directory;
         this.dialog.outputDir_Edit.text = engine.outputDirectory;
      }
   };

   this.outputDir_GroupBox = new GroupBox( this );
   this.outputDir_GroupBox.title = "Output Directory";
   this.outputDir_GroupBox.sizer = new HorizontalSizer;
   this.outputDir_GroupBox.sizer.margin = 4;
   this.outputDir_GroupBox.sizer.spacing = 4;
   this.outputDir_GroupBox.sizer.add( this.outputDir_Edit, 100 );
   this.outputDir_GroupBox.sizer.add( this.outputDirSelect_Button );

   //

   this.outputExt_Label = new Label( this );
   this.outputExt_Label.text = "Output extension:";
   this.outputExt_Label.textAlignment = TextAlign_Right|TextAlign_VertCenter;

   this.outputExt_Edit = new Edit( this );
   this.outputExt_Edit.text = engine.outputExtension;
   this.outputExt_Edit.setFixedWidth( this.font.width( "MMMMMM" ) );
   this.outputExt_Edit.toolTip =
      "<p>Specify a file extension to identify the output file format.</p>";

   this.outputExt_Edit.onEditCompleted = function()
   {
      // Trim whitespace at both ends of the file extension string
      // ### TOTHINK: Add trim(), trimLeft() and trimRight() methods to String
      //              (which departs from ECMAScript specification) ?
      var ext = ""
      var i = 0;
      var j = this.text.length;
      for ( ; i < j && this.text.charAt( i ) == ' '; ++i ) {}
      for ( ; --j > i && this.text.charAt( j ) == ' '; ) {}
      if ( i <= j )
         ext = this.text.substring( i, j+1 ).toLowerCase();
            // Image extensions are always lowercase in PI/PCL.

      // Use the default extension if empty.
      if ( ext.length == 0 || ext == '.' )
         ext = DEFAULT_EXTENSION;

      // Ensure that ext begins with a dot character.
      else if ( ext.charAt( 0 ) != '.' )
         ext = '.' + ext;

      this.text = engine.outputExtension = ext;
   };

   this.overwriteExisting_CheckBox = new CheckBox( this );
   this.overwriteExisting_CheckBox.text = "Overwrite existing files";
   this.overwriteExisting_CheckBox.checked = engine.overwriteExisting;
   this.overwriteExisting_CheckBox.toolTip =
      "<p>Allow overwriting of existing image files.</p>" +
      "<p><b>* Warning *</b> Enabling this option may lead to irreversible data loss.</p>";

   this.overwriteExisting_CheckBox.onClick = function( checked )
   {
      engine.overwriteExisting = checked;
   }

   this.options_Sizer = new HorizontalSizer;
   this.options_Sizer.spacing = 4;
   this.options_Sizer.add( this.outputExt_Label );
   this.options_Sizer.add( this.outputExt_Edit );
   this.options_Sizer.addSpacing( 12 );
   this.options_Sizer.add( this.overwriteExisting_CheckBox );
   this.options_Sizer.addStretch();

   //

   this.ok_Button = new PushButton( this );
   this.ok_Button.text = " OK ";

   this.ok_Button.onClick = function()
   {
      this.dialog.ok();
   };

   this.cancel_Button = new PushButton( this );
   this.cancel_Button.text = " Cancel ";

   this.cancel_Button.onClick = function()
   {
      this.dialog.cancel();
   };

   this.buttons_Sizer = new HorizontalSizer;
   this.buttons_Sizer.spacing = 4;
   this.buttons_Sizer.addStretch();
   this.buttons_Sizer.add( this.ok_Button );
   this.buttons_Sizer.add( this.cancel_Button );

   //

   this.sizer = new VerticalSizer;
   this.sizer.margin = 6;
   this.sizer.spacing = 6;
   this.sizer.add( this.helpLabel );
   this.sizer.addSpacing( 4 );
   this.sizer.add( this.files_GroupBox, 100 );
   this.sizer.add( this.outputDir_GroupBox );
   this.sizer.add( this.options_Sizer );
   this.sizer.add( this.buttons_Sizer );

   this.windowTitle = "BatchConversion Script";
   this.adjustToContents();
}

// Our dialog inherits all properties and methods from the core Dialog object.
BatchConversionDialog.prototype = new Dialog;

// ----------------------------------------------------------------------------

/*
 * Script entry point.
 */
function main()
{
   console.hide();

   // Show our dialog box, quit if cancelled.
   var dialog = new BatchConversionDialog();
   for ( ;; )
   {
      if ( dialog.execute() )
      {
         if ( engine.inputFiles.length == 0 )
         {
            var msg = new MessageBox(
                  "No input files have been specified!",
                  "BatchConversion Script", StdIcon_Error, StdButton_Ok );
            msg.execute();
            continue;
         }
     
#ifneq WARN_ON_NO_OUTPUT_DIRECTORY 0
         if ( engine.outputDirectory.length == 0 )
         {
            var msg = new MessageBox(
                  "No output directory has been specified." +
                  "\nImage files will be written to their current directories." +
                  "\nAre you sure ?",
                  "BatchConversion Script", StdIcon_Warning, StdButton_Yes, StdButton_No );

            if ( msg.execute() != StdButton_Yes )
               continue;
         }
#endif
         // Perform batch file format conversion and quit.
         console.show();
         console.abortEnabled = true;
         engine.convertFiles();
      }

      break;
   }
}

main();

I have tested it thoroughly; however, as is advisable for all scripts that write disk files, please be careful and test it first with temporary copies of your images.

I have added an additional "security check" to ensure that if you omit an explicit output directory, you actually know what your are doing. This can be disabled by #defining WARN_ON_NO_OUTPUT_DIRECTORY as 0.

Output files are written with the current default settings for the specified output format (which is selected with the output file extension). These settings (e.g., JPEG quality, or TIFF compression, etc) can be changed with the Format Explorer window (select the desired format on the left panel and click Edit Preferences on the right panel).

Please feel free to play with this script and improve it as you want. For example, it could be nice to add an option to resample output images by specifying a scaling factor, or a target size in pixels for their largest dimensions (to generate thumbnails).

Enjoy it

[Juan Conejero]

It would be *really* sweet if PCL can record a user interaction like Excel and other office applications do. Ok, I can dream, right?

It can do much more than that, in fact. The processing history of any image can be extracted and reused with virtually no limits. Please do the following test:

* Open an image

* Apply a couple processes to it. For example, Invert it, and apply a simple HistogramTransform (or anything you want).

* Right-click on the image and select "Load History Explorer"

* On the History Explorer window, you can see the whole sequence of applied processes.

* Note that when you select an individual process on the History Explorer list:


- The right panel is loaded with an automatically generated script that can be copied/pasted to reuse it directly in your own script code.


- You can double-click on the '#' item to return the image to the corresponding state, navigating throughout the processing history of the image arbitrarily


- You can double-click on the process identifier ('Proc Id' item) to launch the corresponding interface loaded with the parameters that were originally applied to the image.


- You can drag the process to any image to apply it.


- You can drag the New Instance button of the History Explorer window and drop it on any image to apply the whole sequence. Or you can drop it on the workspace to create a ProcessContainer icon.


- Once you have a ProcessContainer icon generated as explained above, double-click it to open the ProcessContainer interface. On this interface, if you select the <Root> item, a script will be generated that can be copied/pasted to reuse the whole processing in your own scripts. Or you can drag/drop the New Instance button on images, ...

This is how PixInsight's object-oriented arquitecture allows you to manage processes and images independently. Oh, and yes, PixInsight is to dream :wink:

[Nocturnal]

Hello everyone,

thanks for all the informative replies. I'll be quite occupied with that for a while :-)

I think it's great that PCL is so flexible.

Best,

   Sander

[Nocturnal]

The script works great Juan!

Thanks very much,

  Sander

[Juan Conejero]

My pleasure, Sander. Hope it will be useful for you and others.

[Nocturnal]

I think it should be included in the standard scripts supplied with PCL. It's a great example of batch processing with a small GUI that can be adapted to other needs. It's interesting that most code goes into building the GUI. I would probably start by writing interactive scripts that ask the user questions instead of building a GUI. But the GUI is very nice.

  Best,

     Sander