Split fits files

[avdhoeven]

I wondered if the following is somehow possible in pixinsight. I'm working on some professional data that has been made with a 36ccd camera. The fits files contain 36 layers. Now I want to split these files and save them separately where they are named with the layer numbers. I can open a file in pixinsight and do get 36 images. But now I have to save them by hand and rename them all, because they get the original filename. As I have 30 of these files it would mean I have to it over 1000 times. So I wondered if there is some function like this implemented, or could this be added?

Thanks!

[mschuster]

Lacking anything else, a script could do this. But what are you going to do with the 1000 images next? IMO PI isn't set up well for batch processing data sets.

Mike

[avdhoeven]

I already have a workflow ready for the data. I will do it in parts and then integrate the mosaic. Problem is that I'm not that deep into the scripting yet... So now I started redownloading, but then in separate files. It's possible, but another 10Gb of data...

[Juan Conejero]

This is the kind of problem that requires scripting. Please understand that we cannot provide implementations for all possible batch tasks adapted to all possible scenarios and complex problems. PixInsight has a powerful scripting engine that should be used to solve these problems.

[Juan Conejero]

IMO PI isn't set up well for batch processing data sets.

While I'm sure you have your good reasons to say this, which I respect of course, I disagree. We still have to evolve and improve in many aspects, but I think we have a reasonably powerful scripting engine.

If processing 400,000+ images is a good example of batch processing, ask Vicent Peris how he implemented his video analysis tasks in PixInsight when he worked for the Spanish National Police.

[avdhoeven]

I fully understand that not all functions can be implemented, but I thought that maybe a split layers function could be interesting for many more people. The only thing that should happen is to split the layers to separate files. I can go and try to see if I can do this with scripting, but this will for now just take too much time as I do not know the scripting language well enough yet. I hope that maybe somebody with experience in this can give an idea how to do this?

[IanL]

It is easy enough to do if you understand a bit about scripting in PI, but it would take a fair bit of effort to learn if you don't.

I have a script I might be able to adapt but it will take me a couple of days to do due to work commitments.

[bitli]

I could test if I can adapt one of my scripts, but would it be possible to have one image for testing (you may put it on endor, drop box or similar and maybe post the url or send it by PM to the 'volunteers'  for implementation.
-- bitli

[bitli]

Well, here is a first attempt. You can copy the text and paste it to the script editor and execute it (F9) AFTER adapting the parameters at the start.  Check first with a single file.  You can also save it to a file with an extension of js and execute it from the menu.

NOTE:  It is possible that the whole file be read in memory with this simple approach, this may or may not work depending on the size of your files and memory. The generation of fiel name may or may not work with your files.

-- bitli

Code: [Select]

// Extract all images of a multi image fits file,
// Adapt the directories and extension as needed,
// all files in the source directory will be processed and each image
// in each source file will be extracted to the destination directory.
// It is recommended that the destination directory is empty.
// The name will be formed from the name of the source file and ther EXTNAME or,
// if missing, '_part_#'.  There is NO check that the generated file name is valid
// or unique.  AskONOverwrite will alert you in case of duplicates.


// PROVIDED AS IS - NOT REALLY TESTED, USE AT YOUR OWN RISK.

"use strict";

#include <pjsr/UndoFlag.jsh>

var srcDir = "D:/temp/source";
var dstDir = "D:/temp/dest";
var extension =".fit";
var askOnOverwrite = true;


function copyImages(fileName) {
   var sourceFilePath = srcDir + "/" + find.name;

   var images = ImageWindow.open( sourceFilePath, "temp", true );
   Console.writeln("Loaded " + images.length + " images");

   for (var i=0; i<images.length;i++) {
      Console.writeln("  Loading image " + i + " of '" + fileName + "'");
 
      var image = images[i];
      var keywords = image.keywords;

      var extName = "part_" + (i+1);
       
      Console.writeln("    " + keywords.length + " keywords:");
      for (var j=0; j<keywords.length; j++) {
         if (keywords[j].name === 'EXTNAME') extName = keywords[j].strippedValue;
         Console.writeln("      " + keywords[j]);
      }

     
      var newFileName = File.extractName(fileName)+"_"+extName+extension;
      Console.writeln("    new name: '" + newFileName +"'");

      var targetFilePath = dstDir + "/" + newFileName;
      //  path,queryOptions,allowMessages,strict,verifyOverwrite
      image.saveAs(targetFilePath,  false, false, false, askOnOverwrite);

      image.forceClose();
   }
}


var find = new FileFind;


if ( find.begin( srcDir +"/*" + extension) ) {
   do
   {
      if ( find.name != "." && find.name != ".." && find.isFile ) {
        Console.writeln("Expanding images in " + find.name);
        copyImages(find.name);
      }
   } while ( find.next() );

}

Console.writeln("Completed");



[avdhoeven]

Sorry I didn't reply to this post. Somehow I never got a notification there were new replies Thanks a lot, I'll try it out!

[avdhoeven]

Just tested it and it works great!

[bitli]

Great, thanks for the feedback
-- bitli