Getting a list of directories and files in the File system java script

[Dvelledge]

How can you get a list of files and / or directories if you provide a path. I see how we can use OpenFileDialog to let people choose files but I want to just go through a list of directories and pull in the files from each directory to do basic calibration for on a nightly basis.  The observatory is completely automated and stories files, darks bias, flats in well defined folders.  it also stores lights in folder based on the target name.  I'm using the engine of the batch preprocessing script but want to just go through the list of files in directories we have taken that night and process. I don't see the commands that would be similar to the node fs functionality.

thanks

[Juan Conejero]

You have basically two options. The simplest one is the following method:

Array searchDirectory( String dirPath[, Boolean recursive=false] )

This method is a property of the Global object in the PixInsight JavaScript Runtime (PJSR). dirPath should be a wildcard pattern to find files on an existing directory in the local filesystem, such as "/foo/bar/*.jpg" or "C:/foo/bar/*.txt". Note that UNIX directory separators '/' are always used in PixInsight on all platforms, including Windows. If recursive is true, the file search operation will be recursive on the entire directory tree rooted at dirPath.

Here is a real-world working example, excerpted from one of our code maintenance scripts:

Code: [Select]

function FindFiles( baseDirectory, templates, recursive )
{
   if ( !baseDirectory.endsWith( '/' ) )
      baseDirectory += '/';

   if ( templates === undefined )
      templates = ['*'];
   else if ( !(templates instanceof Array) )
      templates = [templates];
     
   if ( recursive === undefined )
      recursive = true;
     
   let files = [];
   for ( let i = 0; i < templates.length; ++i )
      files.push( searchDirectory( baseDirectory + templates[i], recursive ) );
   return files;
}
This function will return a list of full paths for all existing files on baseDirectory whose names match one of the specified templates, optionally recursing the entire tree rooted at baseDirectory. For example, you can call this function as follows:

FindFiles( "/foo/bar", ["*.png", "*.jpg"] );

and it will return an Array object with a list of full paths for all existing PNG and JPEG files under the /foo/bar directory. Note that the search is recursive by default. Since we have passed just two arguments to FindFiles() in this example, the returned array will contain all existing files in the entire subtree rooted at /foo/bar.

This function is quite versatile. You can call it with just a string template parameter instead of an array:

FindFiles( "/foo/bar", "*.png" );

The Global.searchDirectory() method is easy to use. It is probably the preferred method if you want to get just a list of file names. However, if you need to gather more information about individual directory entries, and/or if you need to gain control on each iteration of the directory search operation, you have to use the FileFind object in PJSR. Here is another example borrowed from a working script:

Code: [Select]

function FindFileData_Recursive( data, baseDirectory, template, recursive )
{
   let find = new FileFind;
   if ( find.begin( baseDirectory + template ) )
      do
         if ( find.isFile )
            data.push( { path:         baseDirectory+find.name,
                         size:         find.size,
                         created:      find.created,
                         lastModified: find.lastModified } );
      while ( find.next() );

   if ( recursive )
   {
      let directories = [];
      if ( find.begin( baseDirectory + "*" ) )
         do
            if ( find.isDirectory )
               if ( find.name != "." )
                  if ( find.name != ".." )
                     directories.push( baseDirectory + find.name + '/' );
         while ( find.next() );

      for ( let i = 0; i < directories.length; ++i )
         FindFileData_Recursive( data, directories[i], template, recursive );
   }
}

function FindFileData( baseDirectory, templates, recursive )
{
   if ( !baseDirectory.endsWith( '/' ) )
      baseDirectory += '/';

   if ( templates === undefined )
      templates = ['*'];
   else if ( !(templates instanceof Array) )
      templates = [templates];
     
   if ( recursive === undefined )
      recursive = true;
     
   let data = [];
   for ( let i = 0; i < templates.length; ++i )
      FindFileData_Recursive( data, baseDirectory, templates[i], recursive );
   return data;
}

Parameter wise, the FindFileData() method works exactly the same as the previous FindFiles(). However, FindFileData() returns an array of objects with several properties for each found file. For simplicity, I have limited the data provided in this example to full file path, file size in bytes, date of file creation, and date of last modification. There are however much more informative items available in the FileFind object; see it on the Object Explorer window for more information.

Here is an example of the above FindFileData() in action. With the help of two auxiliary functions, we can implement something similar to an ls command (or a DIR command on Windows/MS-DOS) very easily:

Code: [Select]

function sizeAsString( bytes, precision )
{
   if ( precision === undefined )
      precision = 3;
   const kb = 1024;
   const mb = 1024 * kb;
   const gb = 1024 * mb;
   const tb = 1024 * gb;
   if ( bytes >= tb )
      return format( "%.*f TiB", precision, bytes/tb );
   if ( bytes >= gb )
      return format( "%.*f GiB", precision, bytes/gb );
   if ( bytes >= mb )
      return format( "%.*f MiB", precision, bytes/mb );
   if ( bytes >= kb )
      return format( "%.*f KiB", precision, bytes/kb );
   return format( "%lld B", bytes );
}

function fileDateAsString( date )
{
   return format( "%4d/%02d/%02d %02d:%02d:%02.0f",
                  date.getFullYear(), date.getMonth()+1, date.getDate(),
                  date.getHours(), date.getMinutes(), date.getSeconds() );
}

function FindFileData_Verbose( baseDirectory, templates, recursive )
{
   let data = FindFileData( baseDirectory, templates, recursive );
   for ( let i = 0; i < data.length; ++i )
      console.writeln( format( "%16s", sizeAsString( data[i].size ) ), "  ",
                       fileDateAsString( data[i].lastModified ), "  ",
                       data[i].path );
}

For each found file, The FindFileData_Verbose() function prints its size in a human readable format, the date and time of last file modification, and its full path on the filesystem. This can be easily extended to obtain much more useful information such as file owners, file attributes, etc.

To complete this review, if you need thorough information for each file, you can use the FileInfo object in PJSR. You can combine FileInfo with the searchDirectory() global function to add exhaustive filesystem search functionality to your scripts in PixInsight.

If you want to make quick tests with the functions I have posted here, you can use the Process Console window. Copy the source code above to a new .js file created with Script Editor and execute the code to create the functions in the JavaScript runtime (select Execute > Compile & Run on Script Editor, or press F9 (^R on macOS)). Then you can issue a command such as:

j FindFileData_Verbose( "/home/johndoe/Pictures/", "*.png" )

to get a list of files directly on the console. Let me know if this information is useful for what you want to do

[Dvelledge]

Wow,  Thank you very much.  That is the most complete response to any coding question that I have ever received.

Don