PixInsight Forum (historical)
Software Development => New Scripts and Modules => Topic started by: NKV on 2011 April 19 05:10:09
-
Hi,
In attachment script for build sequential FITS utility. For example: extract "CCD-TEMP" from FITS Header and write it to filename. Or create subdirectories according FITSKeyword.value and put files with similar value to good place for it.
Now it's just skeleton. Script accumulate all FITS Header keyword for all file and show it on the screen. But if you have any idea, welcome!
Just say what you want to see in next version.
Best regards,
Nikolay.
PS FITS engine it stole. Juan thanks and sorry for.
-
Ver.0.02
+move files to subdir according selected FITS KeyWords.
-
Thank you, Nikolay.
Looks interesting.
I will take a look and report back later.
HP
-
first impression: this script works fine on a Mac.
-
Ver 0.03
+Button: Collect all files from sub folders.
+Button: Close selected files.
+DoubleClick: Invert "Checkmark" of selected files.
-
Thanks Nikolay, just what I have been looking for to get a text output of selected FITS keyword entries.
much appreciated, -carl
-
text output of selected FITS keyword entries.
Done
Ver 0.04
+button: Copy
+button: Generate text file of selected keywords
Best regards,
Nikolay.
-
Ver 0.05 Bug fix.
Hello All!
I am try to add keyword sorting to the script. I need it to manage sub folder creation procedure.
As you can see, possible to sort ThreeBox Header ( Use mouse push-move-drop to change column position in GUI ). But, i not understand how to get by JavaScript the result of GUI ThreeBox manipulation.
I mean: TreeBox.headerText( column ) ignore GUI manipulation. How to get real on screen GUI condition? Real column position?
Best regards,
Nikolay.
-
I see only ver 05
I can add a file or folder on a Windows 64
Max
-
I can add a file or folder on a Windows 64
Why not?
-
Sorry,
I messed up that post.
I can not select any files or folders.
The script starts but won't doing anything from there on.
I tried the last version posted.
It is Window 7 64 system.
Thanks
Max
-
I can not select any files or folders.
The script starts but won't doing anything from there on.
Strange...
I have no problem. See PrintScreen.
-
I should just be able to add file with the + icon. I will reboot and try again when I get home.
Max
-
when I try to download the script I get an "error of compilation sript" ... in the first line it seems
-
I checked again on another windows 64 cpu.
No files appear in the list box after selection.
Max
-
when I try to download the script I get an "error of compilation sript" ... in the first line it seems
I think it's Internet Browser problem. Be sure that you get FITSkey_0.05.js
Sometimes i getting .php file instead real attachment.
Any way FITSkey_0.06.js code below. You can copy the code to PI script editor and run.
#feature-id Utilities > FITS Keywords
#define VERSION 0.06
#include <pjsr/DataType.jsh>
#include <pjsr/Sizer.jsh>
//#include <pjsr/FrameStyle.jsh>
#include <pjsr/TextAlign.jsh>
#define DEBUGGING_MODE_ON false
function copyFile( sourceFilePath, targetFilePath )
{
var f = new File;
f.openForReading( sourceFilePath );
var buffer = f.read( DataType_ByteArray, f.size );
f.close();
f.createForWriting( targetFilePath );
f.write( buffer );
//f.flush(); // optional; remove if immediate writing is not required
f.close();
}
function trim( s )
{
return s.replace( /^\s*|\s*$/g, '' );
}
function LoadFITSKeywords( fitsFilePath )
{
function searchCommentSeparator( b )
{
var inString = false;
for ( var i = 9; i < 80; ++i )
switch ( b.at( i ) )
{
case 39: // single quote
inString ^= true;
break;
case 47: // slash
if ( !inString )
return i;
break;
}
return -1;
}
var f = new File;
f.openForReading( fitsFilePath );
var keywords = new Array;
for ( ;; )
{
var rawData = f.read( DataType_ByteArray, 80 );
var name = rawData.toString( 0, 8 );
if ( name.toUpperCase() == "END " ) // end of HDU keyword list?
break;
if ( f.isEOF )
throw new Error( "Unexpected end of file: " + fitsFilePath );
var value;
var comment;
if ( rawData.at( 8 ) == 61 ) // value separator (an equal sign at byte 8) present?
{
// This is a valued keyword
var cmtPos = searchCommentSeparator( rawData ); // find comment separator slash
if ( cmtPos < 0 ) // no comment separator?
cmtPos = 80;
value = rawData.toString( 9, cmtPos-9 ); // value substring
if ( cmtPos < 80 )
comment = rawData.toString( cmtPos+1, 80-cmtPos-1 ); // comment substring
else
comment = new String;
}
else
{
// No value in this keyword
value = new String;
comment = rawData.toString( 8, 80-8 );
}
// Perform a naive sanity check: a valid FITS file must begin with a SIMPLE=T keyword.
if ( keywords.length == 0 )
if ( name != "SIMPLE " && trim( value ) != 'T' )
throw new Error( "File does not seem a valid FITS file: " + fitsFilePath );
// Add new keyword. Note: use FITSKeyword with PI >= 1.6.1
keywords.push( new FITSKeyword( name.toString(), value.toString(), comment.toString() ) );
}
f.close();
return keywords;
}
//----------------------------------------------------------------------------
//----------------------------------------------------------------------------
function MyDialog()
{
this.__base__ = Dialog;
this.__base__();
this.inputFiles = new Array(); //Array of filename with full path
this.inputKeys = new Array(); //keys of all files. Big RAW array of all file keys.
this.keyTable = new Array(); //accumulated names of keywords from all files
this.keyEnabled = new Array(); //true == selected keywords
this.defaultKey = new Array("SET-TEMP","EXPOSURE","IMAGETYP");
var outputDirectory = "";
this.engine_mode = 0; //0=move, 1=copy
if (DEBUGGING_MODE_ON) var outputDirectory = "D:/temp";
//------------------------------------------------------------
this.onShow = function()
{
// this.filesAdd_Button.onClick();
}
//hide columns of unchecked keywords---------------------------
this.hideKey = function ()
{
//if (DEBUGGING_MODE_ON) console.clear();
for (var i in this.keyEnabled)
{
c = parseInt(i) + 1;
if (DEBUGGING_MODE_ON)
console.write("Column: " + c + " enabled: " + this.keyEnabled[i] + " " );
this.files_TreeBox.showColumn( c, this.keyEnabled[i]);
}
}
//----------------------------------------------------------------------------------
// KeyWord Dialog
this.SD = new KeyDialog( this );
this.Key_button = new ToolButton( this );
with ( this.Key_button )
{
icon = new Bitmap( ":/images/icons/text.png" );
toolTip = "KeyWord Dialog";
onClick = function()
{
if (this.dialog.keyTable.length)
{
this.dialog.SD.execute();
this.dialog.hideKey();
}
}
}
//----------------------------------------------------------
// File List TreeBox
this.files_TreeBox = new TreeBox( this );
with ( this.files_TreeBox )
{
rootDecoration = false;
numberOfColumns = 1;
multipleSelection = true;
headerVisible = true;
headerSorting = true;
setHeaderText(0, "Filename");
onNodeUpdated = function( node, column ) // Invert CheckMark
{
for (var i=0; i < this.selectedNodes.length; i++)
{
if ( node == this.selectedNodes[i] ) continue; // skip curent clicked node, because it will inverted automaticaly
this.selectedNodes[i].checked = !this.selectedNodes[i].checked;
}
}
}
//----------------------------------------------------------
this.UpdateTreeBox = function ()
{
this.files_TreeBox.clear();
this.keyTable = new Array(); // clear
// Accumulate all KeyNane in keyTabe
for ( var i in this.inputFiles)
{
var key = this.inputKeys[i]; // keywords of one file
var node = new TreeBoxNode( this.files_TreeBox );
node.setText( 0, this.inputFiles[i] ); //write name to first coummn
node.checked = true;
for ( var j in key )
{
var name = key[j].name; //name of Keyword from file
var k = this.keyTable.indexOf(name);// find index of "name" in keyTable
if (k < 0) // new keyName
{
this.keyTable.push(name);//add keyword name to table
this.files_TreeBox.numberOfColumns++;// add new column
this.files_TreeBox.setHeaderText(this.keyTable.length, name);//set name of new column
k = this.keyTable.length-1;
this.keyEnabled[k] = (this.defaultKey.indexOf(name)> -1);//compare with defauld enabled keywords
}
if (key[j].isNumeric) node.setText( k+1, Number(key[j].value).toFixed(3) );
else node.setText( k+1, key[j].value );
}
}
this.hideKey(); //hide columns of unchecked keywords
}
//---------------------------------------------------------------------------------------
this.getFiles = function (fileNames)
{
if (DEBUGGING_MODE_ON)
{
console.writeln("Found "+fileNames.length);
console.write("Checked:.");
//for ( var b=0; b<fileNames.length.toString().length; b++)
// console.write(".");
}
var qtyNew = 0;
for ( var i in fileNames )
{
if (DEBUGGING_MODE_ON)
{
for ( var b=0; b<i.toString().length; b++)
console.write("\b");
console.write(parseInt(i)+1); processEvents();
}
if (this.inputFiles.indexOf(fileNames[i]) < 0) //Add file only one times
{
var key = LoadFITSKeywords(fileNames[i]);
this.inputFiles.push(fileNames[i]);
this.inputKeys.push(key);
qtyNew++;
}
}
console.writeln(" ");
if (qtyNew == 0) {console.writeln("No new files"); return;}
if (DEBUGGING_MODE_ON) {console.writeln("New ",qtyNew,"\nTotal ",this.inputFiles.length); processEvents();}
this.UpdateTreeBox();
this.QTY.text = "Total files: " + this.inputFiles.length;
this.setMinWidth(800);
this.adjustToContents();
this.dialog.update();
}
// Total file Label ---------------------------------------------------------------------------
this.QTY = new Label( this );
this.QTY.textAlignment = TextAlign_Right|TextAlign_VertCenter;
//enable/disable buttons
this.update = function()
{
var enabled = !((!this.inputFiles.length) || (!outputDirectory));
this.dialog.move_Button.enabled = enabled;
this.dialog.copy_Button.enabled = enabled;
this.dialog.txt_Button.enabled = enabled;
}
// Add files ---------------------------------------------------------------------------
this.filesAdd_Button = new ToolButton( this );
with ( this.filesAdd_Button )
{
icon = new Bitmap( ":/images/image_container/add_files.png" );
toolTip = "Add files";
onClick = function()
{
var ofd = new OpenFileDialog;
ofd.multipleSelections = true;
ofd.caption = "Select FITS Files";
ofd.filters = [["FITS Files", "*.fit", "*.fits", "*.fts"]];
if ( ofd.execute() ) this.dialog.getFiles(ofd.fileNames);
}
}
// Add Dir ---------------------------------------------------------------------------
this.dirAdd_Button = new ToolButton( this );
with ( this.dirAdd_Button )
{
icon = new Bitmap( ":/images/icons/folders.png" );
toolTip = "Add folder including subfolders";
onClick = function()
{
var gdd = new GetDirectoryDialog;
//gdd.initialPath = outputDirectory;
gdd.caption = "Select Input Directory";
if ( gdd.execute() )
{
if (DEBUGGING_MODE_ON) {console.writeln("Start serching FITS file in SubFolders"); processEvents();}
var fileNames = searchDirectory(gdd.directory+"/*.fit" ,true)
.concat(searchDirectory(gdd.directory+"/*.fits",true))
.concat(searchDirectory(gdd.directory+"/*.fts",true));
if (DEBUGGING_MODE_ON) {console.writeln("Finish serching FITS file in SubFolders"); processEvents();}
this.dialog.getFiles(fileNames);
}
}
}
// Close selected files ---------------------------------------------------------------------------
this.files_close_Button = new ToolButton( this );
with ( this.files_close_Button )
{
icon = new Bitmap( ":/images/close.png" );
toolTip = "<p>Close selected images.</p>";
onClick = function()
{
if ( this.dialog.inputFiles.length == 0 ) return;
for ( var i = this.dialog.files_TreeBox.numberOfChildren; --i >= 0; )
{
if ( this.dialog.files_TreeBox.child( i ).selected )
{
this.dialog.inputFiles.splice(i,1);
this.dialog.inputKeys.splice(i,1);
this.dialog.files_TreeBox.remove( i );
}
}
this.dialog.QTY.text = "Total files: " + this.dialog.inputFiles.length;
this.dialog.update();
}
}
//Output Dir --------------------------------------------------------------------------------------
this.outputDir_Edit = new Edit( this );
this.outputDir_Edit.readOnly = true;
this.outputDir_Edit.text = outputDirectory;
this.outputDir_Edit.toolTip ="select output directory.";
this.outputDirSelect_Button = new ToolButton( this );
with ( this.outputDirSelect_Button )
{
icon = new Bitmap( ":/images/icons/select.png" );
toolTip = "Select output directory";
onClick = function()
{
var gdd = new GetDirectoryDialog;
gdd.initialPath = outputDirectory;
gdd.caption = "Select Output Directory";
if ( gdd.execute() )
{
outputDirectory = gdd.directory;
this.dialog.outputDir_Edit.text = outputDirectory;
this.dialog.update();
}
}
}
//engine----------------------------------------------------------------------------
this.apply = function ()
{
var targetFiles = new Array();
for ( var i in this.inputFiles)
targetFiles[i] = outputDirectory;
/*
for ( var i in this.keyTable) // in all possible keywords
{
if (!this.keyEnabled[i]) continue; // ignore disabled keywords
var name = this.keyTable[i]; // keyword name
for ( var j in this.inputFiles) // for in all files
{
var key = this.inputKeys[j]; // key = all keywords from curent file
for (var k in key) // for in all keywords of the file
{
if (!(key[k].name == name)) continue; // skip not equal keyword name
// keyword found in the file >> extract value
if (key[k].isNumeric)
var value = parseFloat(key[k].value)
else
{
var value = key[k].value;
value = value.replace( /'/g, "" );
value = value.replace( / /g, "" ); // delete left space
value = value.replace( /:/g, "." );
}
// use keyword value to make subDir name
targetFiles[j] = targetFiles[j] + "/" + value;
break; // next file
}
}
}
*/
for ( var i = 0; i < this.files_TreeBox.numberOfColumns; i++ )
{
var name = this.files_TreeBox.headerText( i ); // extract keyword name from header files_TreeBox
if ( name == "Filename" ) continue; // skip Filename column
if ( !this.files_TreeBox.isColumnVisible( i ) ) continue; // skip hiden column
if (DEBUGGING_MODE_ON) console.writeln("Column name: " + name );
for ( var j in this.inputFiles) // for in all files
{
var key = this.inputKeys[j]; // key = all keywords from curent file
for (var k in key) // for in all keywords of the file
{
if (!(key[k].name == name)) continue; // skip not equal keyword name
// keyword found in the file >> extract value
if (key[k].isNumeric)
var value = parseFloat(key[k].value)
else
{
var value = key[k].value;
value = value.replace( /'/g, "" );
value = value.replace( / /g, "" ); // delete left space
value = value.replace( /:/g, "." );
}
// use keyword value to make subDir name
targetFiles[j] = targetFiles[j] + "/" + value;
break; // next file
}
}
}
var skip = 0;
for ( var i in targetFiles)
{
if ( !this.files_TreeBox.child(parseInt(i)).checked ) { skip++; continue; }
var fileDir = targetFiles[i];
if (DEBUGGING_MODE_ON) console.writeln("Create fileDir: " + fileDir );
if (!File.directoryExists(fileDir)) File.createDirectory(fileDir);
var s = this.inputFiles[i];
var t = fileDir+"/"+File.extractName(s)+File.extractExtension(s);
if ( File.exists( t ) )
for ( var u = 1; ; ++u )
{
var tryFilePath = File.appendToName( t, '_' + u.toString() );
if ( !File.exists( tryFilePath ) ) { t = tryFilePath; break; }
}
if (this.engine_mode==0)
{
console.writeln("move ", s," to ",t);
File.move(s,t);
}
else
{
console.writeln("copy ", s," to ",t);
copyFile(s,t);
}
processEvents();
}
console.writeln("Total files: ", targetFiles.length,"; Skiped: ",skip,"; Processed: ",targetFiles.length-skip);
}
//Engine buttons --------------------------------------------------------------------------------------
this.move_Button = new PushButton( this );
with ( this.move_Button )
{
text = "Move";
toolTip = "Move Checked files to output directory";
enabled = false;
onClick = function()
{
parent.apply();
this.dialog.ok();
}
}
this.copy_Button = new PushButton( this );
with ( this.copy_Button )
{
text = "Copy";
toolTip = "Copy Checked files to output directory";
enabled = false;
onClick = function()
{
parent.engine_mode = 1;
parent.apply();
}
}
this.txt_Button = new PushButton( this );
with ( this.txt_Button )
{
text = "FITS.txt";
toolTip = "For Checked files wtrite FitKeywords value to file FITS.txt in output directory";
enabled = false;
onClick = function()
{
var tab = String.fromCharCode(9);
var f = new File();
var fileName = "FITS_keys";
var fileDir = outputDirectory;
var t = fileDir + "/" + fileName + ".txt";
if ( File.exists( t ) )
{
for ( var u = 1; ; ++u )
{
for( var n = u.toString(); n.length < 4 ; n = "0" + n);
var tryFilePath = File.appendToName( t, '-' + n );
if ( !File.exists( tryFilePath ) ) { t = tryFilePath; break; }
}
}
f.create(t);
for ( var i in parent.keyTable)//output header
{
if (!parent.keyEnabled[i]) continue;
f.outTextLn(parent.keyTable[i]+tab);
}
f.outTextLn("Filename"+String.fromCharCode(10,13));
var skip = 0;
for ( var j in parent.inputFiles)//output FITS data
{
if ( !parent.files_TreeBox.child(parseInt(j)).checked ) { skip++; continue; }
var key = parent.inputKeys[j];
for ( var i in parent.keyTable)
{
if (!parent.keyEnabled[i]) continue;
var name = parent.keyTable[i];
for (var k in key)
{
if (!(key[k].name == name)) continue;
if (key[k].isNumeric)
var value = parseFloat(key[k].value)
else
{
var value = key[k].value;
value = value.replace( /'/g, "" );
value = value.replace( / /g, "" ); //delete left space
value = value.replace( /:/g, "." );
}
f.outText(value.toString());
for (var w = value.toString().length; w < 8; w++) f.outText(" ");
f.outText(tab);
k=-1;
break;
}
if (k > -1) f.outText(" "+tab);
}
f.outTextLn(parent.inputFiles[j]+String.fromCharCode(10,13));
}
f.close();
console.writeln("FITSKeywords saved to ",t);
}
}
//Sizer------------------------------------------------------------
this.fileButonSizer = new HorizontalSizer;
with ( this.fileButonSizer )
{
margin = 6;
spacing = 4;
add( this.Key_button );
add( this.filesAdd_Button );
add( this.dirAdd_Button );
add( this.files_close_Button );
add( this.QTY );
addStretch();
}
this.inputFiles_GroupBox = new GroupBox( this );
with (this.inputFiles_GroupBox)
{
title = "Input";
sizer = new VerticalSizer;
sizer.margin = 6;
sizer.spacing = 4;
sizer.add( this.files_TreeBox,100 );
sizer.add( this.fileButonSizer );
}
this.outputDir_GroupBox = new GroupBox( this );
with (this.outputDir_GroupBox)
{
title = "Output";
sizer = new HorizontalSizer;
sizer.margin = 6;
sizer.spacing = 4;
sizer.add( this.outputDir_Edit, 100 );
sizer.add( this.outputDirSelect_Button );
}
this.sizer2 = new HorizontalSizer;
with ( this.sizer2 )
{
spacing = 2;
add( this.move_Button);
add( this.copy_Button);
add( this.txt_Button);
addStretch();
}
this.sizer = new VerticalSizer;
with ( this.sizer )
{
margin = 2;
spacing = 2;
add( this.inputFiles_GroupBox );
add( this.outputDir_GroupBox );
add( this.sizer2 );
}
//this.move(50,100); // move dialog to up-left corner
}
//End if Main Dialog------------------------------------------------------------
//Second Dialog------------------------------------------------------------
function KeyDialog( pd ) //pd - parentDialog
{
this.__base__ = Dialog;
this.__base__();
this.windowTitle = "Select KeyWords";
this.onShow = function()
{
var p = new Point( pd.position );
p.moveBy( 16,16 );
this.position = p;
this.keyword_TreeBox.clear();
for (i in pd.keyTable)
{
var node = new TreeBoxNode(this.keyword_TreeBox);
node.setText( 0, pd.keyTable[i] );
node.checked = pd.keyEnabled[i];
}
this.file_ComboBox.clear();
for (i in pd.inputFiles)
this.file_ComboBox.addItem(pd.inputFiles[i]);
this.file_ComboBox.onItemSelected(0);
this.setMinSize(400,600);
}
this.onHide = function()
{
for (var i in pd.keyTable)
{
checked = this.keyword_TreeBox.child( parseInt(i) ).checked;
if (DEBUGGING_MODE_ON)
console.write("Key#: " + parseInt(i) + " checked: " + checked );
pd.keyEnabled[i] = checked;
}
pd.setMinWidth(800);
}
this.file_ComboBox = new ComboBox( this );
with ( this.file_ComboBox )
{
onItemSelected = function( index )
{
for ( var i in pd.keyTable)
parent.keyword_TreeBox.child(parseInt(i)).setText(1,pd.files_TreeBox.child(index).text(parseInt(i)+1));
}
}
//----------------------------------------------------------
// FITS keyword List TreeBox
this.keyword_TreeBox = new TreeBox( this );
with ( this.keyword_TreeBox )
{
toolTip = "Checkmark to include to report";
rootDecoration = false;
numberOfColumns = 2;
setHeaderText(0, "name");
setHeaderText(1, "value");
setColumnWidth(0,100);
setColumnWidth(1,200);
}
this.sizer = new VerticalSizer;
this.sizer.margin = 4;
this.sizer.spacing = 4;
this.sizer.add( this.file_ComboBox );
this.sizer.add( this.keyword_TreeBox );
this.adjustToContents();
}
MyDialog.prototype = new Dialog;
KeyDialog.prototype = new Dialog;
var dialog = new MyDialog;
dialog.execute();
-
No files appear in the list box after selection.
Max, look at line #9 in the script:
#define DEBUGGING_MODE_ON false(Note: fast way to open PI ScriptEditor is Ctrl+Alt+E)
Change "false" to "true" and press F9. You will see many information in Console. I hope it's can help to find the bug.
Nikolay
-
Ver 0.06
+Sorting
+more toolTip
Bag fix of Check-mark invert.
-
Thanks Nicolay! Here is all the information that we can't get when we open files with Windows. The *.fit files are nearly unknow for this OS, and we open frecuently files that we don't want and conversely, because we don't know the due properties of the files.
I want to ask you if is possible to go further with the script and to make it like an "open files" menu >:D
-
an "open files" menu
Sorry, what are you mean?
-
I'm sorry; what I intend to say is that would be interesting if we can browse our files with the data of your script, and have the facility of open it in PI (like an "open file" menu); so would be possible to select the apropiate files for our work, due to the information that the script provides. Now I select and open my files blindly, without to know their time exposure, filter, binning ..., because these data are unknow for the explorer window of Window(that provides the "open file" function).
Thanks!
-
You need file browser with FitsKeyWords support? Right?
It's good idea, but which FitsKeyWords you want to add to browser? (Size,bin,exposure,filter,..) How many KeyWords?
Anyway it's other story for wish list http://pixinsight.com/forum/index.php?board=9.0
I try to write script which help to sorting (move or rename) files after long session. Light to "Light" folder, Dark to "Dark" folder... etc.
Welcome if you have any idea.
-
Yes, I think that are interesting the important data for any processing task, like date, time, object, timeExp, ImageTyp, Filter, temp, Binning, ... If we can know these properties we can decide better what files are interesting in each case. In any case, Thanks!!
-
Maybe this is what you need http://astroshed.com/software.html. Not exactly cheap, but...
Georg
-
Cool app,
I got it to work fine. Probable user error before. :surprised:
It is great to have utilily the allows image sorting based on the fits headers.
Anything you add to it would be great.
-------------------------------------------------------
Thought for features.
Search by keyword and give results as a report rather than moving files only.
See contents of sub directories too.
Flexibilty on how to sort files in to folders... ie choose folder hierarchy bases on the FITs Keys.
Abilty to delete files of folders directly.
Count files
Total exposure time for each each type BIN, FILTER etc.
Data base management a file manipulations are such a headache :'(
Tools like this really help
This one helped me a lot already.
Thanks
Max