FITSFileManager

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
I have significantly modified the FITS_key that copy files based on FITS keywords, it was written by of Nicolay (I hope he doesn't mind).
FITS File manager support generating a file path from the FITS keyword in a flexible way (using a pattern). The file path can include commonly used FITS keywords, parts of the original file name and a count which can be based on selected elements (typically the target directory, but could be the filter or other variables).

The script is attached. This scripts move and copy files, it is potentially dangerous, use at your own risk (however I use it on my machine).
Preferably use load/saveas so that the orignal name of the file is saved as a fits keyword.

Thanks to Nicolay for the original contribution. All bugs are mine.
There is an Help included (button at bottom right)

[EDITED Feb 10, 2013 - Version 1.0 attached for PI 1.7, should be part of an update in version 1.8]

-- bitli
 

Attachments

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
Version 0.6 released, with many enhancements. First post updated.

This script can help sorting out FITS files, especially the ones received from remote observatories or recorded with a less than optimal file naming scheme.
The full project source is also available at https://bitbucket.org/bitli/fitsfilemanager.

Using the Load/SaveAs operation, an ORIGFILE keyword is added with the original file name (if it does not already exsit), so you can still relate the rnamed/moved file to its original file.

This should work on other OS but was only tested on Windows. If somebody try it on another OS, feedback would be appreciated.

-- bitli
 

jlake

Well-known member
Dec 14, 2010
425
0
Thanks for writing this, it will be very handy for me! I occasionally help others get into processing, and sometimes their filenames are just terrible to work with. This will enormously simplify my workflow with them.
 

astrospotter

Active member
Feb 7, 2011
41
0
Took this for a test drive and it looks very nicely done and very useful.  For some time now I have often been doing rename from the  MaximDL format to nicer names that sort well and group like exposures in windows file manager by name  using a command line perl script I wrote so rename for me is my #1 usage.  You have 'upped the game' with this very nice tool and thanks!

The preview and check you have supplied are critical for any tool that modifies data or filenames so that too is 'spot on'.

Your pop-up help menu that indicates the & format mapping is also spot on and very professional.  Nice.

Very nicely done.  AND since you have been generous enough to supply source I will not ask for a 'rename' button as will look into that myself.  Of course for rename the output directory would be a non-required parameter.

Again, very nicely done and I am quite grateful for your contribution to the PI community.

Thanks!
Mark Johnston
 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
Thanks. If you add some function, i would be glad if you provide the patch, si I could include it in future releases. Consider starting from the latest sources in bitbucket, as I corrected some problems. Right now I am in Acatama, so I will not follow this thread very often for a week.
-- bitli
 

vicent_peris

PTeam Member
Sep 3, 2004
995
11
Valencia (Spain)
www.astrofoto.es
Hi,

Thank you very much for your development.

I have a problem: my images don't have standard keyword names. For example, the filter is specified in the keyword "INSFLNAM". I've tried to change the line 20 of the engine JSH file:

Code:
this.defaultKey = ["CCDTEMP","EXPTIME","IMAGETYP","INSFLNAM  ", "CDELT1","CDELT2"];
But this doesn't work. Any ideas?


Thank you again.
Best regards,
Vicent.
 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
Hi,
The defaultKey list only specifies the default keywords shown in the FITS window. I will have a look and I will describe how to change the keywords.
Note that in some cases the script is also aware of the format of the data.  It would be useful if you can post an image with your keywords, so I can test it (you may try to make a very small image to save space, just be sure that it has your original FITS keywords).
-- bitli
 

vicent_peris

PTeam Member
Sep 3, 2004
995
11
Valencia (Spain)
www.astrofoto.es
Hi,

Thank you. I've just uploaded a cropped FITS file (512x512, the original was 2Kx2K approx.).

The keywords are not standard (and I cannot do anything to change them):

CCDTEMP --> CCD temperature
EXPTIME --> Exposure time
IMAGETYP --> Image type
INSFLNAM --> Filter
CDELT2 --> X binning
CDELT2 --> Y binning

I'm also interested in classifying the images by object name (keyword OBJECT). Could this be possible?


Best regards,
Vicent.
 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
Hello,
I have attached a version 0.7 which should be easy to use with your requirements (I hope). The public source repository is also up to date.

The line #define USE_ALTERNATE_KW false should be modified to true to use your keywords (in the original files it is in the -gui.js). The remapping rules are just a few lines below, you can change the FITS keywords as required.

In addition the OBJECT keyword is copied to the &object; synthethic keyword, but without non alphanumeric characters (to avoid creating baroque file names). The method filterObjectName implements the rules to clean OBJECT and can be easily adapted. It is at the end of the -helpers file in the original sources. I am interested in your feedback as to how to best cleanup such textual parameters.

In addition I have implemented the reading of the HIERARCH convention, so your extended keywords can be seen in the FITS keyword window as well and used as source for the synthethic keywords. It is not yet possible to directly refernce a non synthethic &FITS; keyword, but this should come soon.

I will have to find a way to group the "localizable" rules in a file so the source must not be patched at each release, but I am not yet sure how to do it.

[EDITED FEB 2013  - attachment removed, see first post for latest download]

Hope this help, if it is OK I will make a general release.

-- bitli

 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
Version 1.0RC1 has been attached to the first script.
The configuration part has been entirely re-written to be more flexible and hopefully work on Mac and Linux. It is also somewhat more complicated, there are tool tips which should help.

This work on 1.7, it also kind of work on 1.8RC1, except parameters.

I would appreciate if somebody can test it on Mac and Linux.

The full sources and tests are available in https://bitbucket.org/bitli/fitsfilemanager.
Cheers,
-- bitli
 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
It seems to work on 1.8RC2. I also tested it on Fedora 17 64 bits on a VirtualBox, the latest version of the script seems to work (I made just a couple of tests) and does not have dialog layout issue on my machine. 
On fedora you have to right click on the window bar to close the help window explicitely (there may be other ways...)
-- bitli
 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
Version 1.0 is available with minor updates and hopefully better behavior on small screens (I am unable to test on Mac however).  It is available as a single file in the first post for PI 1.7.

It should be part of 1.8 next update or as a later update. The runtime files for 1.0 are availables at https://bitbucket.org/bitli/fitsfilemanager/src/d23c7af704fe/main/js?at=v1.0, the whole project (current development, test files) is also available at https://bitbucket.org/bitli/fitsfilemanager.

The usage of the script may require some familiarity of writing templates, possibly even a regular expression for advanced usage.  I am thinking on a simpler interface (kind of select operations from a list), but this may take some time.

If you have layout problems, please send me a screen dump, the version of the PI and script used, and an information of your OS and screen size.

-- bitli
 

Ghsmith45

Well-known member
Mar 3, 2012
922
5
Sydney, Australia
Hi Bitli
This looks useful.  However, as I understand it, the script generates a filename based on a choice of fits keywords that are already in the file.  Some processes (eg batch preprocessing, image integration) don't preserve all the fits keywords from the component images.  For instance, after creating a master dark via batch preprocessing, the temperature is not shown in the master dark and I would like this to appear in the new filename.  Is there any obvious way around this?
Geoff
 

CCD1024

Well-known member
Sep 22, 2007
407
0
France - Paris
www.cielaustral.com
Hi Jean-Marc
Very useful scrip. Many thanks ! And thanks for the demo of your scripts during AIP Pixinsight training.
I will test on OSX for display layout, I will let you know. Too much busy right now.
Philippe


 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
Geoff:  The script can only take information in the file themselves or the file name (I am not sure where it should take the information from in your case, as it certainly cannot guess where the master is coming from).  I do not remember how BPP is creating the file name - maybe you can rename the raw darks so that they include the temperature in their file name, and then use the file name in the template (like
Code:
&filename;-master;
). Naturally if you have multiple files with the same temperature (for example different filters), you can write the template with the temperature in it (
Code:
dark-master-&filter;_30C_&count;
).  But I do not see how I could find the temperature of the raw darks (they may not even have exactly the same temperature).
Not sure this is helpful, sorry
-- bitli
 

Ghsmith45

Well-known member
Mar 3, 2012
922
5
Sydney, Australia
Thanks Bitli
This is useful. All my dark subframes have the same temperature with this info as a fits keyword and I should be able to go from there after I think about it a bit.
Geoff
 

wfcashwell

New member
May 17, 2016
3
0
I find the FITSFileManager Script is one of the most useful scripts. I use it constantly. Thanks for this wonderful script. I did however have some problems with the 'last' functionality and regExp (clear). I think I have found the problem in the FITSFileManager-parameters.jsh file. I think that the function on line 726 should be 'regExpFromString' (with a lowercase 'r' at the beginning.) Also in the same file I removed the quotes from around the "/*/" entry in the regExpItemListText (for the (clear) regExp). Again thanks for this script.
 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
I think that you are right. I will make the correction, test it and upload it in the new couple of days.
Thanks for your help.
 

bitli

PTeam Member
Feb 1, 2009
520
3
Switzerland
The new version of FITSFileManager (1.5) has been distributed.  It includes that correction, the capability to lodd and save configurations and the capability to convert to XISF file format.
Thanks to wfcashwell for the help in diagnosing and testing the issue and to Juan for the important work to update the script to the lastest GUI capabilities and coding standard.