Output Subframes with new Subframe Selector Process

[andrewbarton]

Everything in the new SFS process is working great, but I can't figure out how to output subframes? In the old script, I would move move rejected subframes, but that functionality does not seem to exist. But, I also can not figure out how to move the approved subframes to a new folder. Apply globally seams to measure only. Ultimately, I am trying to separate approved from rejected subframes. Thanks for any suggestions.

[pfile]

this version of the tool only lets you copy approved files to a new directory. you need to change the mode to "output subframes" and then apply global after setting up an output directory name.

while it's very roundabout, what i do to move rejected subframes is to invert the selection, then create a subdirectory to copy the rejected frames to, and remove the suffix. then, using the shell i go into the directory with the rejected frames and use a for loop to delete the files with the same name from the directory above. i realize this is not something most people using windows would be able to do easily, but i'm running OSX so it's at least feasible.

rob

[andrewbarton]

Thanks Rob! I completely missed the "Routine" selector at the top. The new tool makes a lot more sense now.

Your shell based workflow makes sense to me. I too am an OSX user primarily because of its Unix underpinnings. I also make extensive use of python for file manipulation when I want to go beyond what I would normally do in a shell script. Astropy is quite nice for FITS manipulations.

[pfile]

cool - can you install astropy via fink? i still have not transitioned to homebrew.

rob

[andrewbarton]

What I would recommend is installing Anaconda as it creates a python environment for scientific computing including Jupyter Notebook, astropy, Matplotlib and much much more.
See: https://www.anaconda.com/download/#macos

I can't recall if I installed Anaconda using the provided installer, used HomeBrew or MacPorts. I have played around with lots of package managers. Homebrew is my goto package manager followed by MacPorts at this time. I am guessing all of the above and fink will work for Anaconda.

Andy

[pfile]

ok, yeah, now i remember - i had to use python for something a while back and the environment was really complex (at least as installed thru fink.) i seem to recall having some serious version/dependency problems.

so a meta-package like anaconda is probably a good idea.

thanks for the pointer.

rob

[andrewbarton]

You're welcome. I went through the same thing; dependency hell. For what I am doing Anaconda is working great.

Andy

[drmikevt]

This thread is really interesting  (in terms of new/expanding astro functionality on the Mac side).  I looked at the Astropy  package.  Just curious - what do/would you use it for?

Mike

[dld]

One potential use of astropy and friends, is for querying the VizieR web service in order to generate Custom Catalogs for the Image Annotation Script 8)

[andrewbarton]

Astropy is my astrophoto Swiss Army knife. I primarily use it to filter imagines and correct mistakes.

For example, one night I set my camera cooler too low and it got warmer after I started imaging. So I ended up with a bunch of images taken at the wrong temperature. I used astropy to sort out images that were too warm by reading the fits temperature header. Another time I took a bunch of flats with my acquisition program set to light. I used astropy to set the imagetyp header to flat so that BPP could categorize them correctly. I have also used astropy to give me stats on saturated pixels in my images.

I never know in advance what tool I am going to want, but if I can't find it in PixInsight, astropy is my goto. Also, it's not just for Mac, Python is available for just about every platform.

[Terry Danks]

I have a nightly requirement to discard subs with eccs. greater than 0.60 caused by bad seeing. Typically around 15% of my subs are affected nightly in unpredictable time clumps. I only recently discovered how to get (the old) SFS  script to automatically move these files to a "junk" folder.
I was initially dismayed to find that the "new and improved" SFS process does not allow quite this capability as did the old SFS script.

My new routine is to move the "approved" subs, i.e., those with eccs. < 0.6 to a folder entitled "Approved Subs" rather than have the bads subs moved to a "Discard" folder.
Sort of the "same difference" but, as 85% or so of subs are "good" it seems a tad less efficient in that it requires more data to be moved than did the old script.
But the new SFS process has a huge benefit too. I measured the time taken to measure 84 twenty MB subs taken last night. The old script took 8 minutes 34 seconds on my aging computer. The new process, only 1 minute 2 seconds! Eight times faster! WOW!