PixInsight Forum (historical)
Software Development => New Scripts and Modules => Topic started by: cameronleger on 2017 November 05 13:57:26
-
Foreword
First, I must thank Mike Schuster for the original version of this fantastic PixInsight Script. Second, of course, the PixInsight team and their continuing open-source initiative, without which this Module would be near impossible to complete. The recent release of 1.8.5 is also worth mentioning because two very important inclusions made this Module significantly easier to make.
I'm sure that most users love the SubframeSelector. It seemed such a worthy item in the pre-processing toolkit that I believed it deserved a more 'standard' method of use that fits inside PixInsight. My primary motivator for this was that Scripts 'block' PixInsight from other tasks. With nearly every use, I singled out a few frames to check out; why are the statistics so different for this frame, and how 'bad' is my worst 'statistically significant' frame? This involved remembering the file name, finding it in a file explorer, and checking it out in another image editor. My secondary motivator was speed. By far this is the slowest part of my pre-processing workflow. Less importantly, I prefer prefixes and this Script only supported postfixes.
Initial Release Notes
This Module has become stable and includes nearly every feature as the SubframeSelector Script. I work in Linux, and while I've provided a Windows build, I've only opened it in Windows to ensure that it loads fine. I do not currently plan on including more features, but I am absolutely welcome to feedback and will continue to develop this as needs arise.
Notable Differences
Of course, this is a native PCL Module instead of a PJSR Script. That alone offers many conveniences, including a huge performance boost. With the addition of threading for the measurement stage, this Module ran 12 frames in the time it takes the original Script to run 2.
The Output Files section is based off the common section between many other PixInsight Modules.
I've ported the recent PJSR StarDetector library to PCL and am using that exclusively instead of the StarAlignment tool. Some of the Star Detector parameters are slightly different because of this.
I've ripped the PSF Fitting portion of DynamicPSF. This plus my port of the StarDetector means this Module is more self-contained and does not call on any other PixInsight Modules or Scripts.
For most measurements, I'm using the Median and Mean Deviation from the Median for dispersion calculations. Star-related paramters, such as FWHM, are now a weighted average of the best-fit stars instead of a simple image-wide average.
The Approval and Weighting expressions are mostly the same, but behind-the-scenes are completely different. I've renamed a few parameters, so be sure to check them if you're copying your existing expressions. These expressions are now run as JavaScript code in the PJSR, thanks to recent updates. This vastly simplified the work involved for parsing expressions. Although I've limited the allowed characters, there are more possibilities with expressions. For example, you can use standard JavaScript functions like 'Math.abs()'.
The Graphs are now a WebView which utilizes the Dygraphs JS library also recently introduced. They are a little more interactive now, and display more information. In lieu of the extra line items in the Measurement Table, the Median and Mean Deviation are displayed on the graph and in tooltips that appear as you hover over points. Subframe Weights are a secondary axis, and the dispersion is displayed with a gradation. Clicking frames still toggles the Approved status, and shift-clicking unlocks the frame.
An additional Graph is also included which shows a histogram of the selected property with a line representing the Empirical Distribution Function. While this graph is not as interactive as the previous, it helps to identify ranges within your distribution of data that are best to include or exclude.
A few items are now placed in a 'Routines' dropdown: Measurement, Output, and Star Detector Preview. First, you must add your frames and measure them. Then, you can use the Output Routine. The Star Detector Routine runs only the Star Detector on the first frame that's supplied, and outputs relevant images for this process and the final structure map used to fit stars against. This helps you when tweaking Star Detector parameters.
(https://raw.githubusercontent.com/cameronleger/PCL/SubframeSelector/src/modules/processes/SubframeSelector/Screenshot.jpg)
To-Do
- Live Monitoring
- Support for DATE-OBS keywords
- Map Outputs (currently, only the Star Detector preview is available)
Release Information
My code is available in my forked branch of the PCL repository: https://github.com/cameronleger/PCL/tree/SubframeSelector
You can download a Linux and Windows build at this releases page of the repository: https://github.com/cameronleger/PCL/releases/tag/01.04.01.0001
OS X 10.12 build provided by Rob: https://drive.google.com/file/d/1xWKzGr4YKXbNaaeui5nH8RZR7olD1bIj/view?usp=sharing
I welcome any and all feedback. Thank you and enjoy!
-
Looks good but in my trial case I got completely different results using the script vs the module. I am using Win10.
Geoff
-
Hi Geoff,
Thanks for trying this out and providing those screenshots.
For the FWHM and Eccentricity differences, I do expect them but I'd like feedback as to if it's 'better' or not. The original SubframeSelector averages those values for all stars it finds. This does an average too, but it weights them by the MAD of the PSF Fit compared to the best PSF Fit in the image, so that it gives less preference to stars that weren't fit as well. This seemed to be recommended in the PixInsight documentation.
It may also be that the PSF Fitting settings in Star Detector Parameters is different than your setting for the Script. Can you also provide how many stars each version matched?
The SNR Weight appears the same, and I expect the Median and other image-related statistics to be extremely similar since it looks like you're looking at monochrome images. I've seen differences in those numbers myself, but I'm using debayered DSLR images and I suspect that's because the ConvertToGrayscale process that the Script uses comes up with a slightly different result than the getImageIntensity method that I'm using.
-
i have built this for OSX, but against OSX 10.11 (El Capitan). i am not sure if it would work against 10.12 (Sierra) or 10.13 (High Sierra) but for what it's worth, here it is:
https://drive.google.com/file/d/0B6cW4ae2KyMJcXI5aThGUDZId2s/view?usp=sharing
rob
-
Looks good too, I've done a comparison, using the same expression, FWHM seem better with PCL.
PC with Win10. I utilize Subframe selector all the time and appreciate the PCL evolution. Thank you for all the contributors.
Antoine
-
Hi Cameron,
As it happens every time a developer publishes a new module or script on this forum, you have realized the main reason why I started and have continued my work on the PixInsight project during all of these years. When this happens, the only thing I can say is thank you, so thank you very much for this excellent work, ad welcome to PixInsight development. This module is especially valuable for me because you have shown that all the effort I am putting in version 1.8.5 to strengthen PixInsight as a development platform is worthwhile. So thanks again.
Also as you have noted in your initial post, here we owe our gratitude to Mike Schuster for the SubframeSelector script, which is one of the most useful and beautiful resources available in PixInsight. Thank you so much Mike for making this piece of art possible and available to all PixInsight users.
Needless to say, I want this released as an official module as soon as possible. I have invited you to join our team of contributors to our official GitHub repository. Please submit a pull request to include your code in the PCL master branch. I want to discuss a few things with you (mainly regarding GUI topics and a few items related to high-level parallel processing) but I think the module is basically ready for release.
-
Great news!
Users who take lots of frames with low-end mounts (like myself), or experimenters on lucky imaging using cooled CMOS cameras, should benefit a lot after this development!
I have only one wish; histogram plots or other data exploration plots. So far I plot the results outside of PI in order to judge from the distribution of the measured values where to set an approval/rejection threshold. I am a novice Python/Numpy/Pandas programmer and I had in mind to write a Jupyter Notebook for the task, i.e., importing the SubframeSelector .csv output and do my plots there and get rid of the closed-source tools I use nowadays :D
Thanks again!
-
Hi Juan,
I'm happy to do this. I'll also look into forking Reference-Documentation and providing proper documentation to the standard of the Script.
dld,
Please, let's discuss more on this. From what you've said, I would think it's already in this Module, because every column of data in the CSV Output is also available as a Graph. Could you give an example of some plots you'd like to see? If there's more meaningful data that can measured and weighted/approved against, I want it! I was unsure if by 'histogram plot' you meant a plot of each histogram of the image channels or histogram as a graph of any numerical data ;)
-
dld,
Please, let's discuss more on this. From what you've said, I would think it's already in this Module, because every column of data in the CSV Output is also available as a Graph. Could you give an example of some plots you'd like to see? If there's more meaningful data that can measured and weighted/approved against, I want it! I was unsure if by 'histogram plot' you meant a plot of each histogram of the image channels or histogram as a graph of any numerical data ;)
A histogram as a graph of any numerical data. Suppose our column data is FWHM. I would be happy to be able to plot a histogram of the FWHM values (example: the first attached plot). In my example I'll approve images with FWHM < 3.0 since images with FWHM > 3.0 lies at the bad "tail" of the distribution.
A rather more complicated plot is the second attachment (scatter_matrix) which (may) combine smoothed histograms (diagonal of the matrix) with scatter plots. This is called a "scatterplot matrix". Such plots allows us to visualize correlations between our variables. For each cell (except the diagonal) we plot the column variable as x and the row variable as y. For example, at the top row the second plot from the left plots Eccentricity (y-value) against FWHM (x-value) . Probably such plots are not that important but may help us 1) find and reject outlier frames (example: sometimes a frame has a low eccentricity value due to passing clouds = low StarSupport and maybe high Median) 2) choose what is the most meaningful combination of variables for an approval formula.
Cumulative probability plots may also be useful (no example attached :-( ).
-
Wow Cameron!
One of my most important tools just got better! Thank you. It used to take about an half and hour to measure my frames. Now it takes a couple of minutes.
A couple of suggestions:
1. Add the "Approved # of #" label somewhere--it's convenient to know how many frames pass the Approval expression.
2. Is there any way to add the ability to minimum and maximum on a Weighting symbol. So something like
(FWHM - 3.642) / (8.449 - 3.642) would become something like
(FHWM - Math.min(FWHM)) / (Math.max(FWHM) / (Math.min(FWHM))Use case: I add frames over time and remeasure to see if I have enough "good" ones yet. When I do, I have to redo a bunch of minimum and maximum values in my Weighting expression. Not a huge deal, but it would be nice to just let the tool take care of that.
Thanks again - Shane
-
Cameron, I have just installed and run your Module. It runs FAST! Thanks again for your efforts!
-
Shane,
Thank you for the feedback. I was already thinking of doing the first item, and the second item is a good suggestion. I figure I can add a Median, Min, and Max just like the Sigma is currently handled, and that would do what you want I think. At first, I didn't want to implement a 'file cache' feature because of the complexity versus perceived usage of the feature. However, with what you've said and my recent experience with multi-night shoots, I think this would be another great addition.
dld,
That's great news. I also put histograms on the roadmap. I'm still toying with it in my head, but I also think that allowing the choice for the X axis and generating a scatter plot would also handle some of the other plots you have described. I hadn't thought to get that technical with the graphs, but I can see its usefullness now.
-
Cool. Great idea about the Median, Min, and Max. I looked through the code today and kind of figured that would be the best way to do it. My "multi-night" shoots last months and the file cache, if it survives PI shutdown and startup would save a lot of time too.
Once again, thanks for your work on this module.
- Shane
-
Hello Cameron,
Something like this https://en.wikipedia.org/wiki/Empirical_distribution_function (https://en.wikipedia.org/wiki/Empirical_distribution_function) should be useful too. It helps to visualize what is the percentage of images accepted for a given acceptance threshold. I think people doing planetary imaging are using something similar.
Anyway, let's keep it simple :) My "matrix" example was way too complicated. I am already a happy user :) And I don't feel comfortable suggesting features while someone else does the hard work :)
Thanks and clear skies!
-
Hi,
Congratulations for your new module. I want to add a couple ideas:
- I also think the cumulative probability graph can be very useful to establish a threshold.
- Since this is a PCL module, I think it would be really useful if your tool is able to monitor a directory to analyze newly saved images. To would be open the possibility to monitor in real time the image quality while acquiring data (inside or outside PixInsight).
Best regards,
Vicent.
-
Fantastic work Cameron, really appreciate your hard work on this.
I think may have found a small issue. When outputting approved subs, the FITS Keyword is not written into whichever is the last/bottom sub in the Measurements Table list. It does seem to be related to whichever is the last sub in the list, as when I reorder the list on a different metric (for example Weight, or FWHM) and therefore change which is the last in the table, then that sub becomes the one that doesn't get the Keyword written in the output file.
Thank again, a great addition to the PixInsight toolbox :)
-
I appreciate all your feedback. I'm not ready to provide a new build yet, but I hope to very soon.
Shane,
I've got the Median, Min, and Max special variables working. It offered me a great chance to refactor how some of that was working into much simpler code.
dld & Vicent,
I've got a histogram working and am working on a cumulative function. I feel that the Empirical Distribution Function makes the most sense given that we have the empirical data, but I've hit a hurdle. I want to show both of these on the same distribution-related graph to save space and since they're strongly related, but this graphing library prefers multiple series to have the same independent axis. As such, a cumulative histogram (https://en.wikipedia.org/wiki/Histogram#Cumulative_histogram) seems attractive, except that it would limit the 'resolution' of the cumulative function to the number of bins used in the histogram.
An alternative could be to 'zip' the two series together and provide 'blanks' where either series does not align with the other, it just seems difficult when comparing floating-point numbers and they may not align nicely. Thoughts? I'm just looking for a value or priority on the solutions to compare complexity/benefit of them.
Edit: I've got the union working properly, so an EDF + Histogram is going well.
Frant,
Thank you for finding that. I'll definitely look into it. I already found another critical bug where a memory leak could happen during the exports. It's in the PCL repository, so if this is released by PTeam before my next update you'll have it then.
-
Great Cameron! I look forward to the release. (I've got a large subframe set I'll test it with!)
- Shane
-
Fantastic work Cameron, really appreciate your hard work on this.
I think may have found a small issue. When outputting approved subs, the FITS Keyword is not written into whichever is the last/bottom sub in the Measurements Table list. It does seem to be related to whichever is the last sub in the list, as when I reorder the list on a different metric (for example Weight, or FWHM) and therefore change which is the last in the table, then that sub becomes the one that doesn't get the Keyword written in the output file.
Thank again, a great addition to the PixInsight toolbox :)
Hey frant,
Unfortunately, I don't have a clue as to why this is happening and I haven't been able to reproduce this. I looked at files generated by earlier and current versions of this module and I did not see this happening between batches of 4 - 300. It's confusing because the output routine loops the 'base' set of measured subframes and not the 'sorted' set that's generated for the table view, so I'm not sure how they could be affecting each other. I was checking the output files by opening them and looking at the 'FITS Header' menu entry under the 'File' menu. Do you have any more hints?
-
Hey frant,
Unfortunately, I don't have a clue as to why this is happening and I haven't been able to reproduce this. I looked at files generated by earlier and current versions of this module and I did not see this happening between batches of 4 - 300. It's confusing because the output routine loops the 'base' set of measured subframes and not the 'sorted' set that's generated for the table view, so I'm not sure how they could be affecting each other. I was checking the output files by opening them and looking at the 'FITS Header' menu entry under the 'File' menu. Do you have any more hints?
I don't at the moment, but I'll be using it again this evening, so I'll see if I can come up with a reproducible test case.
-
Based on your feedback and more usage on my part, I have a new update ready for release. I've updated the original post and I'll include a link here too for simplicity: https://github.com/cameronleger/PCL/releases/tag/01.03.01.0003
-
I'll see if I can come up with a reproducible test case.
Well I couldn't :)
That is to say, I used the tool 4 times during the integration of L,R,G and B subs and it worked flawlessly each time, it just isn't doing it now.
I'm a bit puzzled as to what I was doing wrong before, but happy to say it was probably finger trouble on my part, rather than a bug.
Just installed the new version too, keep up the great work.
Cheers,
Richard
-
Fantastic! I just noticed that the measurements are cached! No more anxiety on loosing the measurements by accidentally pushing the wrong button (been there, done that, several times in the past...) Thanks again Cameron!
-
Way Cool, Cameron!
You just reduced my workload and shortened by processing time even further! The Min/Max stuff works great. The number of files accepted is very useful. The histogram adds yet another important visualization of the quality of the entire subframe set. And the cache--yet another time saver when I've already measured 130 frames and have five more to add!
One more (very simple) suggestion: Add the index number to the black rectangle on the graph hover. Some of us end up with quite a few subframes and it's difficult to tell which one the hightlighted dot and hover information represents.
Thanks again for your great tool - Shane
-
Hi, I'm working on a project with about 250 subs that I'd like to check this out on. I can download it from GitHub but I don't know how to install it into my PI setup. Can anyone provide me with a quick set of instructions?
Update: I think I've figured it out. There's an item under the Process menu to install a new module
John
-
Hi
Just installed the new SFS Module and have been looking at it everything seems to work (Mac running High Sierra) but I can't get a probability graph /histogram to come up. Is that agraph with the Mac or am I missing something.
Thanks for the new module
-
it could be that i need to rebuild it? i just did git pull and i can see a lot of changes, though the tag seems to be the same (1.03.01.0003)
rob
-
Rob, can you rebuild it? Your original build was against 01.01.01.0001, and I updated the link in the original post to point to the latest except for the OS X version.
-
ok will do
rob
-
ok here is a new build for osx...
https://goo.gl/X3c9Wc
i noticed a couple of things:
1) when double-clicking a point in the graph, the viewport containing the file list does not always scroll properly to reveal the file in the list. i was not able to detect a pattern. this happened in the previous version as well...
2) if i right-click the graph pane, i get a contextual menu. one of the options is "view source". if i select this i get a dialog that the process segfaulted and from that point on the graph pane no longer works. not sure how to fix it without restarting PI.
rob
-
Still looking for an answer have installed module on my Mac running High Sierra and module seems to run fine except noted 2 things pf mentioned above but I still can't get a graph That is mentioned in the original post (An additional Graph is also included which shows a histogram of the selected property with a line representing the Empirical Distribution Function. While this graph is not as interactive as the previous, it helps to identify ranges within your distribution of data that are best to include or exclude),.
Please help
Steve
-
this is with the latest build i just made - i see the histogram on the right...
did you load the latest build?
rob
-
Thank Rob
Since the downloads file has the same name I was not sure what version it was but I did switch the files to the latest download on this post and now it is present thanks again.
Steve
-
Thanks, Rob, I updated the OP with the new link.
Re: left-click scrolling, I think a single-click works as well, but that's not the problem. I did not see a method to scroll to an item in the TreeBox, so I'm scrolling to a position I gather from the item at that index. It's possible that I'm using the wrong coordinates; I noticed similar behavior but most of the time it worked. I'll look into this.
Re: view-source, while I don't get the segfault on Linux, it can produce an unrecoverable state. I'm not exactly sure what causes this, but I hoped no one would try! I presume it's because I'm directly setting HTML content instead of pointing to a source file, yet it may have to do with my event handling. I'm hoping I can disable this right-click menu. Otherwise, it's a method to get up a creek without a paddle.
-
Hi Cameron,
i'm a late comer here, but just installed and tried this PCL module. it looks very promising! i love the addition of math, which eliminates the extra steps and need for the spreadsheet tool i have been using (posted elsewhere here) to create the weighting value
I may be running into a bug? I was able to run it once successfully, so i know i can get it to work
I was entering various weighting formulas, seeing the red 'x' indicating a formula error, and fixing syntax until i got the green check mark.
however now when I run measure, it errors out with "nan not defined". in addition, i get the red x for some pretty basic approval expressions. I can't get beyond this point
Thanks
Brian
-
Hi Cameron,
i'm a late comer here, but just installed and tried this PCL module. it looks very promising! i love the addition of math, which eliminates the extra steps and need for the spreadsheet tool i have been using (posted elsewhere here) to create the weighting value
I may be running into a bug? I was able to run it once successfully, so i know i can get it to work
I was entering various weighting formulas, seeing the red 'x' indicating a formula error, and fixing syntax until i got the green check mark.
however now when I run measure, it errors out with "nan not defined". in addition, i get the red x for some pretty basic approval expressions. I can't get beyond this point
Thanks
Brian
Hey Brian,
I'm sorry to hear that. Would you mind using the "Edit Instance Source Code" button at the bottom right of the Module and sending me the contents, perhaps via pastebin? That will help me see all the parameters and measurements to try to track this down.
I cannot tell much so far, other than measurements seem to have been taken but the graph view is not visible, so that might be the source of the JS error. I'm wondering if some parameters were not measured correctly, and the translation from PCL into PJSR produces "nan" for non-numbers instead of the "NaN" that JS would read correctly - not that a NaN is a good thing though.
Also, a quick update. I had done the documentation for this Module, and most of the recent changes were not pushed back to Juan. I was unsure how long his process would take and if he would make changes himself, so I wanted to wait to push those afterwards so that I could handle the merging of code instead of giving him more work. Sadly, I haven't had much time to use this since then. I thought winter would be *awesome* for astrophotography, but it turns out in Atlanta winter means few clear skys and very poor seeing!
-
sure - i'll get that together tonight and PM you with those contents
Thanks
Brian
-
here you go:
https://pastebin.com/i90pcZtJ
two attachments show
1. before state, entered the approval and weighting formulas, they both 'check out'
2. after loading data and pressing measure, get the error
-
Thanks Brian,
I can confirm the issue is with the Weighting Expression, not the Approval Expression, although that one can display the X for some reason.
The 'plain' variables such as FWHM and Eccentricity only represent the current subframe being measured. So, Math.min(FWHM) and Math.max(FWHM) will return the same thing as FWHM, and subtracting these values will result in 0. The 'nan' error appears when it's dividing by 0.
The simple fix here is to use the 'special' variables such as FWHMMax and FWHMMin which are calculated before each subframe and represent the global max and min of all subframe FWHMs. Here's that expression modified to work for me:
35*(1-(FWHM - FWHMMin) / (FWHMMax - FWHMMin)) + 7*(1-(Eccentricity - EccentricityMin) / (EccentricityMax - EccentricityMin)) + 18*((SNRWeight - SNRWeightMin) / (SNRWeightMax - SNRWeightMin)) + 40
That said, I will definitely look into 'nicer' or more obvious messages when 'nan's are involved, and try to make the X's work better in these cases.
-
btw there are no logs that i can see, so i don't know how to usefully report these bugs, but i've had a couple of instances where the SFS module hangs up and can't be quit, necessitating a force quit of the whole PI platform. i've taken to saving my project before running it.
i believe these occurred while measuring subframes, but they also may have occurred while interacting with the UI. sorry i can't be of more help...
rob
-
Thanks Brian,
I can confirm the issue is with the Weighting Expression, not the Approval Expression, although that one can display the X for some reason.
The 'plain' variables such as FWHM and Eccentricity only represent the current subframe being measured. So, Math.min(FWHM) and Math.max(FWHM) will return the same thing as FWHM, and subtracting these values will result in 0. The 'nan' error appears when it's dividing by 0.
The simple fix here is to use the 'special' variables such as FWHMMax and FWHMMin which are calculated before each subframe and represent the global max and min of all subframe FWHMs. Here's that expression modified to work for me:
35*(1-(FWHM - FWHMMin) / (FWHMMax - FWHMMin)) + 7*(1-(Eccentricity - EccentricityMin) / (EccentricityMax - EccentricityMin)) + 18*((SNRWeight - SNRWeightMin) / (SNRWeightMax - SNRWeightMin)) + 40
That said, I will definitely look into 'nicer' or more obvious messages when 'nan's are involved, and try to make the X's work better in these cases.
Great thanks. Is there a place where all the special variables are documented? these are the obvious ones I need for now :)
Brian
-
Hey Rob,
I haven't had anything like that; you're on OS X right? I'll look into debugging/logging information.
Hey Brian,
They're currently in the tooltips on those fields. I think it lists the variables, and that each of them have a few special versions and lists those types. I also have it in the documentation with some examples, but that's not released yet. I could look into distributing that before the formal release, but I think it only works in PI.
-
Hey Brian,
They're currently in the tooltips on those fields. I think it lists the variables, and that each of them have a few special versions and lists those types. I also have it in the documentation with some examples, but that's not released yet. I could look into distributing that before the formal release, but I think it only works in PI.
no sweat, you showed me the main ones i need for now, so I am patient for the full release. Thanks again so much, you saved me i don't know how much time.
B
-
Hey Rob,
I haven't had anything like that; you're on OS X right? I'll look into debugging/logging information.
yes, OSX. thanks.
rob
-
Hi Cameron
that works beautifully.
i have one more request - not a huge one, but is there a way to save the weighting formula from session to session or through application restarts?
i typically use the same formula, so i have to either type it in (painfully) every time or copy it from another place.
or maybe there's a way to save and restore different formulas?
minor request, I don't want to look a fabulous gift horse in the mouth ;)
Brian
-
Hey Brian,
One of the reasons I wanted to make this as a Module was to gain Module-specific features, such as saving Process Icons and their included settings. I recommend you do that once you have the formula you want. I think I saw in your screenshots that you had tweaked other parameters too; that will of course save all of them as well.
Another thing to note about that: it will also save all loaded subframes and their measurements. I also find that really handy to go back and quickly check weights for other processes where I'd want a really good frame. I have a couple of these icons saved for different camera setups, and I keep one around in the project for the last measurements I took just in case I have to pause and come back later.
-
Hi,
I am probably missing something really obvious here...but how do I output only the approved frames to a separate directory......after measuring the sub-frames I change to "output Sub-frames but it outputs all of them included the rejected frames?
Paul
-
Hi Paul,
two things come to mind:
1. make sure you have rejected frames, there will be red 'x's on the output graph - this is the obvious one
2. The process window counts all frames, but actually only outputs the approved ones. This threw me too, but i checked the folder and sure enough it was only the approved frames
hth
Brian
-
Thanks.....I guess that's the (not so obvious) thing I missed....P
-
For some reason all my graphs disappeared and I cant seem to get them back.
-
A possible issue with expressions for weighting: the script allowed exponential expressions [FWHM^2, SNRWeight^(0.5)], but I can't get those to work in the process. Is this intentional? I'd like to use roots in some of my weighting expressions.
Kevin
-
@cfosterstars
Could you explain more or provide screenshots? Has a PixInsight or OS restart solved this issue?
@astrovienna
It is weird that it's accepting it; can you provide the whole expression for me? I tried what you sent like "FWHM^2" or even "2^3" and it shows the red X and didn't calculate the weights.
In any case, what you want is possible with the Math module. JavaScript doesn't have a quick symbol for powers, but the function Math.pow(base, exp) will do it. See this page for more details: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Math
-
Thanks for the quick reply. I'm not at my PI machine now, but I'm almost certain that one of the expressions I tested was simply FWHM^2, and it successfully parsed in the script. I'm trying to use a generic approach to weighting rather than one tied to a particular data set - something like SNRWeight/(FWHM^0.5) rather than relying on the min and max of SNR and FWHM in a given data set.
From the page you pointed to, it looks like the Java expression I need would be SNRWeight/Math.pow(FWHM, 0.5). Correct? Sorry for asking, but me using Java scripts is like a gorilla playing with hand grenades. :)
Kevin
-
Looks like it should work to me!
-
Yes, that worked. Thanks for writing the module. This is a great addition to the tool set.
Kevin
-
Feature Request:
Add the ability to include FITS header keywords as columns to aid in grading.
should be part of csv export file as well.
See the Fits Keyword script for a (scripted) example.
Thanks for considering!
-
gnewell
I don't know if this meets your need, but you can already add a FITS header keyword for the weighting when you output your frames. I don't think you can do more than one though
brian
-
I'm really enjoying tinkering with this tool! It's really well done.
My problem at the moment is that I don't have a good solid understanding of what makes a "good" sub. Does anyone have a resource I could read that would help me come up with my own expression?
Thanks very much-
Mark.
-
https://github.com/cameronleger/PCL/releases/tag/01.04.01.0001
All,
There is a technically significant update available, however I only have a Linux build at-the-moment so I'd appreciate any volunteers on the other builds. It was hard to notice at first since it was already much faster than the PJSR version and clearly benefited from more threads, but I have a larger machine now and it was essentially choking on the higher core counts. Turns out, there's a setting that should've increased performance considerably, but it did the opposite :tongue: If you thought it was fast before... well let's keep going :police:
Mark,
I think this would be very relative and dependent on the context, but that's OK because a lot of the parameters can be used in a relative way! It will come down to your strengths, weaknesses, and preferences. Personally, I use this expression as a start on every batch:
(FWHMSigma / -3) + (EccentricitySigma / -3) + (SNRWeightSigma / 3)
This basically says: combine the relative values of FWHMs, Eccentricities, and SNRWeights, where a lower FWHM and Eccentricity are preferred and a higher SNRWeight is preferred. Those values are scaled by 3 sigma units (the negative sign is what's making it prefer lower numbers) as nearly all of my frames are within that range. For example, if a subframe's FWHM is 3 sigmas lower than the median, it will contribute 1, whereas if it were 1 sigma higher it would contribute -0.33. Sometimes it's necessary to add a constant afterwards like 5, so that my smallest weight is above 0. I think otherwise the further processes' weighting calculations are relative so it doesn't matter if your best is 500 or 5, it compares it as a percentage of the best.
Perhaps the original documentation here, https://pixinsight.com/doc/scripts/SubframeSelector/SubframeSelector.html, or my work-in-progress updates, https://github.com/cameronleger/Reference-Documentation/blob/master/tools/SubframeSelector/SubframeSelector.pidoc, will help you start your expressions. If you don't like reading my documentation as it is in that link (and why would you?), you can download that folder and use the 'Development' -> 'Documentation Compiler' Script to view it in PixInsight. Since this is not formally included with PI, it's a little funny to use.
-
i can try to build it for OSX but i think i'm maybe behind a couple of revs on Xcode and don't have the latest PCL installed so it may take me a couple of days.
rob
-
new update looks really nice - it was already performing nicely but i'll take faster performance any day!
i hope someone does windows soon ;)
thanks again
Brian
-
i can try to build it for OSX but i think i'm maybe behind a couple of revs on Xcode and don't have the latest PCL installed so it may take me a couple of days.
rob
well so i made the mistake of pulling the latest PCL from the pixinsight github, and i found that after i built the module that 01.08.05.1352 (which is the latest for OSX) complains that the API version is unknown (161) vs an expected of 160.
i guess i can rebuild PCL from your fork of the PCL tree which i assume is going to have api version 160... but something is funny here. either OSX core is a release behind the PCL release, or the PCL API version has been bumped without a corresponding release of the core application on every platform... (?)
i also have an older build of PCL that i used to build the last SFS so maybe i can just point to that instead.
rob
-
so i'm going to need some guidance here -
the release called PCL-01.04.01.0001 on your github does not seem to contain the SubframeSelector module source code.
the branch called SubframeSelector does contain the source code, however, that branch contains the API version 161, so i can't build in that tree.
if i build PCL in the PCL-01.04.01.0001 release, then build SubframeSelector from the same-named branch, PI gives a segmentation fault when attempting to load the module. not sure exactly what is wrong there.
there is one more thing i can try - i have an older build of the PCL libraries which should be api version 160. but i feel like the SubframeSelector source should have been in your PCL-01.04.01.0001 release zipfile, and it's apparently not.
rob
-
Sorry this is so confusing, Rob.
My SubframeSelector branch, https://github.com/cameronleger/PCL/tree/SubframeSelector, is based off the latest PCL commit and has all the SS code up-to-date. I merged the latest PCL at the end of this development cycle so that a Pull Request would be clean.
The tag for my release was automatically added when I did a 'release' in GitHub, but that appears to have automatically applied to the 'master' branch which is actually the plain PCL repository from a while back... It also appears that the automatic zip of the code has the same issue. I'll see about moving that tag to the correct location, and that's on me for not understanding what GitHub does automatically.
You are right that PCL sets an API version 161, that happened six months ago here: https://github.com/cameronleger/PCL/commit/97e83fd6747bcf5655a4caf395a6109c348c79e9.
Comments indicate it applies for 1.8.5.1357, and yet I'm running 1.8.5.1352 also and had no issues with this build or loading the module. For some reason, the Linux 1352 doesn't care about the API version change while the OS X 1352 does.
For what you want to try, it should work. Since I haven't modified any files outside of the SubframeSelector folder, you can probably checkout any PCL commit you want, copy that folder in, and try building. I can't imagine that I'm using any extremely new portions of the PCL.
Aside from fixing the 'release' on my GitHub, let me know if you'd prefer I do anything else for you. While I cannot do the OS X build myself, I can do whatever magic in the structure/code/files that makes it easier for you.
EDIT: The tag is in the correct place now, and the source zips were automatically updated with that as well. Looks like the same issue applies to previous tags as well.
-
i got it built against the older v160 PCL tree that i had hanging around, and SFS loads and runs OK in PI. juan confirms that 161 corresponds to a future release of PI, so that problem will take care of itself eventually.
at this point the only open question is if building against the 10.14 SDK is kosher for running this module against older versions of OSX. since i am running 10.13 and the latest XCode contains only the 10.14 SDK tree, it seems like the answer is probably yes. hopefully someone who is running 10.12 can try my build.
one other question - when opening the module in PI, besides the main interface window, a 2nd (blank) window opens behind it. any idea what that is about?
thanks,
rob
https://drive.google.com/file/d/1xWKzGr4YKXbNaaeui5nH8RZR7olD1bIj/view?usp=sharing
-
I've seen the blank window too. It used to appear and disappear, and sometimes it would stick, but lately it's been sticking around. This inconsistency happened without code/build changes also! I fear that it has something to do with the QTWebKit views that PI is using and/or the JS graphing library. I noticed the new PhotometricColorCalibration doing it too, but it's much quicker and disappears every time.
Eventually, I'd love to build a native-drawing graphing control instead of relying on this embedded WebView; that's great for quick setups but the interaction code I had to write with it is pretty quirky. I'll look for any hints in the PCC code if it's available.
-
one other question - when opening the module in PI, besides the main interface window, a 2nd (blank) window opens behind it. any idea what that is about?
I have the same minor issue but I thought it was something I've done wrong.
-
i can try to build it for OSX but i think i'm maybe behind a couple of revs on Xcode and don't have the latest PCL installed so it may take me a couple of days.
rob
Sorry for the noobie question - but how do you install the module at the Linux level? I know once it's installed I need to go to the Manage Modules feature in PI to ensure it's checked.
Thanks.
Mike
-
Hey Mike,
That's basically it. Wherever you have it and installed it from, it will keep looking for it there. If you'd like, you can move it somewhere that makes sense to you e.g. somewhere in the PixInsight folder, but that's purely maintenance and shouldn't affect it.
-
was there ever a windows compile done on the latest version?
thanks
Brian
-
Sorry, no, I'll look into it this weekend and post it here :)
-
Sorry, no, I'll look into it this weekend and post it here :)
fantastic! that would be great, i appreciate it
Brian
-
Hey Brian,
I literally got this latest reply as I was uploading the release to GitHub. It's ready :) https://github.com/cameronleger/PCL/releases/tag/01.04.01.0001
I simply pulled the latest changes, built the solution, and opened the module in PixInsight to make sure I didn't have the same OS X API version issues. I wasn't able to run it, but the latest version is built and opens at the least.
-
Hey Mike,
That's basically it. Wherever you have it and installed it from, it will keep looking for it there. If you'd like, you can move it somewhere that makes sense to you e.g. somewhere in the PixInsight folder, but that's purely maintenance and shouldn't affect it.
Worked like a charm. Thanks Cameron.
Mike
-
Successfully ran (and LOVE) the process! A few questions I hoping someone can help me with. Below is a screen shot of my FWHM weights after entering the subframe expression. I guess the first question I have is can we use the same formula structure as we use with the script? Here's the formula that I entered:
(10*(1-(FWHM-2.528)/(2.528-4.482)) + 10*(1-(Eccentricity-0.497)/(0.847-0.497)) + 30*(SNRWeight-1.1167)/(1.1479-1.1167)) + 50
As I understand it the lower the FWHM number the better the image. My next set of questions relate to the following data:
-Index Image #3 has a FWHM of 2.602 (worst number), and a weighted score of 92.772 (highest score)
-Index Image #6 has a FWHM of 2.265 (second worst number) and a weighted score of 74.103 (lowest score)
-Index Image #7 has a FWHM of 2.257 (best number) and a weighted score of 88.084 (second best score)
Why would image #3, which has the worst FWHM number, have the best weighted score? And image #6 has the second worst FWHM number yet it has the lowest weighted score? If I am interpreting the data properly, the worst scored FWHM image (#3) will carry the most weight (92.772) into the final product rather than the best FWHM image (#7).
Lastly, to underscore my complete confusion and ignorance, highlighting my need for help - image #7 has the best FWHM score, has a better weighted score than image #6, but not the best score that I would expect it to have. Someone please point me in the right direction...
-
(10*(1-(FWHM-2.528)/(2.528-4.482)) + 10*(1-(Eccentricity-0.497)/(0.847-0.497)) + 30*(SNRWeight-1.1167)/(1.1479-1.1167)) + 50
You have inverted the FWHM value in your term. It should be (FWHM-2.528)/(4.482-2.528). I notied this because I just made the same mistake. ::)
-
(10*(1-(FWHM-2.528)/(2.528-4.482)) + 10*(1-(Eccentricity-0.497)/(0.847-0.497)) + 30*(SNRWeight-1.1167)/(1.1479-1.1167)) + 50
You have inverted the FWHM value in your term. It should be (FWHM-2.528)/(4.482-2.528). I notied this because I just made the same mistake. ::)
Wow - what a bonehead - here I am typing all this detail... Thanks!
-
Hey Mike,
I think in some cases the old expressions may be used, but in this case I think you'll benefit from reading the tooltips on the expression inputs. For example, you can replace a lot of your constant numbers with smart parameters like FWHMMin, EccentricityMax, and so on.
-
Hey Mike,
I think in some cases the old expressions may be used, but in this case I think you'll benefit from reading the tooltips on the expression inputs. For example, you can replace a lot of your constant numbers with smart parameters like FWHMMin, EccentricityMax, and so on.
Sounds great. Where can I find the expression inputs? I've looked and unable to find. Thanks for all the help!
-
Sorry, I meant 'expression inputs' as the text area inside the SS where you are typing the Weighting or Approval Expressions. Hover over them for a little bit and you'll get a large tooltip explaining the things you can type into them. I think that most things in the SS will have these.
'Inputs' is a common term for a UI developer to say something that receives user information, like a box to type in text, or a checkbox, or a file browser icon.
-
Sorry, I meant 'expression inputs' as the text area inside the SS where you are typing the Weighting or Approval Expressions. Hover over them for a little bit and you'll get a large tooltip explaining the things you can type into them. I think that most things in the SS will have these.
'Inputs' is a common term for a UI developer to say something that receives user information, like a box to type in text, or a checkbox, or a file browser icon.
Got it. So if I understand the documentation the formula would read:
(10*(1-(FWHM-FWHMmin)/(FWHMmax-FWHMmin)) + 10*(1-(Eccentricity-Eccentricitymin)/(Eccentricitymax-Eccentricitymin)) + 30*(SNRWeight-SNRWeightmin)/(SNRWeightmax-SNRWeightmin)) + 50
Btw I love the fact that the data points are cached. I forgot to identify the best weighted image among my LRGB files - what a breeze with that feature!
Mike
-
Where can I get the latest version for windows?
Max
-
https://github.com/cameronleger/PCL/releases/tag/01.04.01.0001
Although, the *latest* version will be released soon :)
-
https://github.com/cameronleger/PCL/releases/tag/01.04.01.0001
Although, the *latest* version will be released soon :)
any update for the release of the latest version on windows? thanks again!
B
-
The changes that I've listed in this thread are in that link. The *latest* version will come with the next version of PixInsight.
-
great thanks. I looked on github and it looked like the date was september, but i'll download the latest, and await the PI update ;)
Brian
-
I think that's because I setup the release for Linux then, and simply updated it with the Windows build after I stopped procrastinating. It wasn't a new release and it doesn't show an "edited" timestamp.
-
A person mentioned in a related post that some Weighting expressions are built in. I’ve looked all over SFS and can’t find them. Is that possibly not the case, or where might they be located?
-
If you're thinking of whole preset Weighting Expressions, then no that's not the case. Maybe they were trying to talk about the "variables" that are pre-defined for use in the Weighting Expression, such as "EccentricitySigma" or "FHWMMax", which would be replaced with the values of the Eccentricity in Sigma units of the whole set or the maximum FHWM value of the whole set, for example. Hover over the expression text boxes for a tooltip that describes them.
Once you come up with one you like, if you save the instance as an icon on your workspace it will be there for you next time you open it.
-
I've just stumbled across this and it looks awesome but i've hit a brick wall here.
I downloaded the Windows version of the zip file from https://github.com/cameronleger/PCL/releases/tag/01.04.01.0001 but i'm not sure how to install/add this script to my PI.
Sorry for being thick.
-
http://pixinsight.com.ar/en/docs/36/pixinsight-install-modules.html
rob
-
this may help
http://pixinsight.com.ar/en/docs/36/pixinsight-install-modules.html
-
awesome stuff. Cheers fellas. I knew i had to install it as a module but i was pointing at the .zip folder extracted contents. That‘s where i was going wrong. :)
-
I tried to output the subframes with overwrite existing files enabled and a blank postfix, but encounter the following error.
I just want to add into the FITS header SSWEIGHT, as there are drizzle files attaching to the original registered files.
Using the new files cannot reuse the drizzle files.
This issue does not exist with the Subframe selector script.
Can you help?
-
is it possible that file is locked because it's open in another process or program?
Brian
-
if i understand what is being done, the file is locked because SFS has it simultaneously open for reading and writing.
sounds like you'll have to write the files to another directory then move the files back to the original directory (thus overwriting the originals) by hand.
the script version of this tool might be reading the whole image into memory, closing the file, then writing the file from memory to disk. the SFS process i guess works differently. cameron would have to comment.
rob
-
Rob is correct, but there's more information than just the image to transfer, and basically all of it is directly from the input file to the output file. I never imagined someone trying to replace the input images, so perhaps it needs to check for an empty pre/post-fix before attempting the process.
I'm not certain that reworking the read/write to support this use case is the best idea. At least in my processing and the other workflows I've seen, using the Subframe Selector would come before registration (and thus before any drizzle files exist). The most important reason for this is that, depending on how much the images are shifted from registration, you can massively skew the metrics it uses and improperly weight the images. Another less important reason is that you might want to use the process to determine which image you're registering to as a reference.
-
oh I see what's going on.
yeah, i have to bite the bullet and create a new set of image files when doing SFS. i do this prior to registration, which avoids your issue. it's extra disk space and hassle, but it works for me
hth
Brian
-
Rob's suggestion of using a different directory and moving them over the originals will work as well.
Overwriting files, especially the input files, is scary! It's unfortunate that it will take disk space, but basically every process will do this so it's not abnormal to me.
With a small drive, I'm frequently deleting the previous few steps' data if I'm comfortable running their processing again if I mess up, in order to be able to run most of my processing.
-
Hello Cameron,
I just used the SubframeSelector process in the last 1.8.6 release of PixInsight. It works very well. Great job thanks :-)
Nevertheless Output Files are .XSIF. Is threre any way to output files in .FIT (even if FITS is a deprecated file format) ?
Thanks
Regards
Igor
-
yeah - nice job on the final implementation. the floating panels will take some getting used to but it continues to be integral to my processing workflow
Brian
-
Hello Igor,
Thanks to Juan it now follows a pattern similar to most processes; that is initially hidden from the user but if you Edit the Instance's Source code with the icon at the bottom right, there's a parameter in there that you can change.
-
Very good ! Thanks Cameron :)
-
Thanks Cameron for the great update to subframe selector.
I really like having both prefix and postfix available now. In the previous version on the script I hacked it so I could add a property like FWHM or Weight to the prefix or postfix of a file. Very useful for a quick sort of files. I'm wondering is there a way to add property variables to the postfix or prefix in the Subframe Selector PCL Module?
Thanks.
Sam
-
There is not, but that sounds like a good idea to me.
-
Would be great if you could add this feature :)
-
Hello at all,
Im used the old sfs for a long time and now the new subframeselector process.
Gratulation for the new tool !!!!
Before some days i did the update to the new PI Version 1.08.06.1448
Sometimes the tool worked well but most of time (and at the moment allways) an issue appears - see attachment.
I dont know what i can do....
Thanks for an idea.
kind regards
Robert
-
Hi Robert,
This is a known problem with the current version. We are working to fix it, sorry for the inconvenience.
However, when this happens, please close PixInsight and run it again. The script execution error only happens occasionally, or at least this is what we have seen so far.
-
Hello Juan,
Thanks for your info. No worries
Good look for your work.
kind regards
Robert
-
Am I missing something here, in your screenshots it shows the Subframe Selector and the Measurements window combined, in my 1.8.6 on Windows 10 it shows them as two separate windows
-
Hello,
I've been using the module with 1.8.5 and I am now trying with 1.8.6. The graph error appears almost all the time.
I'm using a laptop with high density pixels under windows 10, so that there is scaling of the screen and the pixinsight UI. Could it be related ????
Thanks,
-
Tried this evening on my other PC which has a "regular" display (no scaling) still under windows 10. No error appeared during manipulations :)
Other differences can exist (this is a fresh windows install, while the previous laptop has been running for 2 years), but this still is a hint.
Edit : I made another test on the "scaled" screen and it works with a different screen configuration. I forgot to mention that I'm working on multiple screen configuration, which seems to be an issue with Pixinsight : the auxiliary screen cannot be "above" the main screen. So I changed the screen positions to level both screen vertically, and the graphs are working properly. So that may be another cause of the issue, instead of scaling :).
Cheers,
-
New SubframeSelector process is great, now I can automate it from my script.
My problem is that it leaves Measurements and Expressions windows open. Is there a way not to open them, or close them from a script?
Another issue is that my script dialog window goes empty when I use SubframeSelector inside my script. All buttons and text is blank, I see only one frame in the script dialog window. This could be my use of Dialog but it did not happen until I started calling SubframeSelector.
Thanks,
Jarmo
-
Hello at all,
I hope somebody can help me....
Im seeking for a "Approved" formula.
For me it wouldt be fine: FWMW<2.5 Eccentricity < 0.55 and SNRWeight < 17
I hope thats possible to combine this.
I tried but cant get no red crosses.
Screenshot is attached.
Thank everybody !
Robert
-
Try using the boolean AND operation, for example: "FWHMSigma<2.5 && Eccentricity<0.55 && SNRWeight>17" This will approve subs that meet all three criteria. I turned around your SNRWeight expression. For that parameter bigger is better :)
-
Graphs not works for me in new Subframe Selector.
Some clue why?
-
I have PI installed on two computers, and tried that graph works on Windows 7 but not in Windows 8.1 (or perhaps some library missing or unupdated or ...) :sad:
-
Is it possible you didn't have any weighting expression?
Brian
-
Pepe, do you see the graphs from the newer PCC module? Previously, I was simply setting the graph view directly with the generated HTML and web scripts for the graph. However, Juan discovered an interesting bug in QT that was giving some users the crashing graphs issues. So, he reworked it to be like those, which technically write their HTML to a temporary file and load that into the view. It should work the same between these two.
I'm not familiar with the temporary directories on Windows, but checking the PCC and where these temporary files go would be my first troubleshooting steps.
-
Brian:
Graphs works in my PC W7 without any data, as you can see. Thanks.
Cameron:
Thanks for your response, but I'm sorry, I'm not a computer programmer ( only an astrophotographer :D ) and do not understand these concepts. I know only that SubframeSelector process not works in my W8.1 laptop, and I have to return to old Mike Schuster's "SubframeSelector" in the "Script" Menu :-\
-
Cameron, finally I found a solution for me.
Uninstalling process 01.04.04.0029 version integrated on PI, and downloading and installing your 01.04.01.0001 version.
It works fine ;)
Thanks Cameron.
-
I have installed the latest version of PixInsight 01.08.06.1457 and like the new SFS process (Thank-you Cameron) but it is giving me different results to the SFS script that I'm used to using......
I ran the new process and got FWHM results of 7-8. Knowing I had reasonable data that would normally give lower FWHM results, I decided to run the script instead.....FWHM came out around 4-5, which from past experience is typical for good data using my setup. Tbf all the measurements are a little different. I have the System Parameters all set the same in both.
Could someone please help me understand why this is, and if it's just a case of getting used to knowing what numbers represent good data, when using the new SFS process.
I've attached screenshots of both the script and process....if any of my settings look wrong please point me in the right direction :D :D
.....the graph stopped working but thanks to this thread a reboot of PI sorted it. :D
Thanks in advance....
-
Have you used the same or similar numbers under System Parameters? Those directly affect the way you see those numbers. Also, this module uses DynamicPSF to calculate the values, whereas the older script has its own methods for calculations.
-
Thanks for getting back to me. I did use the same values under system parameters so from what you're saying I would assume it's the different method for calculating values that is giving me the differences.
Would you suggest just accepting this and adjusting my approval of subframe values to adjust for this increase ?
eg. 4.5 FHWM was a good sub using the script and the new module measures the same sub a 7.9 FWHM. Maybe set my approval FWHM as 8.5 ?
-
Yes, updating it would be fine if you have tested those values and determined that in your case they are ideal numbers. If you change lenses, focal lengths, or cameras, I think it's better to look into the FWHMSigma, FWHMMedian, etc. values so that you could use the same expression for any case. That'd make the expression relative to the data-set instead of fixed values, though, so it's more like "best of this batch" and not "best ever".
-
Nice one. Thanks again. Whist it wasn't halting my processing, it was making me wonder. I can keep going forward and it looks like I've got some more learning to do......what FWHMsigma and median are. lol
-
When I 'Measure' my subs, some of them have a weight of 'inf'
What's going on here?
Thanks,
Dan
-
In the original SFS, output section, there were two sets of drop down boxes: approved, rejected. In my case where I process batch streams of 200+ images, most of the images are okay and the two or three bad ones I want moved to a "Rejected" folder. So I leave the approved action as "None" and the rejected action as "Move". I then end up with approved images where they were and a new folder "Rejected" containing the bad images. This makes it a lot easier when coming back to this job later and analyzing what was wrong with the rejected images. It also means it is easy to move the rejected images back into the main folder to rerun SFS with a different set of parameters.
Can we get this same functionality added to the new PCL SFS?
Craig
-
hi, I am using the module on 1090 subframes.
But when I get to about 800-900 frames analyzed (in the console) the PC crashes
-
Thanks Brian,
I can confirm the issue is with the Weighting Expression, not the Approval Expression, although that one can display the X for some reason.
The 'plain' variables such as FWHM and Eccentricity only represent the current subframe being measured. So, Math.min(FWHM) and Math.max(FWHM) will return the same thing as FWHM, and subtracting these values will result in 0. The 'nan' error appears when it's dividing by 0.
The simple fix here is to use the 'special' variables such as FWHMMax and FWHMMin which are calculated before each subframe and represent the global max and min of all subframe FWHMs. Here's that expression modified to work for me:
35*(1-(FWHM - FWHMMin) / (FWHMMax - FWHMMin)) + 7*(1-(Eccentricity - EccentricityMin) / (EccentricityMax - EccentricityMin)) + 18*((SNRWeight - SNRWeightMin) / (SNRWeightMax - SNRWeightMin)) + 40
That said, I will definitely look into 'nicer' or more obvious messages when 'nan's are involved, and try to make the X's work better in these cases.
Could you help me understand if is it possible to get the min or max of variables such as FWHM , Eccentricity or SNRWeight of the analysed frames?
-
Thanks Brian,
I can confirm the issue is with the Weighting Expression, not the Approval Expression, although that one can display the X for some reason.
The 'plain' variables such as FWHM and Eccentricity only represent the current subframe being measured. So, Math.min(FWHM) and Math.max(FWHM) will return the same thing as FWHM, and subtracting these values will result in 0. The 'nan' error appears when it's dividing by 0.
The simple fix here is to use the 'special' variables such as FWHMMax and FWHMMin which are calculated before each subframe and represent the global max and min of all subframe FWHMs. Here's that expression modified to work for me:
35*(1-(FWHM - FWHMMin) / (FWHMMax - FWHMMin)) + 7*(1-(Eccentricity - EccentricityMin) / (EccentricityMax - EccentricityMin)) + 18*((SNRWeight - SNRWeightMin) / (SNRWeightMax - SNRWeightMin)) + 40
That said, I will definitely look into 'nicer' or more obvious messages when 'nan's are involved, and try to make the X's work better in these cases.
Could you help me understand if is it possible to get the min or max of variables such as FWHM , Eccentricity or SNRWeight of the analysed frames?
I understand how it works and I have to use variables FWHMmax e FWHMmin.
-
Graphs not works for me in new Subframe Selector.
Some clue why?
By chance I have discovered why can't see the graphs in my PC, and other strange things related with temporaly saved paths in several PI procceses.
And is because problems wtih character codification in path strings.
As example, the browser not show my SubFrameSelector graphs because url generated is
"file:///C:/Users/Jos%FFffffc3%FFffffa9J/AppData/Local/Temp/SFS_graphs_K9D58SCHNGNY.html"
istead of
file:///C:/Users/JoséJ/AppData/Local/Temp/SFS_graphs_K9D58SCHNGNY.html"
This is because my Windows user folder name has a character with accent.
My problem is thay is not easy change it, I should create a new user in Windows and I would not like have do it.
-
I would suggest adding the option to append the calculated Fields as a custom extension for the output file name
-
Dear Cameron,
thank you one more time for remarkable SubframeSelector PCL process, it works great!
But I ran into some difficulties when I've tried to use it in scripting.
As I understand, we should run it with two iterations:
1) Run with routine property set to SubframeSelector.prototype.MeasureSubframes and get array of results through measurements property
2) then run with routine property set to SubframeSelector.prototype.OutputSubframes (and set approvalExpression to something like "FWHM < 3")
But no matter which approval expression is set, all frames are always outputted (i.e. rejection doesn't work)
Even if I manually save SubframeSelector process icon and then run it from my code:
SFSproc = ProcessInstance.fromIcon("test2");
SFSproc.executeGlobal();
it puts to output folder all input files (if that saved process we run manually, it works as expected, copying only approved files).
So maybe I am missing something obvious here?