A Word doc for Subframe Selector

[CharlesW]

I have enjoyed using David Ault's tutorial for Image Integration found on his website at Trapped Photons. Really comprehensive work. But, accurately typing his approval formula in Subframe Selector correctly each time is a bit of a challenge. To overcome that I've created a Word doc to assist. It allows you to type the values into a table and they automatically get populated into the formula. Then you copy and paste the formula into the Expressions area of Subframe Selector. https://www.dropbox.com/s/c1iljaiilytbql1/Weight.docx?dl=0

To use, open the document and type the high and low values into the table. As you move to the next box the values will populate the appropriate places in the formula. Check the final box outside the table after you've filled in LowNoise. That will cause the LowNoise values to populate. At this point, "Stop Protection" of the form in Word's Developer tab. Copy and paste the formula into the Subframe Selector Expression Approval box. Close the Word doc, do not save it, and it will reopen in it's automatic state again the next time you need it. A little simplistic but hopefully you may find some use for it.

If you wish to modify his formula to suit your own requirements, the doc is based off Bookmarks and Cross References.

[jkmorse]

Charles,

Great idea and thanks for the effort.  I agree that's a great tutorial but any help with the formulae is most welcome!

Best,

Jim

[dnault42]

That's excellent Charles.  I'll add a link to this on my website, if that's ok.

What would be even better is if we could extend the SubframeSelector script to have a normalized variable or at least maybe just a min/max for each metric.  That would simplify the equation some, or at least make it more generic.

Regards,
David

[Warhen]

Charles and David, Springer Press offers an 'Extra Materials' download site. When you gents tie this up, would you be willing to have this posted there for readers of my Inside PixInsight book? Web links are to be avoided, but I'd like to be able to post this with credit to both of you of course. You can contact me privately and thanks for considering this.

[dnault42]

Charles and I talked and came up with slightly different version of this.  I wanted to include a few different expressions and allow the weightings to be specified.  I redid the document with Excel, mostly because that's what I know how to use but also because I think it will have better compatibility across other spreadsheet programs.  The new version can be found here:
https://drive.google.com/file/d/0B2J4InZni9UrLVctTjBiZDlHR00/view?usp=sharing

I added some notes about how to use the spreadsheet and put in a section on credits.

Warhen, you are more than welcome to put this in with the extra materials.

Regards,
David

[jkmorse]

Thanks guys, that is great stuff.  I use your expressions with every image and appreciate the update.

Best,

Jim

[Warhen]

Thanks a lot David and Charles, very kind of you- appreciate all of your efforts for the PI community! Question for all including Mike S., Jim, obviously the FWHM/Eccentricity information is important to a master's quality. If SNR is one's main priority- how much better is ordering the files in this way, over simply letting ImageIntegration compare and weight each sub to the reference image chosen for its SNR quality? Thank you! 

[dnault42]

I'm curious what others say, but here's my take.  If my main goal is maximum SNR then I would use ImageIntegration's noise evaluation.  I would still use SubframeSelector and blink to potentially throw out bad data before stacking though.

Regards,
David

[mschuster]

Comments:

- Using an SNR metric rather than just Noise, as David has done, is better IMHO. Juan has discussed this repeatedly. Example: very smoky, low transparency skies. Noise will be low (background sky absorbed by the smoke), but so is dispersion (brighter objects attenuated likewise). Yes low noise, but bad frame anyway due to low contrast. An SNR metric helps solve this issue. But please note that the typical SNR metrics are NOT foolproof. I can give counter example (high thin clouds cause bright star halos across the frame, high dispersion, so SNR metric tends high, but sorry bad frame).

- IMHO if eccen is bad occasionally it is better to figure out why and fix the problem on your setup. I did on mine and so end up never needing to use eccen in the weighting. I do reject on bad eccen, but basically this nearly never happens.

- David asks about normalized values, the script has a form of this already, the Sigma values. Check out the documentation.

- IMHO FWHM weighting is useful, but its mix ratio with SNR seems subjective. Don't have an objective criteria. All I can suggest is to run several integrations with different mixing and pick the one you like best.

Thanks,
Mike

[dnault42]

Mike,

All good points and I agree with almost everything you stated.

I don't rely on SubframeSelector alone but include Blink in the flow to do a visual inspection and make sure that I catch anything that SFS may have gotten wrong.  I've gotten where I do a pass of Blink before and after SFS.

I agree with attacking the problem at the source, however I will say that it is not always possible to fix them immediately.  During the period when you are fixing things or saving up for a purchase that will fix it, including this metric can be useful.  It's also useful when going back over old data or you are processing someone else's data.  I've also found it to be the least accurate of the metrics which is typically why I weight it quite low.

While the Sigma values are normalized in a sense they are not in a 0 to 1 range which is what I was targeting.  Putting each metric in a 0 to 1 range gives me a more deterministic result.  Of course, I could be missing something here, which is definitely possible.

I 100% agree with your last point.  The weighting algorithm is entirely subjective and I usually do end up running several iterations with different result before moving on to tuning the remaining ImageIntegration parameters.

By the way Mike, thanks for putting SubframeSelector together.  It is an amazing tool for getting the most out of your data.

Regards,
David

[mschuster]

Thank you.

While the Sigma values are normalized in a sense they are not in a 0 to 1 range which is what I was targeting.

Yes. Do you map the range min..max to 0..1 linearly? If so, the mapping is sensitive to min/max outlier values, not good. The typical value (location, i.e. median) won't be mapped to the middle of the range 0.5. I am worried that the weighting may suffer from inconsistency across different frame sets.

I have thoughts on an option, I will post more later.

Mike

[mschuster]

Here is an option, which may reduce outlier sensitivity and be more consistent:

SNRWeight * (2^ -(k * FWHMSigma)), where k is a nonnegative parameter.

With k set to 0, the weighting equals SNRWeight, which is a (scale / noise)^2 weighting metric that Juan has argued results a good maximum likelihood estimator for integration purposes.

FWHMSigma equals 0 on frames with median FWHM, +1 on frames with FWHM one sigma above median FWHM, and -1 on frames with FWHM one sigma below median FWHM.

Frames with negative FWHMSigma should be weighted higher than frames with positive FWHMSigma.

Setting k to a positive value will do this. For example, with k set to 1, frames with 0 FWHMSigma are weighted SNRWeight, frames with +1 FWHMSigma are weighted 0.5 * SNRWeight (i.e. less weight), and frames with -1 FWHMSigma are weighted 2 * SNRWeight (more weight).

A k value of 1 may be too large, typically. At least on my frames k values like 0.25 to 0.5 seem more reasonable. Some examples are shown below. As before, run several integrations with different k values and choose the one you like best.

Basically, I like SNRWeight by itself because it has an objective basis. I just want to tweak it a little bit based on FWHM.

PS: You can use EccentricitySigma similarly as an additional multiplicative term in the weighting with its own parameter likewise if you wish.

Thanks,
Mike

Her are SNRWeight and FWHM for a set of frames:




Here is the resulting weighting with k set to 0.25. Frame 1 has a relatively poor FWHM, so its weighting is penalized. Frames 22 and 23 have relatively good FWHM, so their weighting are rewarded:


Here is the resulting weighting with k set to 0.5. Same as before except FWHM variations have more effect:

[dnault42]

Mike,

Thanks for the detailed response.  I like that, instead of weighting SNRWeight and FWHM separately, you are modulating SNRWeight by FHWM (and potentially Eccentricity).  It feels like it is a more elegant solution.  I'll take some time to digest what you said and try some of my own tests and modulations of this idea.

Regards,
David

[mschuster]

David,

Modulation is a nice term.

Note that SNRWeight modulates 1/noise^2 by scale^2. Twice the scale (i.e. contrast) gives four times the weight.

So maybe half the FWHM should give four times the weight also? If so then maybe this:

SNRWeight / FWHM^2

Mike

Here is the result on the prior data. Appears good to me, poor FWHM are penalized and vice versa rewarded and with no parameter to tweak it is easy to use:

[dnault42]

I ran did several comparisons and variations with one of my data sets that includes data from a dark site as well as my backyard.  I thought this might provide a good way to compare what happens across highly variant SNR images.  This is what I found (there is one type-o I just noticed, the multiplication in the 3rd expression should be division):


The chart is a bit busy, but I wanted to have the FWHM & SNRWeight visible in the background for reference.  I normalized the output ranges so that the weights were more easily comparable.

I was surprised how closely all of them tracked.  There are some definite differences though.  Some of the expressions show more variance due to the SNRWeight.  Visually I thought the SNRWeight / FWHM^2 had the most variance, but the standard deviation is a bit lower than the SNRWeight*2^(-0.5*FWHMSigma).  I guess what I am seeing is that the values in SNRWeight / FWHM^2 trend lower than the other expressions.  The expression I was using previously (the long one) shows the least influence from SNRWeight which I expect given that I've weighted that metric very low compared to FWHM.  The (SNRWeight^0.5) / (FWHM^1.5) expression was just a random guess at some different exponents to alter the variance of the results.  It's nearly impossible to see because it lines up almost exactly with the last expression.  That wasn't intentional but I thought it was interesting.

I really like the increased variance of the second expression and it is somewhat controllable by varying the exponents (the 3rd expression).  It can also be mapped to a desired range given that you can find the min and max of the SNRWeight and FWHM values relatively easily.  I can update the spreadsheet with some of these variation, not that it is strictly necessary for all of them, but for the ones with controllable output ranges it might be useful.

Regards,
David