law of diminishing returns

[Enzo De Bernardini]

Hi guys,

What about this short article of Jerry Lodriguss?

Assuming that both, in the rural sky and in the city sky, we are exposing the maximum possible time to reach a similar SNR between both integrated results, will be necessary many more subs in the city sky. At the same time, if photon quantity from weak objects are not sufficiently numerous to distinguish this light from noise, stacking will not show them. So, we need many more subs from low quality skies, but it will virtually impossible to reach the depth achieved from rural skies.

Sounds right to you?

Greetings!

Enzo.

[mschuster]

What about this short article of Jerry Lodriguss?

Jerry's rule of thumb is probably good, at least for dim objects captured wideband.

It might not apply to bright objects where their shot noise dominates even in the brighter sky.

Nor maybe for narrowband that blocks most of the sky.

Thanks,
Mike

[jkmorse]

Mesmerizing discussion guys!  I really wish we would get into more of these as a community of PI users.  Its great to help folks get over the start up hurdles, but we have a mass of experienced talent represented in just this post, not to mention the community at large, that we can actually make a difference in helping the new folks go to the next level of analysis.  Now for my two cents, for what little they are worth:

About the only "scientific truth" we know is the basic square root rule, namely you "only" improve SNR at the square root of the number of images taken.  Thus, I double my bang, going from 1 sub to 4 for the "cost" of only 3 extra subs.  I like to shoot 36 subs (sorry Warhen  ) since it works out nicely that I am getting 6x improvement in SNR and going from 5x to 6x "only" cost me 11 subs for a 20% gain.  But the next jump takes 13 for 16.7%, then 15 for 14%, etc., etc. (please feel free to correct my numbers here as I am a liberal arts guy who dabbles in math as a hobby, not a career  ).   

So far so good.  Its a "simple" and straight forward analysis.  That is until you add in everything else we have been talking about so far and some we haven't, including the quality of your sky, the number of clear nights available to you, your imaging goals, the dynamics of your CCD, the focal ratio and aperture of your scope, whether you have an anti-blooming gate (some of us have sacrificed that for higher quantum efficiency), and, most importantly, the target you are chasing.   

For me and my skies (some of the best skies 7300 ft up a mountain in NM has to offer) and set up, I am firmly with Warhen that beyond a point (pick a point, any point), you are wasting your time chasing ephemeral gains by stretching out the number of subs you shoot, UNLESS you are striving for a truly difficult and faint target.  But for most targets, your restrictions are based on the resolution of your scope which comes down to aperture.  At some point, you have captured all the meaningful data you can use and its time to move on to the next image.

I most heartily welcome all the brickbats you can throw at me.  That's how we all learn.

THANKS for a great discussion!!

Best,

Jim         

[Warhen]

Great discussion indeed! I wonder how many have been following Emil Kraiikamp's 'Lucky Imaging' examples- http://www.astrokraai.nl/viewimages.php?t=y&category=7
Emil (the developer of AutoStakkert! 2) is using a ZWO 1600MM, taking thousands of 1 second exposures. I'd be so curious to see what stacks of varying numbers of Emil's subexposures look like (i.e. 100 subs vs. 1,000 subs, etc.). This may partially speak to RickS's comments on read noise, and an earlier comment that the low-noise, cooled CMOS will be changing our thinking and methodology in the near future.   

[jkmorse]

Checked out his site.  WOW!

[chris.bailey]

As someone who tends towards disbelief until I have my own evidence , I have been playing with a couple of CMOS cameras. Though I have not gone to the lengths of Emil Kraiikamp's 2,000 x 1 second, I have done 500 x 20 seconds on M31 and a few other bright targets and it really only starts to sing when the stack gets up in the hundreds and 500 is a vast improvement on 200. Low read noise (2.7e measured on one of them) small well depths, 12 bits and a dollop of amp glow pushes you towards short exposures so the only way of improving SNR is to stack LOTS and LOTs

Advantages - no need to guide even on modest equipment.

Major con - Calibrating, cosmetic correction, alignment and stacking hundreds of frames is a royal pain.

Chris

[Niall Saunders]

OK - another 'thought experiment':

You acquire 30 subs, and have suitable claibration data to ignore that influence from here onwards.

You 'stack' or 'integrate' those 30 images, and use 'noise analysis' to statictically eliminate 'noisy' pixels for every 'pixel-column' (i.e. a vertical stack of 30 data values for any given pixel location in the image). This then means that yhe final image will consist of data built from pixels each of which is liable to have used *less* than the 30 available pixel values that were originally available.

So, in theory you have not come anywhere near to the 'law of diminishing returns'.

The question then becomes, "How many *more* images do you need to acquire, to be certaion that - given the overall noise present in your data colection - each final pixel was based on 30 'good' pixel values from what - for example - might nave been 50, or 100, original images.

I do not dispute the simple mathematics that suggest there is little to gain for the effort required in obtaining much larger data sets in the first place.

However the mathematics - in my opinion, only applies to a one-dimensional data set (such as the individual 'pixel-coumn' that i tried to conceptualise above).

What seems certain is that the same mathematics 'rule' applies to each and every pixel-column across the whole x-y grid of pixels in your image.

But - it may be that the pixel-column at x-y location A has its 'best' data in the first 30 images of a 40-image collection of raw subs, yet the pixel-column at x-y location B has its best data in the last 30 images of the same data collection (and so on, for every x-y location).

If you had only collected the first 30 images, then the data at location B is going to be 'noisier' that that at A - because it can *only* be based on twenty raw subs.

So - I really feel that the 'law of diminishing returns' doesn't apply to our 2-D world of pixel values in an array. The 'law' has to be 'collect as much data as you can' - so that you can feed the insatiable beast that is our data-hungry statistical analysis imeage intehration engine.

Hopefully, minds imeasurably more powerful that mine, will be able to tell us how many raw frames we might need to 'guarantee' (statistically) that we will be able to extract 30 'good' pixel values for each and every x-y location in our raw frames!

[mschuster]

Niall,

I think ImageIntegration gives pixel rejection statistics. Say 3% were rejected, so on a 30 frame set you would need basically one extra sub. You might also reject a few in Blink and SubframeSelector, so a few more just in case.

But on a 30 sub set it might be hard to see a visual difference, at least for just a couple of frames. The SNR gain or loss for +/- a couple subs is small around 30.

This all assumes randomness in rejected pixel location, the usual case at least for my dithered data. Trying to absolutely guarantee 30 for every pixel stack does not seem easy to do for all situations.

Thanks,
Mike

[sstuder]

Dear Warhen,
by saying "...otherwise there is no improvement to be gained or the quality of the integrated image could even deteriorate" I didn't mean to quote you. My first step was to inform myself what the laws says, and this is more or less what I found as a general explanation, which I tried to translate into "astro-speak".Thank you for your explanation, I'm less bothered now. 

[Warhen]

Thanks SStuder, got it! Enjoy the book otherwise. Glad it sparked this thread.

[RickS]

I have given this a bit more thought and played around with some numbers derived from real data.  Hopefully the results will be of interest to others 

Below is a set of graphs which shows the relationship between SNR and the number of photons captured in an individual pixel. A sub with a 5% contribution from read noise generally seems to be considered sky limited.  The graphs show why it's worthwhile to aim for a small contribution from read noise!

Wrt diminishing returns, we can clearly see that the graph flattens out.  You could pick an arbitrary point to decide where diminishing returns kick in, but it doesn't have anything to do with the number of subs, only the number of photons we have collected.  If we pick 500K photons as our limit then we could as easily get there with 30 subs @ 1200 seconds on a camera with 8e- read noise or 120 subs @ 300 seconds on a camera with 4e- read noise (halving read noise allows us to reduce sub time by a factor of 4 to get the same contribution to overall noise.)  So, our first result is that there's nothing magic about 30 subs.  The number of subs where diminishing returns comes into play could be a very large number for a very low read noise camera.  This is borne out in practice by the folks using cameras like the ASI1600 to capture images with many short subs.

To relate the graph to a real example, I did some analysis of luminance data I captured for an image of NGC 1097, a target with some extremely faint features.  The data was captured with a 300mm aperture astrograph (Ceravolo 300) and Apogee U16M camera (KAF-16803 sensor) using 32x1200 second subs.  The contribution from read noise in the background areas of these subs is about 6.7%, so fairly close to the blue line on the graph.

Looking at the total number of photons captured in the total integration of 10.6 hours, in the background areas it was around 14.6K, in one of the very dim features visible in the image (a jet known as "R1") it was around 14.9K.  The faint galaxy halo was around 15.4K and a brighter area of the galaxy measured at 55.6K.  The bright galactic core comes in at over half a million.  The blue shaded area on the graph represents the dim to medium brightness features (background & jets up to medium bright areas of the galaxy body) with the really interesting stuff at the very bottom end of this range (the jets and galactic halo.)

The interesting thing about these numbers is that the faint stuff is still way down the curve where more subs will have an impact on SNR, even though the SNR of bright features won't improve much at all (and is already as good as we need anyway.)  So, the second result is that diminishing returns don't apply to an image as a whole.  You need to consider the brightness of the features of interest.  If you want to go after the really dim stuff you'll want to target a lot of photons in areas of very low flux, which means long total integration times.  The "good" news is that even if you have lots of subs it's probably worth taking more if you're looking for improved SNR for the dimmest features.



NB. this model only considers shot noise and read noise.  Under normal circumstances these are the major contributors to noise in our images, at least with cooled CCDs...

BTW, here is the image of NGC 1097.  It was processed in 2013 so I could probably do a better job now    I'll add it to the repro queue!
NGC 1097: http://www.astrobin.com/65880/
Inverted image with jets: http://www.astrobin.com/65881/

Comments and corrections are welcomed...

Cheers,
Rick.

[mschuster]

Rick,

Your background is 14.6k and your R1 target plus background is 14.9k in 10 hrs. So R1 itself is 0.3k e- or roughly 1 photon every 100 seconds. R1 SNR is roughly 300 / sqrt(14.9k) or about 2.5. SNR 3 is a typical rule of thumb for threshold detectability.

Say there was an R2 half as bright. Its SNR would be half as large, likely not detectable. To achieve the same detectability as R1 would require 4x the time, ie. 4x the # of subs.

Dim targets require more time/subs. Dim targets don't respect 30 sub limits.

Thanks
Mike

[RickS]

Your background is 14.6k and your R1 target plus background is 14.9k in 10 hrs. So R1 itself is 0.3k e- or roughly 1 photon every 100 seconds. R1 SNR is roughly 300 / sqrt(14.9k) or about 2.5. SNR 3 is a typical rule of thumb for threshold detectability.

Say there was an R2 half as bright. Its SNR would be half as large, likely not detectable. To achieve the same detectability as R1 would require 4x the time, ie. 4x the # of subs.

Thanks, Mike.  I did cheat a little in that I bolstered the Lum with synthetic luminance extracted from another 10 hours of RGB data and I do believe I managed to show the very dim R4, but it might have been averted imagination 

I do 100% believe your statement:

Dim targets require more time/subs. Dim targets don't respect 30 sub limits.

Cheers,
Rick.