integration of different-length subexposures

[pfile]

i have some subs of the veil nebula taken last year - 90 exposures of 4 minutes each - canon 50d @ ISO800. this year i have taken maybe 10 subs at 300s and 20 subs at 360s, but at ISO1600.

what (if any) is the right way to combine all this data? i realize it might not make sense due to the wildly differing SNR, but i thought i'd ask.

HDRComposition does not seem to make sense as there's nothing in any of the subexposure lengths which is overexposed, except for the stars. in the past i've used much shorter exposures with HDRComposition to recover the stars. plus, HDRComposition is not really a tool for improving SNR.

i understand that ImageIntegration can weight images by exposure length, but the calibration process leaves no trace of the EXIF data in the FITS header, so that's out.  i'm guessing that weighting by Average Signal Strength won't work either, because there are tons of LP gradients in the subs.

ideas? or is this just a bad idea?

[sdh]

My understanding is that, to some extent, summing exposures to achieve a "long" exposure length can partially overcome light pollution and noise.

I don't know if it applies in your situation, but I've seen some faint (read "16bit-limited") evidence that it applies in my situation. I am actively pursueing HDRComposition in PI, after seeing a bit of what Photoshop can do with HDR.

Having said that, every shot I've ever done has proven unique when I tried to process the data. It takes me 2 or 3 hours just to come to an informed opinion about what-the-heck I have in a given data set. I've been doing astrophotography for well over 5 years, and I find it considerably more challenging than anything I ever got paid for. I.e, sometimes fun.

[Juan Conejero]

Hi Rob,

Our ImageIntegration process does not need to know exposure times and EXIF data; it simply ignores these items, unless you explicitly tell ImageIntegration to use exposure times (from EXPTIME or EXPOSURE FITS header keywords) to weight images, which in general is a bad idea.

The recommended image weighting criterion is the default noise evaluation mode. In noise evaluation mode, ImageIntegration computes noise estimates for each image and uses them to derive and assign weighting factors. More noisy images have smaller weights. This method provides the most accurate weighting to maximize signal-to-noise improvement in the integrated result. Note that our noise evaluation routines are purely numerical and work exclusively with pixel data; they don't need and don't use any information about physical acquisition conditions, such as exposure times, temperature, etc.

[RobF2]

Nice!  Now that's very handy to know thanks Juan.

[pfile]

Hi Rob,

Our ImageIntegration process does not need to know exposure times and EXIF data; it simply ignores these items, unless you explicitly tell ImageIntegration to use exposure times (from EXPTIME or EXPOSURE FITS header keywords) to weight images, which in general is a bad idea.

The recommended image weighting criterion is the default noise evaluation mode. In noise evaluation mode, ImageIntegration computes noise estimates for each image and uses them to derive and assign weighting factors. More noisy images have smaller weights. This method provides the most accurate weighting to maximize signal-to-noise improvement in the integrated result. Note that our noise evaluation routines are purely numerical and work exclusively with pixel data; they don't need and don't use any information about physical acquisition conditions, such as exposure times, temperature, etc.

thanks, juan. in theory i already knew this, but i think i was tripped up on "noise" vs. SNR. my assumption was that a short light frame at, say 0C could have less noise than a long light frame at 20C, and so the short frame would unfairly receive a heavier weighting. but if we are talking about SNR, then it makes sense. i suppose if your lights are properly calibrated then the dark signal should be gone (and the temperature effectively normalized) and so now the difference in the noise component is just the sky background noise. assuming this is small compared to the signal in the longer frame, that one should dominate.

is that a correct understanding?

[Juan Conejero]

Yes, that's correct. The noise evaluation routines implemented in ImageIntegration assume that all the frames have been properly calibrated, so thermal noise has been ruled out of the problem.

[pfile]

okay, so i'm about to embark on the mother of all integrations then. probably 90 frames of 240s @ iso800, plus 50 frames of 300,360s @ iso1600.

one thing that i have noticed is that ImageIntegration tends to consume significantly more memory than what is specified in the "stack size" parameter.  the other day i was integrating 100 flats. this machine has 18GB and if i set stack size to the current amount of free memory (say 10GB) i find that pixinsight can keep growing until the machine is paging. is this normal?