StarAlignment "Out of Memory"

[RBA]

Machine: Win7x64 and just 6GB RAM...

I'm building a 54 frames mosaic. I've managed to build a reference frame (StarAlign'ing all 54 frames one by one - not a job for the impatient, I assure you that!), and I'm ready to produce the individual 54 sub-frames that will later be thrown at GradientMergeMosaic.

However, when building the structure map of the reference frame, it stops at 12% with an "Out of Memory". Unfortunately, I'm not working with my main 18GB RAM computer (which is in the US) and this machine "only" has 6GB of RAM.

The reference frame is a 32 bit mono image, 32,063 x 15,034 pixels in size.

Is there anything I can do to circumvent this limitation? Processing time is not a problem, so I don't mind if the process starts using the disk, as long as it can finish the job (but don't ask me to use a virtual disk because of not too good past experiences). Other than that, is there anything I can do that doesn't involve throwing to the garbage the RAM I have here and buy bigger memory modules? I'd be rather disappointed if I have to put this project on hold because of memory limitations. Not to mention what could happen to my health if I spent the money on more RAM and then it wasn't enough either (kind of asking how much RAM would be needed).

PS: I managed to build the reference frame by building TWO subframes, then putting them together - because otherwise I wouldn't have been able to build the reference frame with PixInsight alone, as I was running into the 12% Structure Map "Out of Memory" error after I had put together just 2/3 of the whole frame.

Thanks in advance,
Rogelio

[RBA]

BTW I've already considered splitting the mosaic into 3 or 4 parts, but I'd rather not go that way, mainly because eventually I'd still have to put those together anyway. I guess as a last resort I might try that, but still, unless I want to rely on some other software (assuming they don't choke either), the registration between the 3-4 parts will still need to be done... That or put those subparts together manually via Photoshop which doesn't seem to complain about memory limitations. But anyway, I'd rather avoid splitting the mosaic if I can.

[Nocturnal]

Good grief I almost stopped reading when I read '54 frames'

[georg.viehoever]

Wow, these are dimensions beyond my reach...

You may want to try to experiment with the virtual memory settings. I dont have an english windows here, but translating from german, it is under Computer Properties/Extended System Settings/Extended/Performance Options/Extended/Virtual Memory/Change ... see screenshot.

Georg

[Juan Conejero]

Hi Rogelio,

You are indeed brave. Are you trying to break a Guinness World Record or something?

You need a very powerful machine to build this mosaic. I really don't think you can achieve this with just 6 GB of RAM.

kind of asking how much RAM would be needed

Your mosaic requires about 2 GB, just to keep it in memory. Multiply this by two and add 1 GB for the operating system. Now think about memory fragmentation and PI asking Windows to allocate contiguous blocks of these dimensions... Difficult to say, but perhaps 12 GB. This is just speculation though; I've never tried to do anything so insane

The worst part may be GradientMergeMosaic. It will have to generate temporary working duplicates. Georg should have more precise information here.

Now you are not thinking on a color mosaic, do you?

BTW, perhaps I should consider adding a new C++ exception for these cases. Instead of 'out of memory', something like 'did you really think you could do this?'

[pfile]

"i'm building a model of the earth...
...it's a scale model.
...full scale.
...how do you like it?"

you want to send me the files? i have 24GB in my machine

[georg.viehoever]

...The worst part may be GradientMergeMosaic. It will have to generate temporary working duplicates. Georg should have more precise information here....

You are probably right:

- GradientMergeMosaic currently converts everything to double precision, holding all channels of an image concurrently.
- It processes one image at a time, and holds gradients in X and Y direction plus a weight matrix (necessary to see which parts of the image actually have data or background, and to support feathering)
- Finally, it generates the laplace matrix and solves it via FFT, which also requires considerable memory (and runtime).

I never looked into saving on memory (which no doubt would be possible), into the runtime scaling (its probably linear in the number of images, and n*log n in the number of pixels, but I dont know for sure), nor into processing huge mosaics (which would be possible using different algorithms). My recommendation, which is probably valid for any processing with unusually large data:

- start by doing tests with a downsampled version of the problem. Quality is not important here. Observe runtime and memory consumption.
- proceed by enlarging the problem by factors of 1.5 or 2, again looking at runtime and memory consumption.
- when you have enough runs, extrapolate required memory and runtime, and see if this is within your reach.
- If you dont do this, you will spent a lot of time in runs that ultimately fail, or that run "forever" until you cancel them (and you will never know if it was just seconds away from producing a result).

I currently dont have the time to optimize GradientMergeMosaic...but the source is out there for any volunteers.

Georg

[RBA]

Ok, thanks for all the replies. I have a better idea of what NOT to do. I don't think the virtual memory settings will improve anything but I'll still try that anyway. For now I'm going to try to divide the reference frame in to two - I've already checked StarAlignment doesn't complain (we'll see what happens with GradientsMergeMosaic) and if I manage to get the two images nicely done, I'll deal with merging them later.

Juan, so far I only have mono data, but at some point or another, yes, I'd like to add color...

[RBA]

Well, Georg's GMM has been very nice to me. 2 seconds after hitting 'apply' with _half_ of the mosaic (27 frames) it returned a nice "Out of memory" error message.
Better know that right after hitting apply than 7 hours later!!!

Time to get creative I guess.... 

[FunTomas]

Good grief I almost stopped reading when I read '54 frames'

This will be another great image from Rogelio. I prepared for myself set of 49 images, but unfortunately have not computer that can process this set. My set is 7x7 frame mosaic around bubble nebula.

[zvrastil]

Maybe, Juan can set up powerful machine with a lot of memory and PixInsight installed and rent a remote access to this machine... :-).

[georg.viehoever]

Well, Georg's GMM has been very nice to me. 2 seconds after hitting 'apply' with _half_ of the mosaic (27 frames) it returned a nice "Out of memory" error message....

Actually, the number of frames makes not difference to the memory consumption of GMM. It is the mosaic size that matters.

If you need more memory, booking one of the Amazon EC2 High memory instances (68 GB) http://aws.amazon.com/ec2/#instance may be an option. But you first should try to find out what amount of memory you actually need for this project.

If you delay creating the mosaic to after image processing, a panorama software such as Autopano Pro http://www.kolor.com/panorama-software-autopano-pro.html may be an option. For merging images before processing, these software packages are usually useless because their merge algorithms dont have the accuracy of GMM.

Georg

[RBA]

Well, earlier today I restarted PI and threw the first set to GMM and this time it started the processing, but after 5 hours on the first frame (26 to go) I decided to cancel it. Assuming similar speed, this would take 5 1/2 days, and although I can have the patience (then again, remember this is only half mosaic), I worry something might go wrong eventually, and I'm too old to put a computer to work for 5 days and after the "long" wait, go HOLY &*&$#*@!!ING SH*(^**T!!! in the last moment. I know my heart wouldn't take it 

Oh and no, this won't be "another great image". It'll be in fact a rather "boring" image where the only "wow" might just be the size of the project, not the final product itself, but it's something I wanted to do for a while, and that's what matters.

As for how to get this going for now, I'll probably pre-process each of the 54 subframes for now, and stack them together in PS or similar - I have them all pre-registered, so that's not a problem, but of course, the magic of GMM won't be there, at least not until I add more memory to this machine