Hi Tomáš,
I assume it's 32bit system. I'm not able to merge even two 6MPix frames on 32bit system with 2GB of memory. However, 64bit system (Kubuntu Linux in my case), running on same machine does the merge without problem. I was not able to do the merge on 32bits, regardless of process settings or available memory.
regards, Zbynek