Author Topic: 206 frames with 16MP cam is too much for PI  (Read 560 times)

Offline Bayjoe

  • Newcomer
  • Posts: 7
    • View Profile
206 frames with 16MP cam is too much for PI
« on: 2018 August 22 15:58:27 »
seems like a joke, PI cannot integrate 206 frames, it´s always a problem with RAM and ressources as it seems. Is there some effort to solve this problem?

Offline Lightpath

  • Newcomer
  • Posts: 38
    • View Profile
Re: 206 frames with 16MP cam is too much for PI
« Reply #1 on: 2018 August 22 20:59:31 »
I'm not sure it's the software you're having an issue with.   8)  I just did 331 16mp darks.

Then again, my system is an i7/32gb/ssd.  It's a bit of a beast.

Offline pfile

  • PTeam Member
  • PixInsight Jedi Grand Master
  • ********
  • Posts: 4531
    • View Profile
Re: 206 frames with 16MP cam is too much for PI
« Reply #2 on: 2018 August 22 21:51:55 »
i don't know much about windows but is it possible you need to increase the swap file size?

rob

Offline ngc1535

  • PixInsight Old Hand
  • ****
  • Posts: 311
    • View Profile
Re: 206 frames with 16MP cam is too much for PI
« Reply #3 on: 2018 August 22 22:36:18 »
I cannot read the error...
but just a data point. I have a pretty good machine (no problems with RAM). However, I ran into a problem with running too many parallel threads.
I am not certain why- I didn't have the problem on my old machine (though it ran more slowly in terms of speed). So, in preferences I lessened the
number of simultaneous processes...and that helped tremendously. I have it set to 10. It still whips through the processing- but does so with smaller
"chunks."

Just something to consider.
-adam

Offline Bayjoe

  • Newcomer
  • Posts: 7
    • View Profile
Re: 206 frames with 16MP cam is too much for PI
« Reply #4 on: 2018 August 23 00:14:45 »
i don't know much about windows but is it possible you need to increase the swap file size?

rob

I don´t use a fixed swap file size, because PI does not take it. So I have a dynamic swap file

Offline LarryC

  • Member
  • *
  • Posts: 56
    • View Profile
Re: 206 frames with 16MP cam is too much for PI
« Reply #5 on: 2018 August 23 12:23:41 »
It's not a limitation with PI as I too routinely process more than 300 16mp files with no memory problems, but I also have 32Gb ram with 16Gb as a RAM drive, though I never had a problem when I only had a 16Gb Ram.  I do not know if >200 files would work with less than 16Gb. 

How much Ram do you have available for PI and how have you set up your swap files?  And are you running other programs while performing these PI processes?

Offline msmythers

  • PTeam Member
  • PixInsight Jedi
  • *****
  • Posts: 1176
    • View Profile
    • astrobin
Re: 206 frames with 16MP cam is too much for PI
« Reply #6 on: 2018 August 23 12:58:22 »
The computer might being running out of swap space/low harddrive space. Depending on how many threads your computer can allow that is roughly the number of images that are being processed simultaneously by PI during calibration. As each image will be 32 bit during calibration/integration and the file size for each image is large for a 16mp image. Those images are being kept in swap. If the computer only has one harddrive then swap storage file size can possibly be an issue.  Also remember that on a windows computer the Page File that windows uses for the os/memory-management is also changing in size as needed. So PI with it's heavy ram usage during calibration/integration will be taxing that if the computer is low on resources.

As adam said reducing simultaneous processes could help or even turning off parallel processes in the Global Preferences. You could try turning it off and see if things improve but the down side is calibration and integration speed will be greatly effected. 

One last thing all updates for windows and hardware drivers should be installed for the best reliability.


Mike

Offline Bayjoe

  • Newcomer
  • Posts: 7
    • View Profile
Re: 206 frames with 16MP cam is too much for PI
« Reply #7 on: 2018 August 23 13:01:46 »
Larry, i have only 12GB RAM, maybe 10 for PI. When I managed a fixed swap file in windows, PI doesn´t take it. Only with Linux it takes fixed swap files. In Linux a dynamic space does not work, because its too slow to give PI more. PI crashes with dynamic swap @linux.

But i solved the prob otherwise. The Zlib-compression seems to be the problem. When i converted the files to fits, the integration runs through. Seems to be, that the compression is some hungry process  :D

YEAH  :smiley:   8)