DrizzleIntegration process takes more than 3 hours

[frajanssen]

I have 36 luminance files (each 64 MB).
I did a CosmCorr on the calibrated files.
Then a StarAlignment (with Generate drizzle data).
Then I used LocalNormalization (with Generate Normalization data).
Then Image Integration (with the L.Norm. and Drizzle Files).
Then I did a DrizzleIntegration (with the L.Norm. Files).

This DrizzleINtegration process take more than 3 hours.
What am I doing wrong (or is this normal?).

frajanssen

[drmikevt]

It does not look as if you have done anything wrong.  I think you just have a slow computer....

On my quad core i7 it takes about a half hour with that many images.  3 hours is a lot longer, but what is your processor?  Are you using an SSD drive?  How much RAM do you have? How much space is left on your hard drive? Is the data on an external hard drive?  Have you tried using a swap disk?  Are you sure you have not restricted PI's use of your processors? all of these things will greatly affect your speed in processing. 

Mike

[frajanssen]

Thanks for the reaction.

Intel Core i5-6500
SSD: 232 GB
RAM: 254 MB
Space left on hard disk: 500 GB
Swap disk: no
External hard drive: not in use

Fra

[drmikevt]

Fra

Your answers are confusing

 - How can you have a 232GB SSD hard disk, but also have 500GB left on your hard disk?  Perhaps you can see that this makes no sense.

 - If, in fact, you only have 254MB of RAM, then I think this is the problem.  Most people here have 16GB of RAM, about  64 times what you say you have.  PI is a very RAM intensive program.  Re-check this number.  If its correct, you need to look into how to upgrade your RAM to the max that your computer can handle.  If your computer can not handle at least 8GB, it is time for new computer.

Mike

[frajanssen]

Mike,

I have a Samsung SSD750 EVO 250 GB. I Suppose this means 250GB.
About RAM: it is indeed 16GB (sorry!). The RAMDisk is 254 MB.
I have 500GB free space on the hard disk.


Fra

[drmikevt]

Ok, so for the moment, we are going to ignore your hard disk because your numbers are still not making any sense.

A 254mb Ramdisk is very, very small.  Very small.  Did I mention its small?

Open PI, go to Global Preferences and look at the Directories and Network tab. What is listed for your Swap directories?  If the only thing listed is your Ramdisk, then this is definitely the problem (probably).  Your Ramdisk should be at least 8gb, and even that is not very big.  You will probably get better results if you use a swap directory folder on your hard drive.  This is the default configuration.  The best results may come from listing your RAMdisk first (assuming you increase it to 8GB) and then a swap folder on your hard drive second in the 'Swap storage directories' section.  Some people even list their RAMdisk 2 or 3 times before listing the hard drive folder, but I do not have personal experience with this.  Even if it is not the only swap directory listed, a RAMdisk that small is most likely slowing things way down.  I don't know what camera you are using, but it would not even fit one of my subs, which are around 330mb.

I should say that if anyone else thinks the above is incorrect, please chime in.

Mike

[OldSkyEyes]

in case there is 500GB space left (and you have a 250GB SSD)  then most likely there are two hard drives in the system, the one with 500GB is the slow one, you don't want to use that for your swap storage.
Use the other one.
And first try without RAMdisk just your SSD later you can see if a ramdisk can improve it or not. (because you are also removing RAM when you make a RAMdisk)

And do a benchmark and upload it, so it can be compared. 'Scripts' -> 'Benchmarks'

[frajanssen]

I have two hard drives: 1 TB HDD and an SSD 250 GB.  And RAM: 16 GB

I have made a RAMDisk or 8 GB (tests with Benchmarks: this was the best choise).

Now it works faster, but DrizzleIntegration takes up too much time.

Thanks everyone for the information.

Fra