Author Topic: Optimizing PI for speed?  (Read 2418 times)

Offline joelshort

  • PixInsight Addict
  • ***
  • Posts: 260
    • Buckeyestargazer.net
Optimizing PI for speed?
« on: 2017 August 29 14:42:19 »
One of the cameras I own is a QHY163M CMOS camera and because of the camera characteristics I am taking shorter subs (2-5min) and getting lots of them.  This of course results in large data sets.  With my Moravian G3-16200 the data sets were considerably smaller so I never really bothered with trying to optimize PI for maximum performance and speed.

But now I'm finding that processing the larger data sets is annoyingly slow, so I'd like to get some advice on how to configure my system for the fastest possible performance.  (Having said that, I suspect that not much can be done given the specs of my computer).

I have a middle of the road gaming laptop that I use for processing.  That is to say a much better laptop than most consumer laptops out there, but still not like a high end desktop.
i7-3940XM 3.2ghz processor
16GB RAM
1TB Samsung SSD
Nvidia GForce GTX 680M/Intel HD Graphics 4000 - graphics processor

What can I do to configure PI for better performance?
Joel Short
www.buckeyestargazer.net
CFF135 f6.7, SV80ST, G3-16200M, QHY163M, QHY183M

Offline Niall Saunders

  • PTeam Member
  • PixInsight Jedi Knight
  • *****
  • Posts: 1456
  • We have cookies? Where ?
Re: Optimizing PI for speed?
« Reply #1 on: 2017 August 29 15:36:16 »
Hi Joel,

One thing that has helped me get more speed, in general, out of any machine (laptop or desktop) it to get the applications onto one drive, and the user data onto a second (preferably physical, as opposed to logical) drive.

Recent hardware has also allowed the application drive to live on the likes of an on-board M2 drive, and that can liberate other drives for use by PixInsight.

You need to consider the overall memory size of an image data-set as well, and then consider how that can be handled by the main RAM. For example, working with 16-bit resolution, three colour channels and a given image resolution defines the 'byte size' of an individual image - but, how many of those images are you then trying to process 'as a batch'? What are your figures for your new imager, compared to those of your previous model?

So, let's even say that you need 'double' the RAM to hold all the images - and so you then believe that you need to get a hold of even more RAM than you have already got - is this (potentially massive) expenditure actually worth it, just to be able to pre-process your image sets 'a little bit faster'?

After all, once you have your 'MasterLight', and are ready to start post-processing, you might really not need nearly as much RAM at all - instead, what you then (potentially might) need is sheer 'grunt' from your main CPU. So then you have to start thinking about the number of physical cores you have on your CPU die, and the bus-speeds you then have available to allow you to stream data to and from memory (both Graphics and Storage). You would also potentially benefit from much larger on-die memory caches.

So, a lot depends on the age of your machine and the technology that was used to 'create' it. And, obviously, how much it cost you.

Personally, I will now never return to using a laptop - for anything. Three weeks ago I scrapped twenty-three laptops and five desktops. I now only have one laptop left, and it will soon be scrapped as well - to be replaced by what was a super-number-cruncher of a desktop machine purchased eight years ago - purely based on existing PixInsight requirements at the time.

And I don't really believe that a 'games' machine is necessarily the best machine design for the job. I spoke to my PC supplier whose reply was "Leave it with me, and I'll call back tomorrow". The next day the engineer called me back, having spent hours looking at the PixInsight Benchmark Database, and defined exactly the machine he felt would most suit my needs. Better still, ny involving an 'expert', I managed to get a higher-spec machine than I thought I could afford, for about 30% less than my estimates were suggesting.

Try searching the Forum for "swap file ram disk" - there are some useful hints that will pop up.

Hope this helps.

Cheers,
Niall Saunders
Clinterty Observatories
Aberdeen, UK

Altair Astro GSO 10" f/8 Ritchey Chrétien CF OTA on EQ8 mount with homebrew 3D Balance and Pier
Moonfish ED80 APO & Celestron Omni XLT 120
QHY10 CCD & QHY5L-II Colour
9mm TS-OAG and Meade DSI-IIC

Offline joelshort

  • PixInsight Addict
  • ***
  • Posts: 260
    • Buckeyestargazer.net
Re: Optimizing PI for speed?
« Reply #2 on: 2017 August 29 17:45:45 »
Thanks Niall.  I didn't buy the laptop (several years ago) with PI in mind but for general use and I have no intention of replacing it any time soon.  I was just wondering if I could possibly get any better performance out of it with what I already have.  And I suspected the answer was 'no'. 

I've tried the swap file ram disk thing several months ago but I didn't see any improvement because I think I need much more RAM for that. 

Once I have the master files created then of course processing goes quickly.  It's the preprocessing stage that takes a long time.  I'm working on a 2 panel mosaic image that has LRGB for each panel and each panel consists of nearly 300 images.  I'm now on my third day of preprocessing  ;)

For sure, my next computer will be a desktop machine with better specs but that won't happen any time soon. 
Joel Short
www.buckeyestargazer.net
CFF135 f6.7, SV80ST, G3-16200M, QHY163M, QHY183M