PC questions...

[dpastern]

I have worked in IT for many years, but I have not actively followed any new hardware for 7 or so years.  So, I am very out of date, and looking for a heads up from people who have kept up to date with the tech etc...

So...I have a 7 year old computer (from memory, it's running an Intel 6 core i7), 64GB DDR3 ram.  It's in storage at the moment and hasn't been used for several years (long story!).  Running Samsung 840 and 850 SSDs.  How well can I expect it to perform? 

How much of an improvement do NVME SSDs offer vs traditional Sata SSDs in regards to PixInsight performance? 

How much improvement does DDR4 ram offer over the older DDR3 standard?  How much of an improvement does 128GB offer over 64GB RAM in general performance (if anything)?

PixInsight recommends (can't remember which page) "parallel processing".  What is that?  Is it similar to RAID, i.e. just another term for Raid 0?  If that is the case, my old computer doesn't have a NVME slot, so I'd have to get a NVME card and plug it into an existing PCI-E slot as far as I understand.  Do I get 2 of these, and use 2 of the PCI-E slots and use software raid to set up a raid 0 array.  I'm still very new to PixInsight, and have no idea on how to enable parallel processing etc...

I presume that I can use normal SATA SSDs for a raid 0 setup. 

I've used OS X (on an aging MBP mid 2009) for years, so i have no idea if RAID can be set up natively via Windows 10 or if it'll require 3rd party software, if so, what software is recommended by the community?  yes, I know hardware raid is preferred, I have no idea if there's a hardware raid card for NVME SSDs, but I presume you can get such a card for Sata SSDs.

Are there any performance gains from running PixInsight natively in GNU/Linux rather than Windows from users experiences?  I've been using Linux on and off since '97 (I was an early adopter) and I'm pretty familiar with Linux, more so than Windows and OS X to be honest. 

I would really rather use my 7 year old computer, even though the cpu/ram tech is outdated and slow by today's standards and adapt some of it to take advantage of newer tech like NVME SSD.  My motherboard can only take a maximum of 64GB RAM, so is therefore maxxed out.   

Yes, I'd love an AMD threadripper 3970x, but at near AUD 6k for the CPU, motherboard and RAM, it ain't gonna happen lol 

Any and all thoughts appreciated.

[dld]

I'll try to partially answer your question since we probably have a similar computer, mine is based on an i7-5820K (slightly overclocked) 16GB of 2133MHz RAM and an EVO 850 under Win 10. I am a DSLR imager and my typical processing pipeline consists of hundreds of raw DSLR files, and CFA Drizzle. The machine is still pretty usable when PI does something heavy like StarAlignment or ImageIntegration. While I don't use LocalNormalization I find it to be probably the most demanding process and I probably wish for more memory. But overall my old PC does the work well without me holding my chin and doing the finger tapping thing while waiting the machine to multitask

[dpastern]

I'll try to partially answer your question since we probably have a similar computer, mine is based on an i7-5820K (slightly overclocked) 16GB of 2133MHz RAM and an EVO 850 under Win 10. I am a DSLR imager and my typical processing pipeline consists of hundreds of raw DSLR files, and CFA Drizzle. The machine is still pretty usable when PI does something heavy like StarAlignment or ImageIntegration. While I don't use LocalNormalization I find it to be probably the most demanding process and I probably wish for more memory. But overall my old PC does the work well without me holding my chin and doing the finger tapping thing while waiting the machine to multitask

I can't remember which i7 chip I have and until I get the PC out of storage I won't be sure.  All I can remember it was a 6 core i7.  I had a look at 2nd hand motherboards last night (same as used in my PC) and they're going for near AUD $1k!!!!  For a motherboard that went for $350 new back in 2011 lol...of course, I scratched my head as to why they were so expensive 2nd hand, but some research reminded me of the research that I did before buying the parts to build the PC in question - it's compatible with XEON server CPUs.  Must make it highly desirable today I guess...

Basics on why the PC has been in storage...after I built it, I had weird issues where Windows 7 would suddenly refuse to boot, and Windows repair would fail, necessitating a complete reinstall of Windows 7.  After 6 weeks of troubleshooting this, I gave up.  My best mate asked to have a look at it (he's in IT too) and tracked the issue down to the AV that I had installed (Kaspersky) - yes, as soon as he removed the AV, and rebooted the system, it stayed up, without issue, for near 2 years that he had the system, including many reboots to test it etc.  At this stage of my life, I did all my computer stuff on my aforementioned MBP mid 2009 laptop, and gaming on the PS3 and then PS4.  I had no real need for this monster PC that I'd built and was not doing as much macrophotography due to a worsening lower back injury.  My mate ended up giving the PC back to me around 3 years later, but I had zero issues with him using it.  After I got it back, I used it for around 2 months, before my mate's virtual rig started having issues and he asked to borrow my PSU to test his rig.  I ripped out my PSU and he had it for a week and then returned it to me.  Unfortunately, at this stage, the PSU was in its box, on my floor, when my place got flooded very early One morning.  I will admit that I have been too scared/depressed to check the PSU since then...worse, I stored the PC tower, in my dad's old car for near 2 years.  Unbeknownst to me, we had lots of mice living in said car...so here's this juicy PC tower, with a huge hole where the PSU went, for mice to get into...again, I have been too scared to  check the motherboard for rust/chewed cables/oddly coloured resistors or doming caps.  I have put it off time and time again, as it's too depressing to think about a near 2k (parts cost) monster PC at the time, hardly used, being ruined by mice...

With that said, I'm starting to build up courage to check the PC in question out and see if it is damaged, as I want to use said PC for processing in PixInsight, which is not really possible with my MBP (2009) or MBP (mid 2012) laptops.  Worst case scenario, I have an old Q6700 with 8GB ram that I can always fall back on that I know is undamaged. 

I had a spell for Jan 2016 to April 2019 unemployed, so money was non-existent, but I am working now.  The problem is funds are still tight and being able to lash out on computer parts is a slow and timely process of saving money for months at a time, even for cheaper stuff. 

I'd like to, if possible, get the old PC working, and take advantage, as much as possible, of newer tech to buff it and improve performance.  Barring damage to it of course.  After some research on my own, I think a hardware RAID setup with modern Sata SSDs in parallel RAID 0 operation is probably my best bet to improve performance.  NVME will be limited I suspect as a quick search couldn't find any NVME raid controller cards and I'm leery of software raid.  Of course, if the PC is damaged, I'm stuffed.  Building a new PC would require motherboard, ram, nvme SSD and cpu and take me a year or more of saving to afford.  And that's a non fancy i7, which won't be super in performance compared to the higher end, and more costly CPUs on the market, which are out of my reach unless I win lotto!!!!

Sorry for such a long post...

edit: after a long research session tonight, if I need a new PC, the best value for money vs bang is the Intel i7 9700k and Gigabyte Z390 Aorus Master motherboard...a grand for the pair, but double the performance of my old Intel CPU from 2011 (which is a 4820k, not too bad considering, 4 core not 6 core like I remembered). 

[Juan Conejero]

I have a 7 year old computer (from memory, it's running an Intel 6 core i7), 64GB DDR3 ram.  It's in storage at the moment and hasn't been used for several years (long story!).  Running Samsung 840 and 850 SSDs.  How well can I expect it to perform?

It depends on the amount and sizes of the images you want to work with in PixInsight. 64 GB of RAM is a very decent amount in general, although a 6th generation core i7 processor may not be up to current standards. In case you already haven't done so, take a look at the official PixInsight Benchmark project, where you can compare the performance of a large number of hardware configurations:

https://pixinsight.com/benchmark/

As for your Samsung SSDs, they should perform pretty well.

How much improvement does DDR4 ram offer over the older DDR3 standard?

There are significant bandwidth differences, although normally not as important as advertised in the global picture of things, IMO.

How much of an improvement does 128GB offer over 64GB RAM in general performance

That depends basically on the number of images and their dimensions, and on whether you want to perform other highly demanding tasks on your machine while you run a large image preprocessing task, for example. 128 GB allow you to create a large RAM drive, or to use your RAM dynamically on Linux thanks to the tmpfs filesystem, which is great for storage of swap image data (the temporary data used by PixInsight to store unlimited undo/redo image states, among other critical data). With a lot of RAM and a powerful processor, you can execute several instances of PixInsight to run several large preprocessing tasks simultaneously.

PixInsight recommends (can't remember which page) "parallel processing".  What is that?  Is it similar to RAID

This refers to the possibility of reading and writing temporary working data, as the swap files used to store undo/redo states, by means of multiple concurrent threads. With SSDs, where there are no moving mechanical elements, this offers a huge advantage over traditional rotational drives. As you can see on the benchmark page, achieving transfer rates in the 5-8 GB/s range is quite easy on Linux with current hardware. With these transfer rates, you can work with very large images and enjoy real-time undo/redo operations.

To enable parallel swap file storage in PixInsight:

- Select Edit > Global Preferences from the main menu.

- On the Preferences window, select the Directories and Network item on the left panel.

- Select more than one swap storage directory, including directories on different physical storage units as an option. Read the tooltip information for the "Swap storage directories" list for extended information.

- Click Apply Global, or press F6 to apply the changes.

Are there any performance gains from running PixInsight natively in GNU/Linux rather than Windows

Yes, absolutely. Linux is the reference implementation of PixInsight, where you'll get by far the best user experience in terms of stability and performance. I strongly recommend Kubuntu 18.04 LTS as the best platform for current versions of PixInsight.

I'd love an AMD threadripper 3970x

The AMD Ryzen 9 3950X, or the 9 3900X, are excellent options in terms of price vs performance. You can check this easily on the benchmarks page.

[dpastern]

Thank you very much Juan.

My i7 benchmarks at 9700 approximately, which is still pretty reasonable.  So, for now, I think I'll just go with the old PC.  From there, I'll try and save money as possible for a Ryzen 9 3900x + motherboard and DDR4 RAM.  It seems to be the best bang for buck at the moment, but this many change of course by the time I have the $$$ saved.  I'll just have to wait and see.

I think my current best bang for buck will be a NVME adapter card + some Samsung EVO 930 pros running in RAID 0.

Thank you very much for the information on turning on parallel processing in PixInsight.  Stupid question - do I need to have PixInsight installed onto the parallel drives, or can it be on the main drive, and just point the parallel processing drives inside the software?  And of course, the data.  Can that be stored anywhere, or does it have to be on the parallel drives?  The way I understand it, the parallel drives act as a tempfs for the application to process the data and I should be able to have the application and data stored on a standard SSD rather than the NVME based parallel optimised SSD drives that I eventually intend to get.

Linux it is.  It's my preferred operating system anyway.  I really dislike Kubuntu/Ubuntu though, so it'll probably be Debian (stable) providing there's no issues with gnu-dev tools/glibc etc.  It's been a few years, but I ran Debian as my sole operating system (no dual booting) from 2001 to 2006, with a small period of time running Libranet inside that time frame (Debian stable based anyway). 

What are your thoughts on software vs hardware RAID and the impact on PixInsight performance?  There's probably very little in it, making it inefficient $$$ wise to consider the hardware option. 

Cheers,

Dave

[Pompey Monkey]

With 64 GB of ram and a six-core processor, The best speed up you could probably get is by creating a 48 GB RAM-drive and directing the PI swap files to there.

This really helps speed up any PI functions that have been parallelised.

[dpastern]

Yeah, I agree.  Unfortunately, with an older computer such as mine, no NVME ports.  PCI-E ports are taken up by 2 GPUs in SLI mode, so I'd have to drop gaming performance and buy a NVME multi-port card that'll work with my older motherboard and doesn't require bifurcation (and that's if the BiOS updates for my motherboard will allow it to see NVME of course, which it may not). 

A cheaper, easier and more compatible option will be to buy another SATA SSD, partition it into 2 or 3 swaps, and use that.  Won't have the throughput of NVME, but should see some improvements over the stock PC setup that I currently have.  RAM is fine @ 64GB (maximum for the board) and the CPU, despite being old now, is still reasonably decent in throughput it seems  Not modern fast, but not slow either. 

Ultimately, my long-term goal will be a modern PC - Ryzen 9 3900x, suitable DDR4 RAM, NVME drivers (3 of them, 2 or 3 partitions each).  It'd be a dedicated PI rig, with my current machine good enough for general usage and any gaming that I might do on it. 

[Pierre]

Lots of great info in the previous answers. The future of computing is too flexible to keep up.
Anyways I am still using a old HP Z800. It has 2 Zeon Hex core with 96 GB of ram. I use Ubuntu for the OS and set up small 4 GB Ramdisk per processor. I do have SSD drives in the box but not used for swap files. Even if it is an antique compared to modern stuff, I am not ready to hand out a few thousand for a performance increase that is only needed a few times a month. I have better uses for that money.

Read through and page 2-3 have lots of info.
https://www.cloudynights.com/topic/563965-disappointing-performance-of-pixinsight-on-my-laptop/page-2

[dpastern]

I've been pricing a Ryzen 7 2700+ 8 core, 16 thread CPU based system, and it's kinda reasonable (under a grand AUD for all parts).

Your system is more powerful than mine!

[Pierre]

Be sure to turn on all your processors in the PI settings as mentioned in the cloudynights topic.

“I've controlled number of cores available for PI via Edit - Preferences - Parallel Processing and Threads - Maximum number of processors used parameter. All other "Enable.." options at that preference sheet has been set to ON, priority was set to "Time Critical".”

Also try to set up a Ramdisk per processor. Modern versions of Windows will make this difficult as I found out.
Pierre

[dpastern]

Ooh, I had missed that thread on CN (forgot to read it when you linked to it last).  Well read and learn.

Merci!

[dpastern]

One more question - with regards to the tempfs drive, do I just point PixInsight to the drive the required amount of times, or should I partition the actual drive X amount of times (depending on the number of cores on my intended CPU) and each drive be referenced in PI? 

Cheers,

Dave

[Pierre]

You have to make swap files and tell PI where they are. I will attempt to answer better later as I am off to work.
One swap file per drive. Initially I simply called the swap file “PI swap” Having one file per drive did help but ramdisks was the best way to go. Another user has a killer system and with 16 processors he has 32 2 GB ramdisks.
Pierre

[Pierre]

test

[Pierre]

I tried to post a small image of the PI screen showing the settings page but no joy. I did not see any notice of a moratorium on posting images.
pierre