I have recently bought a refurb HP Z840 workstation specifically to give me better PI performance. This came with a single Xeon E5-2680v3, 12-core CPU. I bought a 2nd E5-2680v3 for the 2nd socket and added a lot of RAM and two 1TB HP EX920 M.2 NVMe SSDs for my data and PI files. I am still using a SATA SSD for my boot.
Before I installed the 2nd CPU, PI CPU Benchmark was 10,398 with 1.8.7 improving to 11,141 with 1.8.8. Good, 1.8.8 has marginally improved multi-threading performance! This was with 1 CPU, 32GB RAM, and default swap to SATA SSD.
I then installed 4x16GB to the 4x8GB for CPU0, for 96GB of DDR4 ECC SDIMM and the CPU Benchmark improved to 11,651.
After my CPU heatsink arrived, I installed the 2nd matching Xeon E5-2680v3 and installed 4x16GB for CPU1, giving 64GB for CPU1 and 96GB for CPU0. With this the CPU Benchmark improved to 15,405. I was disappointed as I was hoping the performance would come close to 2x improvement instead of the 32% improvement.
PI 1.8.8 then had some updates.
I then tried adding 4x8GB (from my Z440) for CPU1 so both CPUs would have 96GB each. This RAM was however different from CPU0's original 4x8GB, as my Z440 was a Xeon 1650v4, which had 2400mhz, 1Rx8 DDR4 ECC SDIMMs, while the Z840 CPU0 4x8GB were 2133mhz, 1Rx4 DDR4 ECC SDIMMs. This install of mixed Rank RAM was OK as far as the Z840 and OS was concern. With these changes, the CPU Benchmark dropped to 10,702, worse than my 1st results with 1 CPU and 32GB of RAM.
So I took out the 2400mhz, 1Rx8 DDR4 ECC SDIMMs, returning me to 160GB total, and I exactly matched the RAM between the CPUs, 80+80GB. According to HP's documentation, 80+80GB should have better performance than 96+64GB. To my surprise, the CPU Benchmark remained in the single CPU range, at 10,453.
I briefly went back to the unbalanced 96+64GB, and the CPU Benchmark stayed in the single CPU range at 10,442. There must have been something in the update of a few days ago that trashed the CPU performance.
I then added the two 1TB HP EX920 M.2 NVMe SSDs for my data and PI files (include swap), and changed the RAM back to the 80+80GB balance. With this my overall Benchmark improved significantly, only because of the Swap Benchmark going from 2,229 to 5,364. The CPU Benchmark was still in the single CPU range, at 11,734.
When I looked at the Benchmark printout, I noticed that it was only using 24 threads, NOT 48 threads, but it was seeing all 160GB of RAM. So after the updates, PI is either only using a single CPU with multi-threading or using both CPUs in single threading. Windows Task Manager is showing 24-cores and 48-threads.
What is going on?
How much performance increase is to be expected by adding a 2nd matched CPU?
