Threadripper Ragnarok

https://www.youtube.com/watch?v=jHud34FR3G0

AMD and Intel have simultaneously launched new CPUs today: Intel is launching the 10980XE (18 core); the 10940X (14 core); the 10920X (12 core); and the 10900X (10 core)  on the existing X299 platform (though we will see many motherboard refreshes!). AMD is launching just two new CPUs today -- the Threadripper 3960X (24-core) and the Threadripper 3970X (32-core) on the new high-end TRX40 motherboard platform. Allthough the 16-core 3950X also enters retail availability today, we reviewed that chip previously. 

 I use the word “new” somewhat loosely when referring to the new Intel CPUs, such as the 10980XE,  since it is, essentially, a refresh version of the 9980XE (which was, in turn, a refresh of the 7980XE). The most compelling feature this 18-core CPU is the price -- $979 – which is about half the price of the 18-core 9980XE, last generation. 

I will have more to say on the new X299 CPUs toward the end of this review, and in a separate review.

For the reviews today we have the 10980XE, the Threadripper 3960X and 3970X. 

The 3950X, which I reviewed about a week ago, has launched into retail availability today as well. This is a solid CPU that performs mostly on par with the 10980XE, but the CPU and required platform (motherboard, memory, etc) are a lower-cost platform. 

Generally, though, the 3900X remains the best overall value vs the 3950X if you specifically need 16-cores vs 12-cores (which is typically relegated to rendering type workloads).

For buyers interested in something higher end – higher end motherboards, more & faster PCIe Peripherals, more memory channels and memory capacity support, there are X299 and Threadripper options. At the very highest end from Intel, there is also the C621 chipset and Xeon W – which we will also talk about! 

What’s my initial impression of the new Threadripper CPUs? This: 

Wow! There simply hasn’t been a generational leap in desktop CPU performance this significant in over 10 years.

For the first time we are getting "desktop” CPUs -- 24 and 32 core -- that have no caveats; no regressions; no real performance anomalies. And that can also game. Whether you want to run one thread or 32, you’ll get consistent reliable performance. 

As of today, they are the fastest desktop CPUs you can get, and generally by a wide margin (especially in multi-core workloads).

To be sure, like the 3950X, I don’t recommend these CPUs for gaming. They will not help you in gaming and CPUs like Intel’s 8-core 9900KS offer marginally better gaming performance in some limited scenarios. However, unlike the last 3 generations of X299-compatible Intel CPUs, there is no gaming performance regression when moving from “desktop” class processors to these “high-end” processors from AMD. 

AMD has done a lot of work on their single/lightly threaded topology. They have explored several tricks with CPU topology and thread scheduling order signaling to the host operating system to ensure that, in lightly threaded scenarios, each busy thread essentially gets its own CCX with 16mb of cache just for that single thread.

(Note this is not from a press board but one of the other vendor boards. More work on this is pending, and this type of thread ordering may not be necessary on OSs that are aware of "preferred ordering" of core scheduling. )

The CPU topology is also now a 1-node UMA – Uniform Memory Access node. Far more work and optimization has gone into Windows 10 to support “single-processor” systems than the relatively funky 4-NUMA-node 2990WX 32-core from last generation, which means for this generation programmers don’t have to do nearly as much work to take advantage of all that compute horsepower in these CPUs.

It is still possible to configure the 3000 series CPUs to report 2 or 4 NUMA nodes, which could be important to researchers and scientists with software tuned in very specific ways, and the memory latency is a bit improved with two dual-channel nodes instead of one “quad-channel” node. 

The boost speed of these CPUs is up to 4550mhz and, unlike Ryzen desktop CPUs, I regularly saw that boost speed sustained for extended periods of time in single/lightly threaded scenarios.

I also rotated through all 64 threads in the system (in the case of the 32-core TR CPU) with a 1-thread Cinebench R20 job and noted that every core could boost, and sustain, around 4400mhz (except for the 2 cores that would boost and sustain 4550mhz). This is *great* as the base clocks are 3.8ghz for the 24-core and 3.7ghz for the 32-core.

More benchmarks in a moment.

There are 88 PCIe 4 lanes on this platform; 72 usable. The connection to the PCH is now via PCIe 4.0 x8 connection – something I’ve been eagerly anticipating for many years now. In terms of I/O – Threadripper shreds. Unfortunately, it does mean the X399 motherboards of first and second gen Threadripper are incompatible. 

How can the 32-core 3970X be so much faster than the 32-core 2990WX just a generation before? They’re both 32-cores, right? The move from Zen+ to Zen 2 is a large part of it but “software optimization” is just as large a part of it. Take, for example, Indigo Bench (a 3d rendering program).

The performance of the 2990WX with the Indigo Renderer on the Windows platform was about 1.6-2.0 as reported in many places, including Level1, for the “bedroom scene” render. On Linux, however, the same 2990WX doing the same job could score about 3.2 on identical hardware on an identical setup.

The 3970WX scores a little over 5.30 now on both Linux and Windows platforms, best case scenario. Even today, the 2990WX use-case on Windows hasn’t really been addressed and the performance is still abysmal. (Aside: Bad news for Intel as their 56-core server CPU is a similar topology to the 2990WX.)

AMD has been working closely with Microsoft, however, and there have been improvements in single/lightly threaded handling for the AMD platform from Microsoft in their operating systems. Like the GHz barrier and the integrated memory controller, AMD’s adoption of chiplets heralds an industry-wide trend of the move to chiplets. Chiplets are here to stay and will be a feature of desktop CPUs for some time, I’d wager. 

Phoronix Benchmarks: https://openbenchmarking.org/result/1911252-AS-1911042HU36

Early Adopter Tax

Whether you want to characterize it as the worst luck ever or the best at finding flaws, I usually find edge cases no one else cares about: IOMMU, Virtualization bugs, PCIe Errors, RDRand Linux Bugs, etc on just this platform alone. 

There is, currently, a bug with recent Linux kernels where the machine will panic as it tries to create a device in sysfs for the CPU machine-check endpoint and it is fixable with a one-line change to the kernel. If you’re in a pinch, though, adding mce=off to the kernel line will work around the issue until you can fix up your kernel.

Aside from that, this launch has been essentially flawless. I do not say that lightly. These CPUs (and motherboards) have been essentially flawless.

AMD has dramatically improved their QC/Testing process for the launch of these CPUs (even over the Ryzen 3000 desktop CPU launch) and that kind of thing is incredibly important to the kind of end-user who would shell out $2000US for AMD’s current flagship CPU.

Mind the (Pricing) Gap

Everyone was keen to point out to AMD the $700 price gulf between the $750US 3950X and the $1400US 3960X.

It surely must be part of Intel’s strategy to “right size” the 10980XE in-between those two prices – but if that was their strategy it might not work out for them since the 3950X matches or exceeds the 10980XE in a lot of common scenarios.

AMD has announced they intend to continue producing 2000-series Threadripper CPUs at discounted pricing to fit in the pricing gap between the 3950X and the 3960X. Microcenter here in the US is already selling the Threadripper 2950X for less than $600US, and that 16-core CPU is a trend-setter in its own right. 

Test Systems

We used two test systems configured close to identical and sharing the same GPU between them for all of our testing.

Corsair MP600 1tb NVMe 
MSI Creator TRX40 
Asus Zenith II Extreme TRX40 
64gb (4*16gb) Corsair Dominator (DDR4-3600)

The Linux config featured a Liquid HHHL 4-up NVMe Raid. 

These were provided by AMD; thanks AMD!!

We used both CPUs in both boards. There is a fair amount of variability in performance depending on a number of factors – for example Cinebench R20 scoring on the 32-core could vary between about 16,800 and 17,900 depending on cooler used and UEFI settings. The ROG Zenith board seemed to push CPUs the hardest – thermals were a little worse, but performance was a little better. Careful measurement of “CPU Core Package” showed somewhat higher power utilization but the at-the-wall power draw of both the MSI and Asus system were around 425 watts in the worst-case scenario.

Benchmarks

UEFIs were at vendor defaults except for enabling DOCP or XMP. All memory was tested at DDR4-3600 but the official supported speed for 64gb is 3200 (for 4 sticks) and 2933 (for 8 sticks).

 
*Any speeds beyond these are technically an overclock. Technically my DDR4-3600 setup was an overclock, but it was rock solid for me. YMMV.

I also tested GSkill TridentZ 8x16gb DDR4-3200 without issue (128gb total). It was necessary to bump the voltage to 1.37v but was stable with an 18-hour memory test.

What about ECC support? That will be in our motherboard review but so far It seems to be working on several of the motherboards I have in to test. It is up to the motherboard vendor to qualify and support it, in general.

What about a high-end Xeon? Can that match this new Threadripper? What the pricing from AMD? 

The 10980XE just isn’t an answer to current-gen Threadripper. If you want more PCIe Lanes (no PCIe 4.0, though), or more memory channels it might make sense to move to the 10980XE from the 3950X. There are some X299 refresh motherboards that cost less than and are nicer with more features than the most expensive AM4 X570 motherboards. 

For me personally, it is a bit shocking that Intel didn’t move a socket 3647 Xeon into the ~$2000-ish price point – so I used a Xeon W3175 with a high-end custom water-cooling loop w/an aggressive overclock as a stand-in for our benchmarking here. I was a bit surprised when the 28-core Xeon and 24-core Threadripper were basically at parity with one another, even with the Intel CPU being heavily overclocked.

I managed to get access to a W3275 Xeon, the “upgrade” to the W3175X – new silicon, new stepping and some hardware mitigations in place – but it was slower than the W3175X by the time you factor in the overclock. It was the 7980XE OC vs 10980Xe all over again – except the W3275 is locked (no overclocking). Neither of these CPUs are a match for either Threadripper CPU, unfortunately.

These 28-core CPUs from Intel will set you back $3000US and $4500US, respectively. Suddenly the $1400US and $2000US price tags on the CPUs from AMD are much more palatable – a downright bargain, in fact.

Intel is a smart company with a lot of talented people working there. Competition is alive and well. I think Intel genuinely believed that the laws of Physics and materials science were at a limit when they began to stall around 14nm. These CPUs from AMD clearly demonstrate there are still more opportunities for intel to upgrade their own foundries and that they should have pushed harder. 

The last time Intel was in this position, it (eventually) led to a renaissance of great computational products. Here’s to hoping that happens again!  

What now?

Buy, buy, buy (obviously). No, seriously -- If you need to have the fastest desktop CPU on planet earth, or your work demands it, then the new Threadripper CPUS are objectively the best choice. 

I do not say this lightly: We are witness to the single biggest generational leap of desktop processor performance in more than a decade.

If you want to save $600, the 24-core Threadripper is just as fast until you hit something that could use 25 or more threads – so not much except creative and rendering workloads. They both have the same “280W” TDP power envelope, but the same cache, so you will see slightly better lightly-threaded performance. 

For the “developer power user” workstation with a ton of virtual machines I would recommend either the 3900x or the 24-core Threadripper. The TR will lend itself to more memory, more i/o, more VMs and more background process in general (at double the cores and threads of the 3900x) but it’s also almost 3x the cost. If a 3900x won’t do it for you, for whatever reason, it’s hard for me to imagine anyone that would be disappointed with this purchase. 

The scariest thing of all? AMD is showing no signs of slowing down. The 3990X – a 64-core monster – will be here in early 2020. If my experiences with 256 threads on Windows Server are any indication, I strongly suspect Microsoft and AMD are going to need to hunker down to iron out Windows issues before that CPU is released.

Linux users, however, aren’t likely to have any major issues adopting this platform right away.

Linux Channel Vid:

https://www.youtube.com/watch?v=oRgmq9f6VBY