Bulldozer for Servers: Testing AMD's "Interlagos" Opteron 6200 Series
by Johan De Gelas on November 15, 2011 5:09 PM ESTIntroducing AMD's Opteron 6200 Series
When virtualization started to get popular (ca. 2005-2007), there was a fear that this might slow the server market down. Now several years later, the server market has rarely disappointed and continues to grow. For example, IDC reported a 12% increase in revenue when comparing Q1 2010 and Q1 2011. The server market in total accounted for $12 billion revenue and almost two million shipments in Q1 2011, and while the best desktop CPUs generally sell for $300, server chips typically start at $500 and can reach prices of over $3000. With the high-end desktop market shrinking to become a niche for hardcore enthusiasts--helped by the fact that moderate systems from several years back continue to run most tasks well--the enterprise market is very attractive.
Unfortunately for AMD, their share of the lucrative server market has fallen to a very low percentage (4.9%) according IDC's report early this year (some report 6-7%). It is time for something new and better from AMD, and it seems that the Bulldozer architecture is AMD's most server-centric CPU architecture ever. We quote Chuck Moore, Chief Architect AMD:
By having the shared architecture, reducing the size and sharing things that aren’t commonly used in their peak capacity in server workloads, “Bulldozer” is actually very well aligned with server workloads now and on into the future. In fact, a great deal of the trade-offs in Bulldozer were made on behalf of servers, and not just one type of workload, but a diversity of workloads.
This alginment with server workloads can also be found in the specs:
|
Opteron 6200 "Interlagos" |
Opteron 6100 "Magny-cours" |
Xeon 5600 "Westmere" |
|
| Cores (Modules)/Threads | 8/16 | 12/12 | 6/12 |
| L1 Instructions | 8x 64 KB 2-way | 12x 64 KB 2-way | 6x 32 KB 4-way |
| L1 Data | 16x 16 KB 4-way | 12x 64 KB 2-way | 6x 32 KB 4-way |
| L2 Cache | 4x 2MB | 12x 0.5MB | 6x 256 KB |
| L3 Cache | 2x 8MB | 2x 6MB | 12MB |
| Memory Bandwidth | 51.2GB/s | 42.6GB/s | 32GB/s |
| IMC Clock Speed | 2GHz | 1.8GHz | 2GHz |
| Interconnect | 4x HT 3.1 (6.4 GT/s) | 4x HT 3.1 (6.4 GT/s) | 2x QPI (4.8-6.4 GT/s) |
The new Opteron has loads of cache, faster access to memory and more threads than ever. Of course, a good product is more than a well designed microarchitecture with impressive specs on paper. The actual SKUs have to be attractively priced, reach decent clock speeds, and above all offer a good performance/watt ratio. Let us take a look at AMD's newest Opterons and how they are positioned versus Intel's competing Xeons.
| AMD vs. Intel 2-socket SKU Comparison | |||||||||
| Xeon |
Cores/ Threads |
TDP |
Clock (GHz) |
Price | Opteron |
Modules/ Threads |
TDP |
Clock (GHz) |
Price |
| High Performance | High Performance | ||||||||
| X5690 | 6/12 | 130W | 3.46/3.6/3.73 | $1663 | |||||
| X5675 | 6/12 | 95W | 3.06/3.33/3.46 | $1440 | |||||
| X5660 | 6/12 | 95W | 2.8/3.06/3.2 | $1219 | |||||
| X5650 | 6/12 | 95W | 2.66/2.93/3.06 | $996 | 6282 SE | 8/16 | 140W | 2.6/3.0/3.3 | $1019 |
| Midrange | Midrange | ||||||||
| E5649 | 6/12 | 80W | 2.53/2.66/2.8 | $774 | 6276 | 8/16 | 115W | 2.3/2.6/3.2 | $788 |
| E5640 | 4/8 | 80W | 2.66/2.8/2.93 | $774 | |||||
| 6274 | 8/16 | 115W | 2.2/2.5/3.1 | $639 | |||||
| E5645 | 6/12 | 80W | 2.4/2.53/2.66 | $551 | 6272 | 8/16 | 115W | 2.0/2.4/3.0 | $523 |
| 6238 | 6/12 | 115W | 2.6/2.9/3.2 | $455 | |||||
| E5620 | 4/8 | 80W | 2.4/2.53/2.66 | $387 | 6234 | 6/12 | 115W | 2.4/2.7/3.0 | $377 |
| High clock / budget | High clock / budget | ||||||||
| X5647 | 4/8 | 130W | 2.93/3.06/3.2 | $774 | |||||
| E5630 | 4/8 | 80W | 2.53/2.66/2.8 | $551 | 6220 | 4/8 | 115W | 3.0/3.3/3.6 | $455 |
| E5607 | 4/4 | 80W | 2.26 | $276 | 6212 | 4/8 | 115W | 2.6/2.9/3.2 | $266 |
| Power Optimized | Power Optimized | ||||||||
| L5640 | 6/12 | 60W | 2.26/2.4/2.66 | $996 | |||||
| L5630 | 4/8 | 40W | 2.13/2.26/2.4 | $551 | 6262HE | 8/16 | 85W | 1.6/2.1/2.9 | $523 |
The specifications (16 threads, 32MB of cache) and AMD's promises that Interlagos would outperform Magny-cours by a large margin created the impression that the Interlagos Opteron would give the current top Xeons a hard time. However, the newest Opteron cannot reach higher clock speeds than the current Opteron (6276 at 2.3GHz), and AMD positions the Opteron 6276 2.3GHz as an alternative to the Xeon E5649 at 2.53GHz. As the latter has a lower TDP, it is clear that the newest Opteron has to outperform this Xeon by a decent margin. In fact most server buyers expect a price/performance bonus from AMD, so the Opteron 6276 needs to perform roughly at the level of the X5650 to gain the interest of IT customers.
Judging from the current positioning, the high-end is a lost cause for now. First, AMD needs a 140W TDP chip to compete with the slower parts of Intel's high-end armada. Second, Sandy Bridge EP is coming out in the next quarter--we've already seen the desktop Sandy Bridge-E launch, and adding two more cores (four more threads) for the server version will only increase the performance potential. The Sandy Bridge cores have proven to be faster than Westmere cores, and the new Xeon E5 will have eight of them. Clock speeds will be a bit lower (2.0-2.5GHz), but we can safely assume that the new Xeon E5 will outperform its older brother by a noticeable margin and make it even harder for the new Opteron to compete in the higher end of the 2P market.
At the low-end, we see some interesting offerings from AMD. Our impression is that the 6212 at 2.6-2.9GHz is very likely to offer a better performance per dollar ratio than the low-end Xeons E560x that lack Hyper-Threading and turbo support.
Okay, we've done enough analyzing of paper specs; let's get to the hardware and the benchmarks. Before we do that, we'll elaborate a bit on what a server centric architecture should look like. What makes server applications tick?
Facebook
Twitter
RSS
106 Comments
View All Comments
veri745 - Tuesday, November 15, 2011 - link
Shouldn't there be 8 x 2MB L2 for Interlagos instead of just 4x?ClagMaster - Tuesday, November 15, 2011 - link
A core this complex in my opinion has not been optimized to its fullest potential.Expect better performance when AMD introduces later steppings of this core with regard to power consumption and higher clock frequencies.
I have seen this in earlier AMD and Intel Cores, this new core will be the same.
C300fans - Tuesday, November 15, 2011 - link
1x i7 3960x or 2x Interlagos 6272? It is up to you. Money cow.tech6 - Tuesday, November 15, 2011 - link
We have a bunch of 6100 in our data center and the performance has been disappointing. They do no better in single thread performance than old 73xx series Xeons. While this is OK for non-interactive stuff, it really isn't good enough for much else. These results just seem to confirm that the Bulldozer series of processors is over-hyped and that AMD is in danger of becoming irrelevant in the server, mobile and desktop market.mino - Wednesday, November 16, 2011 - link
Actually, for interactive stuff (read VDI/Citrix/containers) core counts rule the roost.duploxxx - Thursday, November 17, 2011 - link
this is exactly what should be fixed now with the turbo when set correct, btw the 73xx series were not that bad on single thread performance, it was wide scale virtualization and IO throughput which was awefull one these systems.alpha754293 - Tuesday, November 15, 2011 - link
"Let us first discuss the virtualization scene, the most important market." Yea, I don't know about that.Considering that they've already shipped like some half-a-million cores to the leading supercomputers of the world; where some of them are doing major processor upgrades with this new release; I wouldn't necessarily say that it's the most IMPORTANT market. Important, yes. But MOST important...I dunno.
Looking forward to more HPC benchmark results.
Also, you might have to play with thread schedule/process affinity (masks) to make it work right.
See the Techreport article.
JohanAnandtech - Thursday, November 17, 2011 - link
Are you talking about the Euler3D benchmark?And yes, by any metric (revenue, servers sold) the virtualization market is the most important one for servers. Depending on the report 60 to 80% of the servers are bought to be virtualized.
alpha754293 - Tuesday, November 15, 2011 - link
Folks: chip-multithreading (CMT) is nothing new.I would explain it this way: it is the physical, hardware manifestation of simultaneous multi-threading (SMT). Intel's HTT is SMT.
IBM's POWER (since I think as early as POWER4), Sun/Oracle/UltraDense's Niagara (UltraSPARC T-series), maybe even some of the older Crays were all CMT. (Don't quote me on the Crays though. MIPS died before CMT came out. API WOULD have had it probably IF there had been an EV8).
But the way I see it - remember what a CPU IS: it's a glorified calculator. Nothing else/more.
So, if it can't calculate, then it doesn't really do much good. (And I've yet to see an entirely integer-only program).
Doing integer math is fairly easy and straightforward. Doing floating-point math is a LOT harder. If you check the power consumption while solving a linear algebra equation using Gauss elimination (parallelized or using multiple instances of the solver); I can guarantee you that you will consume more power than if you were trying to run VMs.
So the way I see it, if a CPU is a glorified calculator, then a "core" is where/whatever the FPU is. Everything else is just ancillary and that point.
mino - Wednesday, November 16, 2011 - link
1) Power is NOT CMT, it allways was a VERY(even by RISC standards) wide SMT design.2) Niagara is NOT a CMT. It is interleaved multipthreading with SMT on top.
Bulldozer indeed IS a first of its kind. With all the associated advantages(future scaling) and disadvantages(alfa version).
There is a nice debate somewhere on cpu.arch groups from the original author(think 1990's) of the CMT concept.