Bulldozer for Servers: Testing AMD's "Interlagos" Opteron 6200 Series
by Johan De Gelas on November 15, 2011 5:09 PM ESTIntroducing AMD's Opteron 6200 Series
When virtualization started to get popular (ca. 2005-2007), there was a fear that this might slow the server market down. Now several years later, the server market has rarely disappointed and continues to grow. For example, IDC reported a 12% increase in revenue when comparing Q1 2010 and Q1 2011. The server market in total accounted for $12 billion revenue and almost two million shipments in Q1 2011, and while the best desktop CPUs generally sell for $300, server chips typically start at $500 and can reach prices of over $3000. With the high-end desktop market shrinking to become a niche for hardcore enthusiasts--helped by the fact that moderate systems from several years back continue to run most tasks well--the enterprise market is very attractive.
Unfortunately for AMD, their share of the lucrative server market has fallen to a very low percentage (4.9%) according IDC's report early this year (some report 6-7%). It is time for something new and better from AMD, and it seems that the Bulldozer architecture is AMD's most server-centric CPU architecture ever. We quote Chuck Moore, Chief Architect AMD:
By having the shared architecture, reducing the size and sharing things that aren’t commonly used in their peak capacity in server workloads, “Bulldozer” is actually very well aligned with server workloads now and on into the future. In fact, a great deal of the trade-offs in Bulldozer were made on behalf of servers, and not just one type of workload, but a diversity of workloads.
This alginment with server workloads can also be found in the specs:
|
Opteron 6200 "Interlagos" |
Opteron 6100 "Magny-cours" |
Xeon 5600 "Westmere" |
|
| Cores (Modules)/Threads | 8/16 | 12/12 | 6/12 |
| L1 Instructions | 8x 64 KB 2-way | 12x 64 KB 2-way | 6x 32 KB 4-way |
| L1 Data | 16x 16 KB 4-way | 12x 64 KB 2-way | 6x 32 KB 4-way |
| L2 Cache | 4x 2MB | 12x 0.5MB | 6x 256 KB |
| L3 Cache | 2x 8MB | 2x 6MB | 12MB |
| Memory Bandwidth | 51.2GB/s | 42.6GB/s | 32GB/s |
| IMC Clock Speed | 2GHz | 1.8GHz | 2GHz |
| Interconnect | 4x HT 3.1 (6.4 GT/s) | 4x HT 3.1 (6.4 GT/s) | 2x QPI (4.8-6.4 GT/s) |
The new Opteron has loads of cache, faster access to memory and more threads than ever. Of course, a good product is more than a well designed microarchitecture with impressive specs on paper. The actual SKUs have to be attractively priced, reach decent clock speeds, and above all offer a good performance/watt ratio. Let us take a look at AMD's newest Opterons and how they are positioned versus Intel's competing Xeons.
| AMD vs. Intel 2-socket SKU Comparison | |||||||||
| Xeon |
Cores/ Threads |
TDP |
Clock (GHz) |
Price | Opteron |
Modules/ Threads |
TDP |
Clock (GHz) |
Price |
| High Performance | High Performance | ||||||||
| X5690 | 6/12 | 130W | 3.46/3.6/3.73 | $1663 | |||||
| X5675 | 6/12 | 95W | 3.06/3.33/3.46 | $1440 | |||||
| X5660 | 6/12 | 95W | 2.8/3.06/3.2 | $1219 | |||||
| X5650 | 6/12 | 95W | 2.66/2.93/3.06 | $996 | 6282 SE | 8/16 | 140W | 2.6/3.0/3.3 | $1019 |
| Midrange | Midrange | ||||||||
| E5649 | 6/12 | 80W | 2.53/2.66/2.8 | $774 | 6276 | 8/16 | 115W | 2.3/2.6/3.2 | $788 |
| E5640 | 4/8 | 80W | 2.66/2.8/2.93 | $774 | |||||
| 6274 | 8/16 | 115W | 2.2/2.5/3.1 | $639 | |||||
| E5645 | 6/12 | 80W | 2.4/2.53/2.66 | $551 | 6272 | 8/16 | 115W | 2.0/2.4/3.0 | $523 |
| 6238 | 6/12 | 115W | 2.6/2.9/3.2 | $455 | |||||
| E5620 | 4/8 | 80W | 2.4/2.53/2.66 | $387 | 6234 | 6/12 | 115W | 2.4/2.7/3.0 | $377 |
| High clock / budget | High clock / budget | ||||||||
| X5647 | 4/8 | 130W | 2.93/3.06/3.2 | $774 | |||||
| E5630 | 4/8 | 80W | 2.53/2.66/2.8 | $551 | 6220 | 4/8 | 115W | 3.0/3.3/3.6 | $455 |
| E5607 | 4/4 | 80W | 2.26 | $276 | 6212 | 4/8 | 115W | 2.6/2.9/3.2 | $266 |
| Power Optimized | Power Optimized | ||||||||
| L5640 | 6/12 | 60W | 2.26/2.4/2.66 | $996 | |||||
| L5630 | 4/8 | 40W | 2.13/2.26/2.4 | $551 | 6262HE | 8/16 | 85W | 1.6/2.1/2.9 | $523 |
The specifications (16 threads, 32MB of cache) and AMD's promises that Interlagos would outperform Magny-cours by a large margin created the impression that the Interlagos Opteron would give the current top Xeons a hard time. However, the newest Opteron cannot reach higher clock speeds than the current Opteron (6276 at 2.3GHz), and AMD positions the Opteron 6276 2.3GHz as an alternative to the Xeon E5649 at 2.53GHz. As the latter has a lower TDP, it is clear that the newest Opteron has to outperform this Xeon by a decent margin. In fact most server buyers expect a price/performance bonus from AMD, so the Opteron 6276 needs to perform roughly at the level of the X5650 to gain the interest of IT customers.
Judging from the current positioning, the high-end is a lost cause for now. First, AMD needs a 140W TDP chip to compete with the slower parts of Intel's high-end armada. Second, Sandy Bridge EP is coming out in the next quarter--we've already seen the desktop Sandy Bridge-E launch, and adding two more cores (four more threads) for the server version will only increase the performance potential. The Sandy Bridge cores have proven to be faster than Westmere cores, and the new Xeon E5 will have eight of them. Clock speeds will be a bit lower (2.0-2.5GHz), but we can safely assume that the new Xeon E5 will outperform its older brother by a noticeable margin and make it even harder for the new Opteron to compete in the higher end of the 2P market.
At the low-end, we see some interesting offerings from AMD. Our impression is that the 6212 at 2.6-2.9GHz is very likely to offer a better performance per dollar ratio than the low-end Xeons E560x that lack Hyper-Threading and turbo support.
Okay, we've done enough analyzing of paper specs; let's get to the hardware and the benchmarks. Before we do that, we'll elaborate a bit on what a server centric architecture should look like. What makes server applications tick?
Facebook
Twitter
RSS
106 Comments
View All Comments
zappb - Tuesday, November 29, 2011 - link
Thanks Johan for the ungodly amount of time you and your team spent on this review, also thanks to all contributors to the comments which was very useful to get more context to someone like myself who is not very up to speed with server tech.The 45 Watt and 65 watt Opterons not mentioned on the front page of the article (but mentioned in the comments - are these based on Interlagos?)
To me it looks like a big win for AMD - and these benchmarks are are not even optimised for the architecture (Linux kernel 3 was not used - can't wait to see updated benchmarks, something like FreeBSD or when we get an updated scheduler for the windows server OS's...should make a big difference.
Really low idle power consumption is nice, and Im planning to pick one of these up (for home use) to play around with FreeBSD, vm's, etc...just for training purposes,
The other point about Intel's sandybridge Xeons, these are just going to be 8 core 3960x right? Which may not change the current server landscape very much depending on their prices.
JWesterby - Friday, February 10, 2012 - link
Respect is due, Johan! You did a very useful review under significant limitations. The very best part is to point an unbiased light at a damned interesting CPU. There is an important "next step," which I will address shortly.As always, just the mention of AMD brings out hysterical attacks. One would think we were talking about Stem Cell research!! There is no real discussion -- it's pitchforks, lit torches, and a stake ready for poor Johan and anyone else ready and willing to consider the mere possibility that AMD have produced worthy technology!!
Computer technology - doing it, anyway - has changed. It's become ALL about the bloody money, and the "culture" of the people doing technology has also changed -- it has become much more cut-throat, there is far less collegiality, and the number of people willing to take risks on projects has become really uncommon. Qualified people doing serious technology just because they can is uncommon.
There is no end to posers (including some on this board), Machiavellian Fortune 500 IT managers, and "Project Managers" who are clueless (there ARE some great IT managers and wonderful PM's but their numbers are shrinking). My hat to those in Open Source - they are the Last Bastion of decency for the sake of decency, and technology merely for the joy of doing it !!
"Back in the day" people seemed really into the technology, solving difficult problems, and making good things happen. There was truly a culture. For example not taking a moment to help someone on the team or otherwise made you a jerk. Development was a craft or an art, and we were all in it together. We are loosing that, and it's become more dog-eat-dog with a kind of mean spirit. What a shame. Many of the comments here are perfect examples -- people who would rather burn down the temple than give a new and challenging technology a good think.
Personally I can't wait to get my hands on a couple of AMD's new CPU's, build a decent server, and carefully work out the issues with patience. These new Opterons are like a whole new tech that may be the beginning of all new territory.
My passion and some professional work is coding at the back end in C/C++ and I'm just beginning to understand CUDA and using GPU's to beef up parallel code. My work is all around (big) data warehousing, cutting edge columnar databases, almost everything running virtual, all the way through to analytics on BI platforms. I do all of that both on MS Server 2008, Solaris and FreeBSD. All that is a perfect environment to test AMD's "new territory."
Probably worth a blog at some point because these processors are new territory and using them well will take some work just keeping track of all the small things that shake out. That's the "next step" that this and other reviews require to really understand AMD's Bulldozers. Doing that well, if AMD is right with these chips, means being able to build some great back-end servers at a much more approachable price; more importantly without paying an "Intel" tax, and in the end having two strong vendors and thereby more freedom to make the best choice for the requirement.
PhotoPrint - Sunday, December 25, 2011 - link
you should make fair comparison at the same price range!lts like comparing GTX580 VS AMD RADEON 6950!
g101 - Wednesday, January 11, 2012 - link
Wow, anad let the truth about bulldozer leak out.ppennisi - Wednesday, March 7, 2012 - link
To obtain maximum performance from my Dell R715 server equipped with dual Interlagos processor I had to DISABLE C1E in the BIOS.Under VMware the machines performance changed completely, almost doubled in performance.
Maybe you should try it.
anti_shill - Monday, April 2, 2012 - link
Here's a more accurate reflection of Bulldozer/ interlagos performance, untainted by intel ad bucks...http://www.phoronix.com/scan.php?page=article&...
But if u really want to see what the true story is, have a look at AMD's stock price lately, and their server wins. They absolutely smoke intel on virtualization, and anything that requires a lot of threads. It's not even close. That would be the reason this review pits Interlagos against an Intel processor that costs twice as much.