The OpenPOWER Saga Continues: Can You Get POWER Inside 1U?
by Johan De Gelas on February 24, 2017 8:00 AM ESTBack in April 2016, Tyan has introduced its IBM POWER8-based 1U servers at the OpenPOWER Summit in San Jose, California. The server market is dominated by five OEMs: HPE, Dell, IBM, Lenovo, and Cisco, with ODM Quanta and HPC specialist Supermicro being the challengers. So the fact that Tyan also sells POWER8 servers can hardly be considered a game changing fact. However, Tyan does punch well above its weight. Note that there are only four OpenPOWER POWER8 platforms out there:
- The Wistron's Polaris was the basis for the heavy duty S822LC (see our review here). An improved version, the Polaris Plus was the first Power8 + P100 NVLink, probably the fastest 2U HPC server available (see here).
- The Supermicro "Briggs" which is found in the latest IBM S822LC "For Big Data".
- The Rackspace "Barreleye"
- The Tyan Habanero.
As far as we know, the Supermicro board is only available as an IBM server.
The Rackspace Barreleye is the odd one out: it is 1.25 U (instead of 2U) high, 21 inches wide, and needs the power shelves of "OpenRack". The cool thing is that it is no longer a prototype; it is a server that can actually be bought. This means that those who want to start over and make use of the many advantages of OCP have a real choice between the Intel Xeon and IBM POWER. Such servers - even in low quantities - are available from Penguin Computing (Magna 1015), Mark III systems, and Stack Velocity.
Of course, most datacenters are not switching to OCP racks (yet). If you want a standard 19 inch wide OpenPOWER server, there is a pretty good chance your server is based upon the Tyan Habanero. Habanero was not only the platform in most of the Tyan POWER8 offerings, but it is also the board that can be found inside the IBM S812LC and the Penguin Computing Magna 2001. Of course, the OpenPOWER server market is still a young and small market, but Tyan did have quite a bit of impact.
So when we saw that Tyan made a 1U server based upon this Habanero platform, that caught our eye. The power-hungry POWER8 inside a density optimized form factor? And they feed it with a PSU of "only" 750 W? Remember, most POWER8 servers come with 1000+ Watt PSUs.
OpenPOWER: Survive or Thrive?
Now at this point you may be asking: yet another article about OpenPOWER? Isn't all effort going to waste as IBM's piece of the server market is shrinking and Intel reigns supreme? Why would IBM be able to succeed where so many others have failed?
We are among the first to recognize that Intel still rules the server world, and that there is no alternative to the midrange Xeon E5 when you want the best performance-per-watt available. Furthermore the arrival of servers based upon AMD's Naples CPU will put even more pressure on all non-x86 server CPU alternatives, as it is expected to ship with up to 32 cores, 64 thread, offer 8 DDR4 channels and integrate no less than 128 PCIe lanes. And in the long-run (and perhaps most importantly), it will force Intel to come up with better performance/watt/dollar offerings, ratcheting up the pressure on non-x86 server CPUs even further.
As important as performance-per-watt is, several markets – HPC, Analytics, and AI chief among them – consider absolute performance the most important metric. Wattage has to be kept under control, but that is it. Who cares whether the CPU is consuming 200W or 120W when it is placed in a machine with a Terabyte of RAM and two 300W GPUs?
The fact that Rackspace and Google have invested quite a bit in the "Barreleye" and will continue to do so is a good sign (there is a POWER9 Barreleye) but not a guarantee. Both companies and all large datacenters still rely on Intel Xeons for the foreseeable future. The experiment can quickly be terminated.
However, the announcement that China's largest Internet portal – Tencent – will also be using OpenPOWER servers is another sign that the OpenPOWER technology convinces people that it is a viable alternative. At first, the deal was just to license the affordable IBM S822LC (made by Supermicro) to the Chinese reseller Inspur, and Inspur – being a "local" reseller – would sell it to Tencent. However, this deal spurred Inspur to begin developing their own OpenPOWER products. Consequently, OpenPOWER is allowing IBM and partners to break into the fastest growing server market: China. The openness of the software and hardware ecosystem and the strong performance of the CPU cores puts OpenPOWER is a very unique position in China compared to both Intel and ARM. That is a very solid business plan if you ask us.
28 Comments
View All Comments
Zzzoom - Friday, February 24, 2017 - link
"As important as performance per watt is, several markets – HPC, Analytics, and AI chief among them – consider performance the most important metric. Wattage has to be kept under control, but that is it."What a load of garbage.
JohanAnandtech - Saturday, February 25, 2017 - link
And now maybe some arguments that substantiate your opinion?SarahKerrigan - Sunday, February 26, 2017 - link
In HPC specifically, power consumption is a major issue. This was the entire root of the success of the Blue Gene line back in the day, and why NEC is shifting its supercomputing CPUs to progressively more efficient cores instead of higher-performance cores now (SX-9: 102.4GF/core; SX-ACE: 64GF/core.) . HPC is sensitive to running cost, and power dissipation is a critical factor in that.Zzzoom - Monday, February 27, 2017 - link
Go read the 7+ years worth of materials from the EE HPC Working Group.JohanAnandtech - Wednesday, March 1, 2017 - link
In a system with 2-4 GPUs, 512 GB of RAM, the TDP of the CPU is not a dealbreaker. I can agree that some HPC markets are more sensitive to perf/watt; but I have seen a lot of examples where raw performance per dollar was just as important.Zzzoom - Wednesday, March 1, 2017 - link
POWER8 TDP is 45W-102W higher per socket than the highest spec Xeon E5. That's 90W-204W higher per node where each node consumes 1500W-2000W, or 6-10% total on a site with a multi-million dollar power bill that went to great lengths to bring down the PUE by a similar amount. So for anyone to pick POWER8 it has to do better on energy to solution through its unique features, or be considerably cheaper (ha!). POWER8's advantage is NVLink, but TSUBAME3 going with Intel+PLX switches on top of NVLink shows that it's not that big of a deal.Anyway, the efficiency requirements on the CORAL procurements are pretty strict so scale-out POWER9+Volta will have to shed a lot of weight.
Zzzoom - Wednesday, March 1, 2017 - link
I forgot about the memory buffers. It's even worse.mystic-pokemon - Sunday, March 5, 2017 - link
Guys, I know shit ton of stuff about a server Johan listed above. He has a point when he says Power consumption is only so much important.In short, when you combine all aspects to TCO model: POWER8 server delivers most optimal TCO value
We consider all the following into our TCO model
a) Cost of ownership of the server
b) Warranty (Lesser than conventional server, different model of operations)
c) What it delivers (How many independent threads (SMT8 on POWER8 remember ? 192 hardware threads), how much Memory Bandwidth (230 GBPs), how much total memory capacity in 1 server ( 1 TB with 32 GB)
d) For a public cloud use-case, how many VMs (with x HW threads and x memory cap / bw ) can you deliver on 1 POWER8 server compared to other servers in fleet today ? Based on above stats, a lot .
e) Data center floor lease cost in DC ( 24 of these servers in 1 Rack, much denser. Average the lease over age of server: 3 years ). This includes all DC services like aggers, connectivity and such.
f) Cost per KWH in the specific DC ( 1 Rack has nominal power 750W)
All this combined POWER has good TCO. Its a massively parallel server, what where major advantage comes from. Choose your workload wisely. That's why companies continue to work on it.
I am talking about all this without actually combining with CAPI over PCIe and openCAPI. Get it ? POWER is going no where.
Michael Bay - Friday, February 24, 2017 - link
I think at this point in time intel has more to fear from goddamn ARM than IBM in server space.Okay, maybe AMD as well.
JohanAnandtech - Friday, February 24, 2017 - link
Personally I think OpenPOWER is a viable competitor, but in the right niches (In memory databases, GPU accelerated + NVlink HPC). Just don't put that MHz beast in a far too small 1U cage. :-)