February 12, 2014
After developing the theory of relativity, Albert Einstein’s next quest was to come up with a unified theory that explained all of the phenomena of the physical world. Despite his brilliance he was unsuccessful, and as he focused on the development on this lofty goal he increasingly found himself at odds with practitioners of the new school of physics that he helped found, quantum mechanics. I recently read an article the raised the question of whether Intel’s Software Defined Infrastructure (SDI) was destined to be the “One Datacenter Architecture to Rule Them All?” The author’s thesis was that due to Intel’s predominance within data centers, via the technology found in most servers and storage devices, that SDI could be the basis for the architecture and building blocks that will define the future of the “scale-out”—new term alert—data center industry. Dare to dream big, baby. While I perused the article I couldn’t help but notice that this “Lord of the Rings” derivative didn’t seem to address the people concerns of the average data center. Was it possible, I wondered, that, like Einstein, in our quest to control everything we might be overlooking a few things that could really impact data center operations?
As the data center environment becomes more dynamic, I would venture that if you asked the data center technician what innovation would have the greatest impact on the daily operations of the facility, he probably wouldn’t talk about evaporative cooling or DC to the rack. You see to him, all that water management or trying to find an electrician who can install the circuit is more of a pain and a real management issue that is much more important than a 0.01 reduction in PUE. Instead, locating the server staging area next to the data hall thereby eliminating the half-mile walk required to install a new server is a bit more of a welcome feature. To a certain degree, this gap between simple ideas that contribute to operational simplicity and the data center equivalent to the unified theory is just a reflection of human nature. Everybody wants to be the guy who comes up with the big idea and garner the glory and fame that comes along with it, but often times it’s the folks who come up with the little things that really make everything work. For example, Edison may have devised a way to bring electricity to the masses, but without the guys who invented the standard wall outlet and the plug he would have just been someone who had something really cool to look at in Menlo Park. I think this is where we are with our search for the universal scale out architecture, while we are intent on finding a way to integrate and manage every system and all of our applications within the facility, we overlook the fact that if we relocated the loading dock we could eliminate a security guard and reduce the time it takes to add and remove equipment by half. It may not be terribly sexy, but it is efficient.
Our focus on the grand vision also often manifests itself in misinterpreting the value of an innovation. For example, I was at an enterprise the other day that was interested in the servers that Facebook had created for OpenCompute. They talked about how cool it was that the servers had no covers and thus could dissipate heat so much easier than traditional servers. While I did not dispute the dissipation claim—or bring up the other issues that it presents with lack of containment since no one likes a smarty-pants–I did mention that Facebook’s “innovation” of the coverless server has nothing to do at all with cooling. It had to do with the fact that their operations personnel did not want to have their techs spend any wasted efforts, like removing covers, when doing break/fix on servers. The coverless design was for ergonomic efficiency, not cooling efficiency. The disappointment in the room was palpable but you can bet the Facebook tech who doesn’t have to remove a cover every time he has to touch a server thinks it’s a great idea.
A lot of the genius in scale-out data centers is in operational ergonomic efficiency. Since they have to do things at such scale, seemingly little improvements in time translate into huge increases in operational efficiency at scale. For example, Microsoft’s decision to use containers wasn’t so much a blessing of the technology for everyone, but it was the right decision to facilitate the deployment of thousands of servers per month.
IBM used to have a saying that they used to alleviate a customer’s concerns when a problem arose: “It’s a known situation and we have our best people working on it”. Isn’t this kind of like this whole SDI thing? We’ve got a lot of really smart people theorizing, postulating and working on the concept that will bring everything together, but sometimes when you’re focused on the next big thing, you miss a the little things. Einstein, for example, was brilliant, but rumor has it his wife had to lay out his clothes every day to make sure he wore socks. I’m not saying that we don’t need the “one architecture to rule them all”, but I happen to believe that the pure brilliance that we can get from the scale-outs is in the people related solutions—not everything is in the technology. Or as Einstein’s wife realized, sometimes you have to find your socks too.