The Uptime Institute occupies a unique position in the data center industry, bridging the gap between data center owners, and the designers delivering capital projects.
Our technical team, in the course of delivering hundreds of Tier Certifications around the globe, often finds communication breakdowns between these two groups, which can cause performance and capacity problems over the life of the data center.
In order to address these problems, Uptime Institute convened groups of data center owners from the Uptime Institute Network community, and the elite data center designers in the Uptime Institute’s Accredited Tier Designer alumni, to identify the key problems and create a list of solutions and best practices for the industry.
Addressing the Communications Gap
Beginning in May 2013, Uptime Institute engaged the industry to address this management and communication issue on a broader basis. The Uptime Institute’s approach was to meet with the designers and owners separately to gather feedback and recommendations and to then reconcile the feedback and recommendations in a publication.
During a special session at Uptime Institute Symposium in May 2013, the design engineers’ overwhelming guidance to owners could be summarized as “know what you want.”
The following issues were raised specifically and repeatedly:
1. Lack of credible IT forecasting
- Without a credible IT requirements definition, it is difficult to establish the basic project profile (size, density, capacity, phasing, and Tier). As this information is discovered, the profile changes, requiring significant delays and rework.
- In the absence of an IT forecast, designers have to make assumptions about IT equipment. The designers felt that that this task is outside their design contract and they are hired to be data center design experts, not IT planning experts.
2. Lack of detailed Facilities Technical Requirements
- The absence of detailed Facilities Technical Requirements forces designers to complete a project definitions document themselves because a formal design exercise cannot be launched in its absence. Some designers offer, or are forced, to complete the Facilities Technical Requirements, although it is out-of-scope.
- Others hesitated to do so as this is an extensive effort that requires knowledge and input from a variety of stakeholders.
- Others acknowledged that this process is outside their core competency and the result could be compromised by schedule pressures or limited experience.
3. Misalignment of available budget and performance expectations
- Owners wanted low capital expense, operating expense, and cost of ownership over the life of the project.
- Most solutions cannot satisfy all three. The owners should establish the highest priority (Capex, Opex, TCO).
Data Center Owners Respond
Following the initial meeting with the data center design community, Uptime Institute brought the discussion to data center owners and operators in the Uptime Institute Network throughout 2013, at the North America Network Meeting in Seattle, WA, APAC Network Meeting in Shenzhen, China, and at the Fall Network Meeting in Scottsdale, AZ.
Uptime Institute solicited input from the owners and also presented the designers’ perspective to the Network members. The problems the engineering community identified resonated with the Operations professionals. However, the owners also identified multiple problems encountered on the design side of a capital project.
In the owner’s words, “designers, do your job.”
According to the owners, the design community is responsible for drawing out the owners’ requirements, providing multiple options, and identifying and explaining potential costs.
Common problems in the owners’ experience include:
- Conflicts often arise between the design team and outside consultants hired by owners.
- Various stakeholders in the owner’s organization have different agendas that confuse priorities.
- Isolated IT and Facilities teams result in capacity planning problems.
- Design teams are reluctant to stray from their preferred designs.
The data center owner community agreed with the designer’s perspective and took responsibility for those shortcomings. But the owners pointed out that many design firms promote cookie-cutter solutions and are reluctant to stray from their preferred topologies and equipment-based solutions. One participant shared that he received data center design documents for a project with the name of the design firm’s previous customer still on the paperwork.
Throughout this process, Uptime Institute worked to collect and synthesize the feedback and potential solutions to chronic communications problems between these two constituencies. The following best practices will improve management and communication throughout the project planning and development, with lasting positive effect on the operations lifecycle.
All communities that participated in discussions understood the need to unite stakeholders throughout the project and the importance of reviewing documentation and tracking changes throughout. Owners and designers also agreed on the need to invest time and budget for pre-design, specifically including documenting the IT Capacity Plan with near-term, mid-term, and long-term scenarios.
The owners and designers also agreed on the importance of building Facilities Technical Requirements that are responsive to the IT Capacity Plan.
After preparing the Facilities Technical Requirements, invite key stakeholders to ratify the document. This recommendation does not prohibit changes later but provides a basis of understanding and launch point for the project.
Following ratification, brief the executive (or board) sponsoring the project. This and subsequent briefings can provide the appropriate forum for communicating the costs associated with various design alternatives, but also how they deliver business value.
RFP and Hiring
Provide as much detail about project requirements as possible in the RFP, including an excerpt of Facilities Technical Requirements in the RFP itself and technology and operations preferences and requirements. This allows respondents to the RFP to begin to understand the project and respond with most relevant experience. Also, given that many RFPs compel some level of at-risk design work, a detailed RFP will best guide this qualification period and facilitate the choice of the right design firm. Inclusion of details in the RFP does not prohibit the design from changing during its development and implementation.
Negotiate in person as much as possible. Owners regretted not spending more time with the design firm(s) before a formal engagement as misalignments only became evident once it was too late. Also, multiple owners remarked with pride that they walked out of a negotiation at least once. This demonstrated their own commitment to their projects and set a tone of consequences and accountability for poor or insufficient communication.
Assess and score the culture of the design firms for alignment with the owner’s preferred mode and tone of operations. Owners commented that they preferred a small and local design firm, which may require some additional investment in training, but they were confident would get more careful and close attention in return.
Notify the design engineer from the outset of specific requirements and indicators of success to pre-empt receiving a generic or reconstituted design.
Should the owner engage an outside consultant, avoid setting an aggressive tone for consultants. Owners may want to augment their internal team with a trusted advisor resource. Yet, this role can inadvertently result in the consultant assuming the role of guard dog, rather than focusing on collaboration and facilitation.
Design and Subsequent Phases
Owners and designers agreed that a design effort was a management challenge rather than a technical one. An active and engaged owner yields a more responsive and operable design. Those owners that viewed it as outsourcing the production/fabrication effort of a data center struggled with the resulting solution.
The following recommendations will reduce surprises during or after the project.
- Glossary of terms: Stakeholders will have varying experience or expertise and some terms may be foreign or misconceived. A glossary of terms established a consistent vocabulary, encourages questions, and builds common understanding.
- List of stakeholders: Stakeholders may vary, but identifying the ‘clients’ of the data center helps to establish and maintain accountability.
- Document all changes: The owner must be able to evidence the circumstances and reasons behind any changes. These are a natural aspect of a complex data center project, but knowing the decision made and why will be key to setting expectations and successful operation of the data center.
- Notify stakeholders of changes to IT Capacity Plans, Facilities Technical Requirements, and design documents. This will also help executive and non-technical stakeholders to feel engaged without disruption(s) to the project low and allow the project stakeholders to provide accurate and timely answers when decisions are questioned during or after the project.
As the recommendations were compiled from this initiative, many of the recommendations resonated with Uptime Institute guidance of years past. Over 10 years ago, Ken Brill and Pitt Turner held seminars on project governance that touched upon a number of the items herein. It is an old problem, but just as relevant.
This link provides key comments from the data center community, and a video of Uptime Institute COO, Julian Kudritzki presenting the findings of this yearlong effort.