Necessity is the Mother of Innovation
With a small on-site crew and remote workers, Compass and TierPoint commissioned a new facility in Raleigh. In an effort to minimize risk to COVID-19, we took the process virtual, and greatly reduced the number of people on-site.
Announcer: Welcome to “Not Your Father’s Data Center Podcast,” brought to you by Compass Datacenters. We build for what’s next. Now, here’s your host, Raymond Hawkins.
Raymond: Thank you for joining us today for another edition of “Not Your Father’s Data Center.” Today, we’ll be joined by TierPoint’s vice president of Data Center Facilities, Ricky Vasquez. And I’m so blessed that Ricky is here in the Dallas-Fort Worth metroplex with us. So, Ricky, thanks for joining us today.
Ricky: Sure. Thank you, Raymond.
Raymond: Ricky, I think home is, what, out there west of town, right, Weatherford or out that direction?
Ricky: Exactly, yup, just about 20 miles west of Fort Worth.
Raymond: All right. Well, good to have you. You and I are both DFW guys, but it’s a big area. So, for those of you who don’t know the area, Ricky and I are not quite an hour apart, but good to have you on the podcast today, Ricky. Today, we’re gonna talk about specifically virtual commissioning, something that we did in light of COVID, but I really want to talk about what’s going on with COVID, how it’s impacting our businesses, how it’s impacting things at TierPoint, how it’s impacting our ability to get into our facilities and do things, and how we’re changing that approach based on the new world and the way the world looks. Ricky, is that an okay set of subjects to talk through today?
Ricky: Sure. Yeah. It sounds good.
Raymond: So let’s just start. Ricky, if you don’t mind, will you start with just a two or three-minute overview of what TierPoint does, where you are, how you do it, who are your customers, what they look like? We don’t necessarily need to name anybody, but what the marketplace looks like to you guys, and then we’ll switch gears and talk a little bit about how COVID is impacting that.
Ricky: Sure. TierPoint is one of the largest privately-held cloud and colocation providers in the U.S. We were formed in 2010 and we’re now in over 20 markets with 40 locations across the country. We provide a full suite of services, including colocation, public and private cloud, managed services, disaster recovery, network services as well as security and compliance. We do have quite a few customers, as you may know, Raymond, and those markets are pretty broad. We have a very large, not one particular vertical, so we have anywhere from government, healthcare to smaller businesses, as well as other technology firms.
Raymond: Ricky, if you don’t mind, I’m gonna ask you to give me a little more history on you.
Ricky: Yeah, absolutely. So, I’ve been in the information technology and operations area for over 20 years and specifically in the data industry for…data center industry for over 15 years. I’ve been with TierPoint since the beginning, which was in 2010. And I was brought in as a part of the first acquisition, which became TierPoint after having previously worked with and supported the growth of the acquired company. Over my career, I’ve served in different roles and capacities at TierPoint, but all within the facilities management and operations. I currently serve as the vice president of Data Center Facilities. I have nationwide responsibility, including various programs and oversight such as our energy portfolio management, our emergency fuel program, facility management support, assisting with and establishing KPIs and company-wide standards.
Raymond: So, Ricky, to clarify, how many data centers do you operate and oversee for TierPoint?
Ricky: We currently have over 40 locations across the country.
Raymond: Great. Ricky, could you give me two minutes… You know, I think when we say the word cloud, I think the names that come to everybody’s mind, AWS and Microsoft’s Azure, and starting to be Google’s GCP. I know, I think that’s really what you would call public cloud. Could you give us just 30 seconds on TierPoint’s world when you say private cloud, what you mean and how TierPoint offers that solution, solves that problem for customers?
Ricky: Sure. So, essentially with the private cloud, it’s very much like it sounds. It is your own space within our space. So, we do actually have dedicated areas in the facility that are specifically geared towards this, not only to meet compliance needs but, of course, the specific customer needs that’s driving the need to have the private infrastructure. So, to your point, you know, referring to AWS, etc., it’s not on kind of the general equipment or general population, but more so geared towards the customer specifically. So, they actually have their own instance of their cloud deployment.
Raymond: So, if I wanted to have cloud compute resources, but I didn’t want them at say, in an Azure instance, I could put them in a…I could buy them from TierPoint and that infrastructure that would support my cloud instance would be inside TierPoint data center and it’d be TierPoint equipment, but I’d be able to get services from that equipment directly, is that a fair way of saying it?
Ricky: It is. Yes.
Raymond: And then I guess the next step up from there, Ricky is the managed service. And is that where I’ve put my equipment in your building and you guys are now overseeing it from an operational security and other perspectives? Is that the wrinkle between running a private cloud and running a managed service?
Ricky: That’s correct. So, you could think of the managed services and, of course, that encompasses various offerings there, but essentially that is more of a high-touch approach, not that we don’t have that for other services, but that’s the general purpose of the managed piece.
Raymond: Gotcha. And, Ricky, would it be fair for me to describe this as that TierPoint does a great job offering customers what I would consider a hybrid cloud environment, so they may have some resources in AWS, they may have some resources in Azure, or they may have some of their own resources that they manage, they may have some resources that they have TierPoint manage in a managed service environment, and they may also have some private cloud resources that they get from you guys? And, for me, that would be a hybrid cloud solution. Is that a fair way to think about where TierPoint fits in the cloud and IT service spectrum?
Ricky: Yeah. I think that’s a fair statement. I think the biggest piece with regards to the hybrid approach is more referring to the colocation offering as well. So, you can think of it as you’ve got perhaps some of your assets in you know, a physical form and deployment within one of the facilities, and then you have the cloud offering also depending on your needs. Some of that is whether it’s compliance-driven or perhaps part of your disaster recovery approach. So, it would be a blend between the two, colocation and cloud.
Raymond: Ricky, I know you’re on the operations and facilities side and I keep peppering you with sales questions, but I’m just trying to set the bar for folks that listen to us to make sure that they fully understand what TierPoint does and where you guys fit in the customer solution set. So, today, I think you said…did you say 20 markets, Ricky? How many markets are you guys in today?
Ricky: Over 20 markets.
Raymond: Yeah, 20 markets. And a typical…somebody that goes, okay, “That’s a perfect customer for us,” is that an enterprise client? Is it a large institutional client? What’s the best fit for you guys as you’ve been in the business…which, by the way, Ricky, how long have you been with TierPoint?
Ricky: I’ve been with TierPoint since the beginning actually, so, since 2010.
Raymond: All right. So, a decade-plus. So, when you see a customer come along and you go, “Wow, that’s a perfect fit for us,” what’s a perfect fit?
Ricky: So, to the enterprise question, you know, it’s not necessarily that we’re looking for the enterprise clients directly or in and of themselves. We’re also going after even the smaller businesses. So, there’s not really one particular client that we’re necessarily seeking. I think where we come in is being able to fit the needs of any given customer or a potential customer, what they’re looking for and seeing if it’s a fit for them and, of course, for us, but trying to determine what they’re looking for, isolate those needs, and come up with the best solution for them regardless of the size of the company.
Raymond: Gotcha, gotcha. All right. Well, I’ll stop peppering you with sales and marketing questions, but I certainly wanted our listeners to get to hear where TierPoint fits in the solution set. Let’s switch gears a little bit. It’s for those of you who may listen to this later, today is May 20th. The world is still trying to make sense of…Ricky and I are recording today on May 20th, the world is still trying to make sense of COVID-19 and how it impacts the way we do business, when can we go back to working like normal, and what’s the world gonna look like in a post-COVID world. It has certainly changed the data center industry. Ricky, if you don’t mind, especially with your facilities and operational expertise, could you take a few minutes and talk to…about what is changing in the data center from a people perspective, and then we’ll get into how to do commissioning and some other things which are normally pretty high-touch functions, how we’re doing those in a COVID world today?
Ricky: Sure. So, you know, as with those who have been following social distancing guidelines per the CDC recommendations, we also require face mask while on-site from not only our vendors, but also the personnel and customers. And we have been limiting the amount of personnel on-site. You know, our customers have been great to not only work with in this but also understanding. They too are also dealing with challenges within their own businesses. So, we’ve had that aspect or angle to deal with as well. But I think, by and large, you know, for us, it’s been, I really hate to say business as usual, but I think, for the most part, it has been business as usual in an unusual time. And by that, we’ve been able to continue to conduct our business and have done that very well.
Raymond: I think that our business, that being the data center business, we’re not a high traffic business. We’re not like retail or even you know, transportation business where there’s lots of people close to each other. We’re fairly spread out. So, the concept of social distancing is pretty easy for us. I think this has been about, let’s make sure that who’s coming in our facilities is not symptomatic and let’s make sure that our facilities stay clean. That’s been our approach as Compass operates data centers. Have you guys seen a similar, “Hey, let’s check our folks at the door. Let’s make sure our staff is healthy,” and that that’s really been the main focus and that we keep our areas clean? Beyond that, we’ve not, like you, we’ve not seen significant changes. It’s been things that are already part of our security protocol that we’ve just added in temperature and, you know, personal protection to that same protocols. Has that been the world for TierPoint?
Ricky: Yeah. I would echo those comments. And it has been… So you know, much like you guys are handling it, we do have various questions that are asked when folks come into a facility. Some of those are posted. Some of those are direct. And so, we’re taking those appropriate measures as people are entering. We are fortunate, to your point, that the majority of our locations I would say it’s not necessarily a high traffic environment, although we do have some sites that…where we certainly participate and they have DR seats available. So, in those situations, it’s a little bit of a different approach just because of the higher volume of folks coming in and out of the building.
Raymond: Right. Another service offered and just another area to monitor, but same thing, let’s check the people coming in. Let’s make sure they’re not symptomatic and let’s make sure we’re staying clean. And as long as we do those things, I think our business has been able to run business as usual and it sounds like the same for our friends at TierPoint.
Ricky: That’s correct.
Raymond: So, Ricky, let’s change gears a little bit and let’s talk about, so commissioning. I’m gonna guess, with more than a dozen years at TierPoint, you’ve commissioned a data center or two, is that fair to say?
Ricky: It is. I certainly have.
Raymond: So, do you mind telling us in a normal environment, who’s involved in commissioning? What takes place? What needs to happen? I know some of the folks that listen to our podcast are very deep in the data center business. Other folks are just trying to understand where this business is. Do you mind walking us through, what’s a normal commissioning like, who’s involved, what gets done, what are we trying to figure out?
Ricky: Sure. So, commissioning number one is a huge milestone for any project from a construction perspective or expansion. So, when you get to that point, it’s always exciting for everyone. It’s certainly a stressful time because you’ve worked pretty hard to get to that level of completion and then once you’re there, you know, you’re ready to make sure things function as they should. So, the ultimate goal for commissioning is to make sure that the building functions as designed. So, in this case, a data center, we need to make sure that it’s going to function per the equipment design as well as the overall facility design, and that it does what it’s supposed to do. We have a number of folks involved. That’s a very, I would say kind of hectic time in terms of the amount of personnel typically required to get through one of these functions, but they’re also very well-organized at the same time and scripted. Usually, you’re going to be working with your commissioning firm, which could vary as well as, of course, your general contractor and the various other folks that were involved in the project, all the way down to some of the technical team for specific equipment, whether it’s the installation team or the manufacturer crew.
Raymond: So, Ricky, when I move into a new house, I think of the things that I want to go check here is the fit and finish good? Do the light switches all work? Does the plumbing all work? As I think about it from a residential perspective, those are the things I walk around and go, “Hey, is everything the way I think it is?” I would assume that there’s that in a data center project, but there’s also the critical facility side. Do you mind just giving us a little bit of insight on the critical facility side of what needs to get checked in commissioning?
Ricky: Absolutely. So, and you bring up a good point. But so, to refer to your comment there, it’s more so that’s regarding kind of a punch list approach, which certainly we’re going to do that and do those on each project. But the commissioning itself is more geared towards the critical infrastructure and things such as think of the HVAC systems, the generators, the backup UPS systems, anything that is actually running that facility to make it operate per design. So, some of the testing that we complete as any commissioning would is you’re going to do failover testing, ensure that if there’s a utility outage, for example, that the building is transferring to and from like it should. That all the gear and respective switchgear performs as it’s designed and so forth.
Raymond: So, Ricky, when you’re going through commissioning a new facility, you talked about that there’s your operational team on-site, that there’s a commissioning firm and a GC, everybody’s involved. The number of folks, what’s the typical number of people involved and what’s a typical duration for commissioning a facility?
Ricky: So, the number of people can vary greatly. Again, it depends on who was involved, to begin with, the size of the project. The, I would say the scope of the commissioning firm, and to what depth they are involved as far as some of the operations of the facility. But by and large, you could estimate that there would be at least 20, 25 folks involved on a commissioning process on an average. That will vary on the day depending on what is being conducted that day. Sometimes it may be more of a smaller crew, but it usually takes a fair number of folks to accomplish the commissioning process.
Raymond: So, the logistics of getting that many people around in a world where travel is a challenge today and crossing state lines and those kinds of things get to be really challenging. As we think about commissioning new facilities and bringing new infrastructure online in a COVID world, our two firms had to face that challenge. We commissioned one of your new facilities in Raleigh together and we did it virtually. Do you mind talking about that process, that challenge, how you thought about it, how we came up with the concept of commissioning it virtually? And let’s just start down that path a little bit, even with the logistics of getting, “We can’t get the people there. What do we do?”
Ricky: So, as you know, we were nearing completion of this project and getting close to commissioning process itself, obviously, the impact of COVID-19, we determined that it wasn’t possible to perform the commissioning in a traditional sense like we would normally do. And because of that, we ended up working with, of course, Compass and with TierPoint’s support as well as the commissioning firm to try and come up with the way that we could do this potentially remotely. And, you know, with that, working with the commissioning agent, a lot of the foundational work is already in place ahead of this, whether it was on-site or remote. So, things such as the scripts, the tasks, and anything that was going to be needed from a personnel standpoint was already predetermined. So, that allowed us to at least to have the details at hand and then decide whether or not this was something that we were able to achieve in a remote fashion.
Raymond: So, we were able to start with a checklist from our commissioning agent and the first thing being, “Hey, we can’t physically get people…” It was in Raleigh, right, so we can’t get all the folks that need to be involved to Raleigh. So, how do we start going through this process virtually or remotely? Do you mind taking us that next step down? Okay, folks can’t get here. We still need this facility to come online. We still got growth and customer needs that we need to meet. Can you start helping me understand how do we do this remote? Was it a Zoom call? How do we get down the path of starting the commissioning process and doing it all remote?
Ricky: So, once we decided that we could do this potentially remotely, the decision by and large was based on, you know, CDC guidelines and social distancing needs. That, and the fact that, you know, of course, we needed to come up with a way to do this remotely. As far as the amount of personnel and that’s really, I would say that outlines how we came to that determining factor there. And then as far as the commissioning itself, from a virtual perspective, to your point, I mean, yes, it was done using various methods in the field. Some of that was live video feed. We actually had two simultaneous feeds going at the same time. We also had a couple of different teams meetings that were active, one of which was in place to monitor some of the building management system aspects of the project as well as the fieldwork that needed to be done. So, we had both of those going in parallel throughout the process.
Raymond: So, your in-place operational staff in your facility was there and on-site and able to have video feeds whether from a computer or a phone and be able to look through the systems and have it managed remotely by folks that couldn’t physically get there?
Ricky: That’s correct. Yeah. From our side, we had a limited staff, I will say. And then, of course, the commissioning team and the other…a couple of technical folks that needed to be in the field as well from the manufacturer perspective.
Raymond: Is it right for me to be picturing guys walking around with their iPhones, looking through power centers and looking through switch panels and those kinds of things? Did we get to that level?
Ricky: We had iPhones, yes. We also had you know, other cameras that…you know, think of GoPro, things like that that were available and they were using those tools.
Raymond: So, as we think about the world of travel, I think it’s going to be a long time before that gets back to normal. Could you see us approaching virtual commissioning and doing this remote as a way to be able to do it in the future, or was this really a COVID response and the tried and true method of having a couple dozen people descend on a facility still the best answer?
Ricky: I think the traditional approach is certainly going to win out overall and would be you know, the preferred methods certainly from our perspective. But at the same time, you know, I think ultimately, as to whether or not the virtual approach was fully adoptable is going to be up to the third parties. And the third parties that I refer to are folks like Uptime Institute, etc. So, you know, that would be something that…
Raymond: The commissioning agents and things like that. Right.
Ricky: Exactly. So, without that buy-in, you know, we couldn’t I would say replicate this on an ongoing basis.
Raymond: Right. Well, I think as our world changes, right, the response based on COVID changing how we do virtually everything. I think this is an early example of our business, the data center business, really being vital to how we fight the pandemic, how we solve the problem, and as well as how we enable people to do their work. Our business, mine and yours in the data center space, I think we’re the…it reminds me of that Cisco commercial that used to say that the network is the computer. I think we’re seeing my two college kids are home and they’re doing college classes from home all through Zoom. The network is getting taxed in a way that we’ve never seen before. And so, mine and your business are being asked more and more to ramp up capacity, deliver capacity, have space available, deliver services to our customers. And I think as that part of the world grows rapidly in this pandemic, we’re going to have to come up with different ways to do things. And even though a couple dozen guys going to Raleigh might be the absolute best way, if we could make it to where it’s six guys that go to Raleigh and we could do, you know, 70% of the commissioning virtually, I think this step that you partnered with us to do in Raleigh was a great example for solving a problem in a tough environment, but it may be a new way for us to think about stuff and a new way to solve the problems moving forward because I look at the way the world has changed and I think travel, especially when we think about travel outside the U.S., I think travel is fundamentally changed for years.
Raymond: And as we think about, how do we do this, how do we help our customers get online? The concept of commissioning virtually might just be one answer in the puzzle. Because I think our businesses, I looked at your guys’ Q4 and your Q1, it looked like you guys had great quarters and the business isn’t slowing down in this world, I think our industries are both being asked to deliver more capacity, and we gotta do it in creative ways. How does the business look to you from a facilities perspective and growth perspective in the midst of this pandemic?
Ricky: Yeah. I think I would agree with your comments there. And to your point, you know, we have been very fortunate to be very busy not only prior to this, but also…to COVID-19, but also during and, you know, certainly after, but… So that’s been a good thing for us and we’re able to accommodate the customer needs. Some of the demand has increased due to this pandemic. We’ve also seen, again, like I said earlier, you know, business as usual with the exception of folks not being able to maybe travel as much as they could before. You know, I think we’ve been able to also, you know, personally and within the company, see the things we’re able to accomplish remotely that don’t necessarily require someone to be on-site. Sometimes you just don’t, you know, you don’t really have a way around that, but there are pieces of project that you’re able to accomplish without having to physically be there. So, to your point of, you know, maybe the way business is conducted going forward will be changed by some level of measure once we get through this.
Raymond: Yeah, Ricky. I’ve been in the technology business most of my professional working life and the concept of virtual meetings and video conferencing, that’s been around that industry for a long, long time. But I think this COVID reality of this pandemic has caused industries that never thought about doing things virtually, I think it’s been you know, forced on them. And I think a lot of people are going to go, you know, “This is a handy way to do things.” I think that what it means for the, you know, office real estate business might be changing fundamentally, but I think industries that were slow to adopt the concept of virtual meetings and remote access to things, having that forced on them, I think it’s gonna end up ultimately being good for our business, good for the industry, and maybe even good for the environment in the long run.
Ricky: Well, if nothing else, it may have sped up the process, right, to get to that point of adoption.
Raymond: I read in an article the other day, a CIO said that we had two years of IT transformation taking place in two months. And I thought that was a pretty good analysis. Just people were forced.
Ricky: I believe that.
Raymond: You gotta go. You know, I know you’ve thought about this idea, but you better do it now because you can’t get there. Well, that’s good stuff. More than two decades in the technology business, that’s a lot of change. We’ve gotten to live through the dot-com explosion and then the dot-com implosion and the financial crisis of ’08/’09 and now, this pandemic. So, you’ve seen some changes in the way the world does business in your time.
Ricky: Absolutely. I’ve been fortunate to be a part of that. As a matter of fact, the company I worked for previous to TierPoint had started during 2000. So, it was the best and worst time to start a business. But we were able to make that work.
Raymond: Well, Ricky, this has been great. I really appreciate you talking us through why we needed to virtually commission in Raleigh, how we went about it, and how it might impact commissioning in the future. Thank you for joining us.