Batteries and Data Centers

Estimated reading time: 34 minutes

The I-Connect007 team met with electrical engineer Mike Mosman, who has spent the majority of his career designing some of the country’s largest data centers. Mike recently sold his engineering company CCG Facilities Integration; he is now the vice president of electrical engineering in mission-critical facilities for Morrison Hershfield, and is still designing leading-edge data centers for large Internet companies. In this discussion, Mike explains how evolving battery technology and a strong demand are playing a key role in this area and others.

Barry Matties: Mike, thanks for joining us today. Let’s start with how you started your career and began designing data centers.

Mike Mosman: I graduated in power electrical engineering from Washington State University in 1974. In 1975, I entered the electrical engineering field, and started out in a one-man electrical engineering shop. In a short time, I connected with a construction company working on a nuclear powerhouse in Washington. So, I was a quality control engineer for a while for the Washington Public Power Supply System, and later at Plant Vogtle in Georgia. I was there at the end of the nuclear age.

After that, I went to work for a company that designed and built data centers, and this was actually before there were such things as email. And so, since 1986, I have done nothing but design data centers. I probably am now one of only a handful, maybe five data center engineers left in the United States from that time. I’ve seen data centers progress from tiny 500-kilowatt to one-megawatt data centers stuck in the back of a bank building, to what we are now calling hyperscale data centers that use more power than the city I live in.

So, we are at a cusp right now of a sizeable shift in the way that data centers are being built, which customers are going to be buying into them, how they’re being financed, and where they’re going to be located. And right now is a very, very exciting time to be in the data center industry. I am very near retirement. As a matter of fact, I’m past retirement, but I’m still working for this company.

We currently have about 80 people within Morrison Hershfield that concentrate on data centers. And that is in three of our offices, Toronto, Atlanta, and the one that they just purchased, which is my office in Baltimore.

Barry Matties: In your many years of designing data centers, batteries have played a key role. Tell us a little bit about the battery technology.

Mosman: The technology has improved over the 35 years that I’ve been working with batteries. The first time that I ran into a battery—an uninterruptible power supply system—was for one of the nukes I was working on. And I looked at that thing, and I thought, “Wow. Look at all these huge jars full of funny-looking lead plates. This is really interesting.” That was lead-acid technology. And I worked with that for 20 years, trying to figure out how to get those things to last more than seven or eight years. And they were huge. They were expensive. When we build data centers, 55% of the cost of the projects is electrical. Of that, half of it is in electrical equipment. And of the electrical equipment in data centers, approximately 8 or 10% percent of it might be in the cost of the batteries.

And so, figuring out how we could jam more power into less space, make it more energy-dense, has been the job of battery manufacturers for many years. But now, the technology has shifted away from lead-acid open cells. We had for a time what were called the VRLAs, valve regulated lead-acid batteries, which are sealed batteries. They’re smaller, with a little bit more power density. You can put them into cabinets, and that was a step-up in battery technology for many years. It made them more economical. But there were still some alternative technologies out there.

Nickel-cadmium batteries, for instance, were three times as expensive as lead-acid batteries, but they lasted longer, and they could take more discharges, especially the type that you find in a data center, hooked up to an uninterruptible power supply system. But then came the lithium-ion technology. It was developed mostly for automotive, electric cars, buses, and things like that, and in small sizes for appliances because they were very good for weight. They were very good for power density. Unfortunately, they did seem to have a propensity to be expensive and flammable, things that really prevented them from being widespread used in huge UPS applications. But that’s changed.

Now, the lithium-ion batteries have dropped in price where not only are they competitive with lead-acid batteries, but they’re also competitive with other forms of power backup such as massive engine generators. All of these data centers that I built, they’re connected up to huge engine generator plants. When you start talking about Environmental Protection Agency restrictions and terrible licensing requirements, plus the fact that you have to build special spaces for them that can handle the airflow and fuel distribution, it can be quite problematic. And the maintenance for them is just incredible.

So, a new trend in using lithium-ion batteries as a replacement for engine generators is changing completely what data centers look like now. And lithium-ion batteries are, by the way, not just one type of lithium-ion battery. There are about eight or 10 different metallurgies that go into them. And right now, what seems to be the most promising to me are the iron phosphate type. There are also some silicon carbide-type batteries. But these different metallurgies all have their pros and cons. Some of them are not subject to bursting into flames if you puncture them. Some of them are better for a larger power density or a higher number of discharges. Some of them are suited for high-power demand over short periods instead of low-power demand over long periods of time.

Right now, I’m thinking that most of these lithium-ion batteries for UPS applications or utility grid stabilization applications (which use huge banks of batteries) are going to be the iron phosphate type. They have a good safety. We’re not worried about what they weigh because they’re not in a car or some form of transportation. We’re a little bit concerned about how much space they take up. But the iron phosphate seems to be a good combination of characteristics. It’s fun. It’s been fun, over the three decades, watching what’s happening with batteries.

Feinberg: I’ve had solar on my house for 20 years. The first ones were lead-acid. And they lasted about three and a half, four years, maybe. The second set lasted about six years, seven years. And now, the third set I’ve had is 10 years old, and they’re running just fine. They’re all lead-acids. But I’m thinking I’m going to have to replace them soon. And you’re thinking I’ll be able to replace them with lithium-ions?

Mosman: I’m pretty sure you will be replacing them with lithium-ion batteries because the price of lithium-ions has gone down factor of three. You can get those about one-third of what they used to be because of their penetration into the market more than anything else. But lead-acid batteries, since those early decades in large UPS applications, have improved quite a bit. I remember in the late ‘90s I was designing data centers for United Parcel Service. They heavily tested their batteries weekly, and they only lasted a few years. By the way, every time you ship a package with a brown truck, you’re using a data center I designed. And every time you use your Visa card, you’re using a data center I’ve designed. Did you guys ever use AOL? I’ve designed all of AOL’s data centers.

Feinberg: AOL and Prodigy.

Mosman: Oh, Prodigy. That data center could fit in my bedroom back then.

Matties: Times have changed. You’re saying it’s a really exciting time right now in data centers. Why is it so exciting right now?

Mosman: Well, data centers used to be a brick-and-mortar extension. I did a lot of banks, financial companies, designing things for Morgan Stanley, Bear Stearns, Goldman Sachs, those types. And First American Bank and Bank of America, I’ve worked with those banks. These are all data centers that had large mainframes. Most of their work was batch work, and the computers were used by maybe 100 people.

And then along came AOL. Suddenly, we’re not talking about $20 million mainframes backed up with a whole bunch of DASD. We’re talking about racks of servers, racks and racks and racks of servers. And it was one of those sea changes that happened within two or three years.

I designed the first ground-up data center for AOL. When we finished that job, one could say that I was responsible for designing half of the internet because half of the internet flowed through America Online. But that’s not the way it is anymore because good ideas soon have many adherents. We saw just an explosion of data centers, up until the internet crash, which happened around the turn of the century. One of the things about the internet is that the demand continues to go up and up and up. But data center construction, seems to go in cycles or in steps. And that goes along with the innovation that goes into them. And right now, we’re taking another one of those steps.

We were talking about the batteries back in the old days. People who were making batteries then, Exide and C&D, were trying to get as much power out of their batteries as possible. And so, they would build them without all of the required separating pads between plates, which meant that if you actually used them, if you were to actually do a full discharge in those things, you could pretty much destroy them. We had to be very, very careful not to let the conversion of the Pb++++ that turns into PbSO₄ during discharge get so deep into the lead paste on the plates that it actually starts to distort the lead grid out of which they built the battery plates. Once you do that, the battery will disintegrate. And so, people who were really actually using their batteries back in the old UPS days would get a year and a half or two years out of the batteries before they literally fell apart.

But then, they started figuring out, well, we’re going to build the batteries with different configurations in their plate design. They started building these little sealed batteries, the VRLAs, which are still lead-acid batteries, but they’re gas recombinant type and they don’t have venting. And you don’t have to water them. But they had a warranty that was only 10 years whereas the old lead-acids were supposedly a 20-year battery. The problem is that when you actually start to use batteries, it’s very, very hard to get any battery to go past seven or eight years, no matter what they say about 10-year or 20-year batteries. There are some pure lead batteries or round batteries that supposedly would last for 30 years. And you could probably get 20 years out of that type of battery. But nobody could afford them.

Now, the way we use batteries, up until a couple years ago, was mainly for short-duration outages because we were building data centers with huge banks of engine generators. With all of this rolling iron on these projects, we had to have enough battery to carry the critical load through the starting time of all the engines. But that really took only half a minute to a minute. Ultimately, we learned we didn’t have to put 15- or 20-minute discharges on those batteries. When we started buying 5-minute batteries of the sealed lead-acid type, the VRLAs, we found we were actually able to get a useable life out of them comparable to the old, 15-minute, vented lead-acid batteries. We would have loved to have had batteries like the nickel-cadmium batteries or these lithium-ion batteries back then. And they did exist, but you could not afford them.

But suddenly, now, we can because we’ve developed technologies that can build them to scale. And with the manufacturing capabilities that we have now to pump these batteries out, factories like the huge Tesla plant outside of Sparks, Nevada, just down the road from one of the data centers that I designed for Apple. They can build these batteries now so cheap that it almost doesn’t make sense to buy engine generators for your data center. That’s the big change that’s happening right now in data centers. We are seeing what we call hyperscale applications.

There’s a lot of buzzwords in my industry. And “hyperscale” just happens to be the one that they’ve come up with to differentiate “now” from those “old days.” Not too long ago people would build a data center, and it would have maybe 25 to 50 megawatts of critical power. They would have their computer data halls divided up with cages, and they would go out and look for people to come and put their servers in your caged areas. Maybe 60% of them were mom-and-pop porn shops. But then came the Big Five; Microsoft, Facebook, Google, Amazon, and Apple. And now, these guys are not building 25- or 50-megawatt buildings. They want 200- to 500-megawatt campuses.

Page 1 of 3

• Next Page >