Data Center Power Distribution

[jitsking]

I work in a data center. My question is in regards to power distribution. I have two UPS units.

UPS A ? Percent Full Load ? 32%
UPS B ? Percent Full Load ? 40%

Each UPS feeds four PDUs. The PDUs then feed to server racks that house servers, switches, and other misc IT equipment. There may be some other misc. equipment tied to each UPS as well but the primary load should be coming from the PDUs.

How would I go about figuring out why UPS B is so much more utilized then UPS A (there?s an 8% difference in load!). Seems like a very large difference. Especially considering that these are both 500KW UPS units. So that 8% makes up a 38KW gap. That?s a lot of energy.

If there?s any more information that I can provide that will help, please do let me know as I?m not 100% sure I gave enough information for you to go based off.

Any help would be very much appreciated and remembered.

 

[dereckbc]

Because you have more load on UPS b than A.

 

[BigBri]

Re: Data Center Power Distribution

I would not worry about that difference, but I would worry that some of the equipment on UPS A is single power supply and not plugged into UPS B. Might cause a surprise if you turn down UPS A for electrical work or there is a power bump when on maintenance bypass.

 

[GoldDigger]

You cannot figure the load just by counting the number of racks and you cannot even assume that two physically identical servers will pull the same amount of power independent of the processing they are doing.
You certainly cannot assume that whoever designed the system calculated to get an exactly balanced load.
In short, what Dereck said.

Tapatalk!

 

[ed downey]

Couple of questions

1.) Are these Single Module UPS Systems?
2.) Are these Multi-Module UPS Systems?
3.) If they are single Module are they set up in a 2N Configuration?
4.) Are all of the servers dual corded?

 

[ron]

As mentioned before, I'll reiterate that even if the loads are dual corded, maybe not all of the server power supplies share load across the cords evenly, maybe some of the dual cords are plugged into the same source.

 

[jitsking]

More Detail

More Detail

Because you have more load on UPS b than A.

So let me explain more detail.

These are double conversion UPS units.
Most everything in the DC runs off dual power (one power going to a PDU fed from the A UPS and the other going to a PDU fed from the B UPS).
There is some one-off equipment that is single-fed, but not enough to allow for this much of a difference..

 

[jitsking]

As mentioned before, I'll reiterate that even if the loads are dual corded, maybe not all of the server power supplies share load across the cords evenly, maybe some of the dual cords are plugged into the same source.

Ron,
All of our servers are dual-fed to different PDUs which feed to different UPS.

I wish where I worked we monitored at the branch-circuit level. All I have available to me is the reading on the front of the Liebert PDU that gives me real-time power usage.... Although I did walk today and made some discoveries. Looks like there is some load on the UPS units that is non-data center load. and they didn't dual feed it. So I can already see that's part of my problem.

 

[gadfly56]

Ron,
All of our servers are dual-fed to different PDUs which feed to different UPS.

I wish where I worked we monitored at the branch-circuit level. All I have available to me is the reading on the front of the Liebert PDU that gives me real-time power usage.... Although I did walk today and made some discoveries. Looks like there is some load on the UPS units that is non-data center load. and they didn't dual feed it. So I can already see that's part of my problem.

I don't know much about the power monitoring capabilities of these set ups so this may be the Stupid Question of the Day, but how accurate are the meters on the Lieberts? It's not like they're being used for billing purposes.

 

[__dan]

Typically the DC PDU's may have a dual source static switch, dual feeds, at the PDU input. So it's not clear if you have four single feed PDU's on each UPS, four PDU's total with dual feed from each UPS, or eight PDUs total each with dual feed from each UPS.

If the PDUs are dual fed, you could have the odd numbered, 1 3 5, on UPS A and the even PDUs, 2 4 6, on UPS B. So it would be easy for one of the PDUs to be loaded on the wrong UPS (if dual feed), PDU 1 on UPS B instead of A. That would be easy to check, source available and source connected at the PDU.

My feeling and suggestion, which was not implemented, is that dual cord loads should be commission tested for dual power as they are added. Plug it in (each new load), run a benchmark, and yank one cord at a time and see what it does for current draw and alarm reporting.

Some will load equally with dual cord and swap half for outage, some will share 100% 0% and swap 100% for outage (static switches), some will share 50% 50% and swap 0% for outage but degrade performance by 50% (some raid stuff and stuff that does not do what the manufacturer or salesman promised).

 

[vicdog]

Don't forget there could be bad power supplies beeping away.

 

[Sahib]

The query from the OP becomes more interesting and may be more beneficial to the OP if it is framed to include this also: suppose the load growth in such a way that load on UPS A is 70% and load on UPS B is 130% for example. What to do now? Obviously paralleling the UPS is the answer for equal sharing of loads throughout the operating range of the two UPS. There are many additional benefits in operating UPS in parallel such as extra capacity or redundancy. In an 1+1 redundant configuration in your case, you would have at least one more UPS module than needed to support the load. As a parallel system, each UPS stands ready to take over the load from second UPS whenever necessary, without disrupting protected loads.

 

[JoeStillman]

I wonder how the dual power supplies in the servers share load. My guess is that at least some of the redundant power supplies have no load on them until they are needed. The actual computer load might only be on one of the power supplies.

If that's true, then you have more of the primary power supplies plugged into the UPS with the greater load.