Justifying data center electrical preventive maintenance

Status
Not open for further replies.

Natfuelbilll

Senior Member
Any information (Standards, Codes, Recommendations, Outage data, etc.) available that can justify data center electrical preventive maintenance?
 

jim dungar

Moderator
Staff member
Location
Wisconsin
Occupation
PE (Retired) - Power Systems
Any information (Standards, Codes, Recommendations, Outage data, etc.) available that can justify data center electrical preventive maintenance?
They needto do a cost analysis of an unplanned outage due to a breaker not transferring from normal to standby mode?

The last data center I was in, had the bypass breaker in a non-operative mode. There is no way the transfer scheme would operate as it was designed, therefore the data center did not have the redundancy they thought they did. To my knowledge, it has not been repaired.
 

ron

Senior Member
It is a calculated risk made by the owner.
There are loads of standards like NFPA 70B and NETA, but none require compliance because they are not code.
The only time they may come into play, is if there is a catastrophic failure due to lack of maintenance occurs and someone gets hurt, then OSHA comes knocking with a fine, because a nationally recognized standard was not followed.
You could squeeze a little bit of Code out, if you find the equipment manufacturer has PM requirements, then the NFPA 70 (NEC) requires compliance with the manufacturers instructions.
 

alfiesauce

Senior Member
Sounds like the poster knows that you have to convince the owner that it's cheaper to fix it before it breaks then when it breaks, but as for sources to help prove that point.
What services are you providing? Infrared camera imaging? Backup Generator testing? Load testing? UPS battery Maintainance?

Try googleing data center preventive maintenance
You'll get lots of hits- filter though it. Below was one of the first ones I saw-
http://www.comnews.com/wp_library/APC_whitepaper.pdf
 

charlietuna

Senior Member
When insurance companies that provide "down time" insurance, offer sizable credits for proper electrical preventative maintainance--there is a proven reason. Being in the infrared scanning business since 1989 i have seen first hand, positive results in many installations where the failure of electrical systems is critical. And i have witnessed failures of critical systems just due to the failure to operate emergency backup systems on a weekly or monthly basis! There is a big difference to have the ability to plan your next next power outage in the data industry..
 

petersonra

Senior Member
Location
Northern illinois
Occupation
engineer
Any information (Standards, Codes, Recommendations, Outage data, etc.) available that can justify data center electrical preventive maintenance?

My personal opinion is that much of what is sold as preventive maintenance is basically a waste of resources. A lot of bigger companies that were on this bandwagon years ago have realized its dubious value and quietly stopped wasting money on most of it.

I am a big fan of doing actual testing of emergency and standby systems on a regular basis though. I also think its a good idea to periodically run off the backup/standby systems for s few hours just to make sure they will work when called on. Very few places do much more than fire up their generators once a month for a few minutes without load and call it good.

A long time ago, a certain data center experienced a power failure. The UPS systems came on line fine and kept the computers up. The air conditioners for some reason did not like the power they got from the UPS systems and all of them shut down, that would have caused the computers to shut down or fail from overheating. The story I heard was that it being winter they managed to cool the computers down by breaking out some windows and bringing in cold air (I gather the maintenance people used sawzalls to cut holes in interior walls to get the cold air into the computer room). This led to a static electricity problem, but somehow they kept stuff going, although I heard the final bill for repairing the stuff they lost and damaged keeping things going was several millions dollars.
 

ohm

Senior Member
Location
Birmingham, AL
Sounds like the poster knows that you have to convince the owner that it's cheaper to fix it before it breaks then when it breaks, but as for sources to help prove that point.
What services are you providing? Infrared camera imaging? Backup Generator testing? Load testing? UPS battery Maintainance?

Try googleing data center preventive maintenance
You'll get lots of hits- filter though it. Below was one of the first ones I saw-
http://www.comnews.com/wp_library/APC_whitepaper.pdf

Managers are responsible for making a profit. Preventative maintenance is a cost which lowers profit.

No money is available for preventative maintenance. Tons of money is available for corrective maintenance. Once they're up and running..there is no more money available.
 

Natfuelbilll

Senior Member
Any information (Standards, Codes, Recommendations, Outage data, etc.) available that can justify data center electrical preventive maintenance?

My questions go beyond routine testing inspections as these are being completed on UPS and generators.

What is the value of shutting off equipment and physically inspecting wiring terminations and connections on power, controls and communications? Any good data out there?

How complete of a PM can be made without de-energizing and "shutting down the data center"?

IR is being considered. What problems won't IR pickup? I know some circuits are either "spared" or lightly loaded. Is adding temporary load a recommended means of uncovering problems on those lightly loaded circuits?

What types of specs or PM scopes do monster (IBM, GM, Microsoft, Goog, US Government, Los Alamos, etc.) companies follow?



How much of NFPA 70B... (I don't have my copy this morning)... is done live .vs. de-energized?
 
Any information (Standards, Codes, Recommendations, Outage data, etc.) available that can justify data center electrical preventive maintenance?

Power analysis software often has a reliability model where failure analysis can be performed. That will tag real numbers to failure probabilty and slo give you a tool to analyse how can you reconfigure the system for better reliability.

NFPA 70B contains recommended maintenance practices.
Each manufacturer have maintenance recommendation.
IEEE493 - Gold Book - is the national standard for power reliability.

Any power system without maintenance WILL fail.
 
My questions go beyond routine testing inspections as these are being completed on UPS and generators.

What is the value of shutting off equipment and physically inspecting wiring terminations and connections on power, controls and communications? Any good data out there?

How complete of a PM can be made without de-energizing and "shutting down the data center"?

IR is being considered. What problems won't IR pickup? I know some circuits are either "spared" or lightly loaded. Is adding temporary load a recommended means of uncovering problems on those lightly loaded circuits?

What types of specs or PM scopes do monster (IBM, GM, Microsoft, Goog, US Government, Los Alamos, etc.) companies follow?



How much of NFPA 70B... (I don't have my copy this morning)... is done live .vs. de-energized?

Inspecting connections in the de-energized state is a pretty useless excercise IMO. IR with load reading gives you the predictive tool taht allows you sufficient time to plan outage.

Any system that requires high availability should be planend that maintenance and repair can be performed without shutting the system down or that any repair, replacement or maintenance can be performed within the alloted time for outage.

Predicitve maintenance replaced preventive maintenance.

Any decent maintenance program should be tailored to the specific configuration.

Frequency of maintenance for electrical equipment should be based on a matrix where the frequency is determined based on the conditions - temperature,ambient conditions, dust and moisture - under which the system is due to operate and the load/switching frequency of the system.
 

charlietuna

Senior Member
Management companies are in business to make a profit--but they also need to keep their building's full of tenants to produce a profit. Many maintain a reputation to provide five star services to their tenants which works by looking at the names of their tenants. These management companies have buildings in every major city which remain 90+ in occupency! We had the oppurtunity to work for them as well as their competitors across the street in buildings with similar sizes and construction. The five star buildings had their own specs as far as testing-weekly,monthly and yearly. They were the first buildings to ask for infrared scans and load tests on their generators. One of them had 42 asco transfer switches--and six generators, four of them 12,000 kw. The would have their switchgear cleaned and tested every two years and all main breakers tested. This preventative maintainance was common to all building systems,electrical and mechanical. We were in a position to easily see the advantage of schedualled maintainance. At first i thought this was overboard -- but as time went on i realized those buildings insisting on proper excersize and equipment maintainance had very few critical failures effecting tenant operations. Just consider 42 transfer switches(relay type) and all the moving parts required to function for a full transfer sequence. And items that fail raises a flag-which allows them to change this part on other devices BEFORE they fail. The same with infrared scanning, they would see a heating problem in elevator disconnect switches -- rather than wait for them to fail -- they had us upgrade their sizes to prevent failures. These companies will tell you -- Preventative Maintainance does not cost--IT PAYS !
 

dbuckley

Senior Member
How complete of a PM can be made without de-energizing and "shutting down the data center"?

Depends on how the data centre is designed. A data centre designed with electrical Tier 3 or better is able to be "concurrently maintained", so no downtime due to maintenance.

Anecdote: got this message in a log:

Dec 18 09:48:01 bigip2 system_check[1760]: 010d0006:0: Chassis power supply 102 is not supplying power (status: 0): make sure it is plugged in.

I enquire of the data centre people, and they tell me that distribution B is offline for installing a new static switch. Distribution A is still up, so everything is still working. And so it was, no outage to anyone.

They also reminded me that it was declared on the change control calendar, but thats another story...
 
Status
Not open for further replies.
Top