Planning for a scheduled Electrical Outage

[Installer]

I'm trying to write a procedure for a schedule Electrical Outage. The Outage is necessary because wish to move a Circuit from the Main Essential Panel to the Main Critical Panel.
The Outage will affect a building containing a lot of Electronics Equipment.
This is a planned removal and restoration of power at the Main Breakers of both Panels, so I don't thing Surges or Spikes are an issue.
I'm not that experienced in this and this is all I can think of:

1 All electronics equipment not on UPS should be shut down prior to shutdown of Panel ELPA. Restoration of power may result in loads drawing high startup currents initially cause fluctuations in voltage.
2 All computers should be shut down through their Operating Systems to prevent software corruption.

I really can't think of much else. Please don't laugh, but I'm assuming this isn't such a big deal, or is it? Can sensitive Electronics equipment be damaged by removal of Power at the Outlet vs removal of Power by the On/Off Switch?

Thank you in advance.

 

[PaulMmn]

1A. UPSs don't last forever-- depending on the size of the battery, they may only run 15-30 minutes. So unless your circuit move is very fast, it would be a good idea for all of the computers and other equipment to be shut down.

2A. You need to specify a time when the balloon goes up! "All power to building 42 will be shut down at precisely 18:47:13" And send reminders every half hour at least 3 hours in advance!

Usually electronic equipment needs to have a power supply that understands power failures. Your mileage may vary. Programs may not be as forgiving!

The people in charge of the affected building should know how to power down their equipment gently.

As far as transients go-- Does the affected building have its own panel-- or does the room with all the electronics? If so, once they've powered off all of their equipment, I'd suggest throwing the breakers (or the main) for the building or equipment room. Just in case!

 

[iceworm]

Random thoughts:
My work is all industrial, and generally shutdowns are a big deal. A shutdown procedure is mandatory. The shutdown procedure will be a checklist of all the stuff that has to happen (and verified) before de-energization, and again before re-energization.

Q: Does anyone in the facility worry about damage to electric powered equipment when a utility outage happens? Let that be your guide. Orderly shutdown is still a good idea.

For a commercial offices, some additional concerns might be:

bathrooms
fire protection
elevators
emergency lighting​

If the facility is commercial and does not have continuous process, consider nights and weekends.

 

[zbang]

The last time I went through a major shutdown, we (with a lot of help) had to identify pretty much every piece of electronics in the building, who owns/manages it, how long it takes to shut-down and start up, and probably some things I've forgotten. We also had to get everyone to agree to the shut-down and startup sequences. It can take weeks/months to develop for a large and complex setup.

Will this affect fire alarms or security systems? That could mean additional guards/watches.

Are there other tasks that you can work in during the outage, even if it extends that by a couple of hours? Basically, if you're going to do a shutdown, get your money's (time's) worth out of it. (Panel/switchboard cleaning/PM, breaker test, rearrange other circuits if needed, etc)

 

[steve66]

Exact procedure is usually noted and followed: Shut down main panel, remove wires, turn main panel back on, shut down critical panel........

Along with any safety checks: shut off breaker, test for voltage......;

 

[Installer]

The biggest thing I'm worried about is we have rack after rack of Electronics equipment. My belief it all has to be powered down before the Outage. The only way I can justify requiring the manpower to do this is if I can definitively prove the statement that bring power back on after an outage sometimes results in spikes or fluctuations in power that can damage equipment. But I can't find anything that substantiates this. That being the case, is using the Main panel as on/off switch for , say a Phone Switch, or a Microwave Radio receiver, much different than using the equipment's on/off switch. I really can't prove that.

 

[iceworm]

The biggest thing I'm worried about is we have rack after rack of Electronics equipment. My belief it all has to be powered down before the Outage. The only way I can justify requiring the manpower to do this is if I can definitively prove the statement that bring power back on after an outage sometimes results in spikes or fluctuations in power that can damage equipment. But I can't find anything that substantiates this. ....

So, your management is saying, "Unsubstantiated claims of, 'I don't like it." does not have any credibility."
Yes, that is generally true.

Where are you looking for creditable evidence?

Has there ever been a utility power outage? If so, what happened when the power came back up?
No power system is zero failures. What is the plan if the power goes out?
Check the maintenance records on the suspect equipment. Do any failures coordinate with any power outages?​

I'm thinking that the original design for the power system feeding "rack after rack of sensitive (my addition) electronic equipment" would have surge (spike) suppression and noise filtering. And if not - maybe the equipment is not particularly susceptible to spikes/surges.

If you are not finding any evidence, There are only about two things left.

1. Point out the cost of manning up to do an orderly shutdown is $XX and the cost (including downtime) to replace one alien containment field is substantially more.
2. Check to make sure the normal compliment of spares is on-hand.

What ever decisions you get, don't look back - have at it.

 

[PaulMmn]

If the management is confident that you can switch the power off/on with impunity, make him put it it writing!

 

[iceworm]

...
2 All computers should be shut down through their Operating Systems to prevent software corruption.
...

Two things here:

1. If the computers are data processing or process control, and are subject to failure on power outages, they better damn well be on high reliability UPS systems. If the company management does not know that, then nothing you are doing will bother them.
2. Any outages that involve desktop computers, we always warn the users a few days ahead of time, and then again on the day. Tell them to save their work - power is going out.

 

[PaulMmn]

Two things here:

1. If the computers are data processing or process control, and are subject to failure on power outages, they better damn well be on high reliability UPS systems. If the company management does not know that, then nothing you are doing will bother them.
2. Any outages that involve desktop computers, we always warn the users a few days ahead of time, and then again on the day. Tell them to save their work - power is going out.

Your point #1 - unless the UPS includes a generator, the UPS probably won't last long enough for the switch over. And will the generator be involved in the switch over?

 

[iceworm]

If the management is confident that you can switch the power off/on with impunity, make him put it it writing!

I've never seen that work well. I got caught up in one of those. I didn't like the OSHA exception to working hot for a particular job. The top management was not about to put out a memo saying they thought it was, "Safe". It took a couple of years for the bad blood to go away.

Document the concerns.
Document reasoning - even if it is "I don't feel good about this."
Copy everybody on the emails.
Don't expect any management answers. There is nothing in it for them to go out on a limb.
Move on with the job.

 

[iceworm]

Your point #1 - unless the UPS includes a generator, the UPS probably won't last long enough for the switch over. And will the generator be involved in the switch over?

From my side of the OP keyboard, I can absolutely, with extreme certainty, tell you:
.
.
.
.
.
.
.
.
.
.
I wouldn't have a clue.

That would be part of shutdown planning - wouldn't it?

 

[iceworm]

Your point #1 - unless the UPS includes a generator, the UPS probably won't last long enough for the switch over. And will the generator be involved in the switch over?

I must not have phrased my response very well. You are pretty good at picking up on most any of this. I'll try again:

If the management does not understand that they have equipment that requires an orderly shutdown, and that if not orderly shutdown, will fail on start up - there is nothing the OP can do to fix that.

No power system, built by man (well, maybe built by women would be slightly better reliability) is 100%.

If the management does not see a need for a plan, there is nothing the OP can do to fix that.

In post 6, the OP said he couldn't substantiate any issues with the powering on equipment causing damage.

I don't see an issue here

 

[zog]

What is the condition of the main breakers you will be opening for this outage? Have they been properly maintained? Do you have a spare on hand?

Often we see shutdowns like this where the main is a 50 year old bolted pressure switch that has never been serviced, good luck getting that thing closed again.

 

[GoldDigger]

Two things here:

1. If the computers are data processing or process control, and are subject to failure on power outages, they better damn well be on high reliability UPS systems. If the company management does not know that, then nothing you are doing will bother them.
2. Any outages that involve desktop computers, we always warn the users a few days ahead of time, and then again on the day. Tell them to save their work - power is going out.

Not so much a thing anymore, but back in the late 60's a typical large IBM mainframe (e.g. 360/90) was expected to require replacement of several logic boards to restore operation after a 100% planned power shutdown. There were so many components that could possibly suffer from power cycling that you could almost guarantee that one or more would fail. An unplanned power outage could be even worse as disk and tape drives could damage media during an unexpected power loss.

 

[PaulMmn]

Along the same lines as the 50-year-old breaker that's never been thrown--

Computer disks on mainframe computers usually run forever. They are known to suffer from 'sticktion' when they power down completely. The read-write heads actually 'land' on the disk surface. When they try to start up again, the oomph of the motor isn't enough to break the sticktion between the disk and the head.

Sometimes a more-or-less-gentle >thwack< in line with the disk platters is able to get it going again... but your mileage may vary!

So if the OP has computers that have not shut down in a while....

[definitely NOT related-- but New York City has 2 main water valves that cut off water to the East Side or the West Side. These valves have never moved since they started letting water through them!]

 

[Installer]

Problem is we are doing it at night --Saturday night--and everyone wants to be home. And I'm writing the procedure and they will be cursing me --but that's life. If everyone likes you, you are doing something wrong

 

[Tony S]

I wish you luck. The moment you kill the power someone will show up shouting we need power to XXXX.

Everything planned, I moved six transformers and associated switchboards. It developed in to a nightmare. The substation looked like a spaghetti factory explosion with temporary feeds here there and everywhere.

 

[zbang]

Computer disks on mainframe computers usually run forever. They are known to suffer from 'sticktion' when they power down completely. The read-write heads actually 'land' on the disk surface. When they try to start up again, the oomph of the motor isn't enough to break the sticktion between the disk and the head.

Sticktion was also a problem with IIRC some Maxtor drives in the mid 1990s.

...some mainframe drives use three-phase motors... after a wall of disconnects was moved to another place in the machine room, everything came up properly but one row of drives.... the ones with three phase motors.... took a few people watching closely to notice they were spinning backwards ... no damage done since the controllers could tell, they just didn't tell anyone...


It all comes down to whether the customer is so confident that their equipment will all come up at once, or whether they're realistic about it. (And some interconnected computer/network/data systems have to start in a specific order- C depends on B being up and B won't work properly unless A had been online for a few minutes.)

 

[cowboyjwc]

My thought is this, if an unplanned outage doesn't hurt the equipment then what's the difference between that and a planned outage. Yes years ago it would take a long time to restart some of this equipment, but now days not so much.