Mysterious vibration in long conduit between solar inverters and main service panel

Status
Not open for further replies.
Please bear with me as I'm not an electrician but a certified thermographer with more experience in simply identifying electrical anomalies that represent potential issues than actually understanding or mitigating them. I usually just point and say, “ya know, sooner or later the smoke’s gonna come out of that one. You might want to fix that while the guys are off one weekend soon. Either that or stock up on brooms so you can keep ‘em busy during your impending unscheduled shutdown.”

I was called into a 50,000 sf metal commercial building that had a 125kW rooftop solar array installed about two years ago. Over that time a vibration with an audible hum that corresponds to how heavily the system is loaded has developed and according to building occupants continues to get worse with time. Under full sun it’s loud enough in some office spaces to drive occupants away from their desks for the afternoon.

The installing electrician and local utility have both evaluated the system, installed power monitors, etc., and have no idea what’s causing the issue.

The arrays feed into 11 inverters on the south side of the building. The inverters feed into a combiner box that routes 3 phase power via aluminum conductors 250 ft away to an AC Solar Disconnect (ACSD) next to the Main Service Panel (MSP) at the north side of the building. Due to the long distance and several corners the owner elected to use aluminum wire splitting the load across redundant conductors, with a pair of parallel conduits each carrying all three phases, neutral, and ground.

It is along these conduits through the building that the vibration hum is observed.

Array components, inverters, and combiner box panel are grounded back to the MSP via these parallel grounds. The parallel conduits, which pass through the panel boxes at both ends with metallic hardware, themselves represent yet another parallel ground path. None of the array, inverter, or combiner box components have separate grounds at their locations, only the connection back to the AC Solar Disconnect, which is then bonded to the MSP, which is in turn connected to the building frame, with a separate GEC originating just a few feet away and entering the slab, interrupted by a strap around a water pipe along its path.

Ultrasound indicated only audible frequencies so we can remove corona or other high frequency sources of vibration.

Using FLIR I determined that both the combiner box bus bars and main disconnect near the inverters and the knife switches in the 600A AC Solar Disconnect 250 feet away were over 72°F rise over ambient, which NEMA Thermal Severity Criteria classify as seriously deficient - and this was on a day with a thin cloud layer so we were only seeing about 75% of production capacity, or about 200 amps per phase so it’s unlikely the panel components ever see more than 50% of their rated capacity.

Hot spots observed appeared to originate at points of contact in the switchgear and thermal gradients showed heat dissipating along the length of the conductors. I did not observe significant inductive heating where conductors exit the conduits, which is sometimes the case in certain situations.

Significant vibrations could be tangibly felt by hand along insulated sections of conductor accessible well away from contact points within the AC Solar Main Disconnect.
I theorize there are two possible contributors to the problem vibration and it could be either or both simple induced currents due to inductive effects and/or a ground loop owing to the multiple parallel ground paths connecting the inverter combiner box to the AC Solar Disconnect. At the terminal end it appears the conductors exit the conduit and enter the box under some lateral strain such that some but not all of them are in pretty tight but insulated contact with the edge of the conduit/panel interface. I have seen examples where conductors coming off a high current bus bar and mechanically fastened to the panel using metallic clamps or cleats can buzz like the dickens, and the problem goes away when the cleats are replaced by nonmetallic straps. Could the conduit itself be acting as the secondary in a crude linear transformer and that’s the only thing causing the vibration?

But upon further inspection of the ground system I have trouble not believing it's at least part of the equation.

First off, we have those multiple ground paths between far ends of the building, all running in parallel to current carrying phases and neutral. Again, with no separate ground at the point of the inverters/combiner box. All components within the system are bonded together and tied to a single ground point at the line side of the building’ MPS.
Second, I measured what some would classify as ‘objectionable current’ * in the ground itself, totaling as much as 8amps, split unevenly between the two parallel grounds at about a 2:1 ratio. Note that these ground conductors share the exact same origination and termination lug points at each end of their run but run through separate parallel conduits along with the other parallel redundant conductors only in order to reduce the required wire size for the installation.

(*I did enough searching for answers before posting this that I am aware “objectionable current” is poorly defined and can be a hotly debated topic but objectionable or not, I don’t run across it often and it strikes me as unusual.)

I also measured what I perceive as odd currents in the ground wire running from the MSP to the building frame at about 2 amps, then about the same 2 amps along the next ground wire a few feet away running from the building frame to where it’s interrupted by the water pipe, but it jumps to about 8 amps from the water pipe to the slab. Unfortunately I only had a Fluke T5 on hand so my CT couldn’t wrap around the pipe but I’d have to assume there’s 5 amps in that as well.

The one other ground effect observation is that when we went on the roof (metal roof deck with a recently applied thick elastomeric treatment), every person in our party received a significant static shock upon first contact with any metal component. None of the grounding cables running from the array panels to the rooftop combiner boxes nor the combiner boxes to the inverters appeared to be carrying any measurable current.

From that my non-electrician brain would infer that there is a significant potential difference developing at different grounded points within and on top of the building. Whether this is or is not exclusively or even partly responsible for the audible hum in the conduits remains unknown to me.

I am about to suggest having the electrician carry out a few tests the client might carry out in order to eliminate potential sources of the problem.

  1. Going for the obvious problem first, I suggest they mitigate the heat issues identified with the FLIR survey. Lug connections had already been re-torqued before my arrival but obviously fixed nothing so I suggest removal and visual inspection with renewal of conductor endpoints with fresh application of Noalox if warranted. If upon re-inspection the combiner box main disconnect is still flagged under NEMA Thermal Severity Criteria I’ll recommend replacing that component. I'll be surprised if this eliminates the hum but it needs to be done regardless.
  2. Since we have redundant parallel ground conductors between panels, I suggest lifting one of them under controlled conditions to see if its removal has any effect. If it does not, we might need to try to figure out a way to temporarily also remove one or both of the electrical bonds represented by the conduits themselves. Logistically that will be much more challenging but I believe there must be available some sort of insulating bushing that could be installed to isolate the conduit from the panel to which it is mechanically fastened. Unfortunately this will require substantial disassembly of at least one of the panels to achieve.
  3. We might also try temporarily lifting the ground connection to the water pipe, and/or tying the MSP directly to the ground by removing its connection to the building’s steel frame.
  4. We could try installing a direct ground at the location of the inverter combiner box.

I realize any or all of 2 through 4 might be unconventional, non-code compliant, and perhaps flat out unsafe in the long run. But for purposes of figuring this out I believe – IF it is possible to do the tests safely under tightly controlled conditions – we might be able to collect some very helpful observations.

Or better yet, one of you geniuses who has read this far and hasn’t already decided I’m an idiot and not worth the trouble might already know exactly what’s going on here and can just tell me? :D

I thank you in advance for your indulgence and any thoughtful responses this might elicit.
 
Welcome to The Forum.

High currents develop high magnetic fields, enough to make the wires chatter in the conduits. If this noise and vibration was not present initially, I would look for loose hardware contributing to conduit movement and objectionable sound.

Depending on what and where the vibrations are, you may be able to dampen them with rubber sheeting or wedges.
 
Comments, such as I can make:

- The 8amps on the solar ground could be from capacitive leakage currents, if the DC side of the solar system(s) are grounded. That is a common feature of such systems, and 8 amps is within reason for a 125kW system.
- The 2amps on the main premises GEC likely just represents building load having nothing to do with the solar.
- I'm somewhat doubtful that the parallel ground paths for the solar grounding is inherently contributing to the noise problem. I would more likely suspect insufficient bonding of some metal component to contribute to the problem; that is, trying to unbond certain grounding conductors and components and not others would, to my intuition, possible make the problem worse. It might be interesting to try to unbond and unground all components for a short (minutes) test to see what happens, but it would be pointless to advise on how to do that permanently since it would be non-compliant and significantly unsafe.
- Has the inverter manufacturer been contacted for any comment or advice?
- Your item (4) would probably be code compliant but is unlikely to make any difference. Ground rods really don't do much at those voltages.
 
The easiest and most likely productive option would be to pull the wires out of the conduits, twist them tightly and pull them again (hopefully you will have enough extra length to do this, otherwise replace the wire.) This will both reduce extended magnetic fields and keep the wires from vibrating against the conduit and each other.

Sent from my XT1585 using Tapatalk
 
The easiest and most likely productive option would be to pull the wires out of the conduits, twist them tightly and pull them again (hopefully you will have enough extra length to do this, otherwise replace the wire.) This will both reduce extended magnetic fields and keep the wires from vibrating against the conduit and each other.

Sent from my XT1585 using Tapatalk
Number of times I've found a problem with conductors inside the conduit, 10. Number of times I've found a problem with the way things were wired, 1000.
 
Number of times I've found a problem with conductors inside the conduit, 10. Number of times I've found a problem with the way things were wired, 1000.
A valid point. Assuming, for the moment, that there is some kind of problem with the wiring, I would probably put a clamp-on ammeter (amp-clamp) around the three phase conductors from each conduit to make sure that the net current is near zero. A relatively small extra termination resistance or other problem on just one wire could lead to very different currents between the two elements of what should be a parallel path. That imbalance could then cause non-zero net current in both conduits. Just one example of a potential problem that would not be seen by simple continuity checks.
My next step would be amp measurements on all of the individual conductors.
 
Ultrasound indicated only audible frequencies

So, what are those frequencies ? all 60 Hz harmonics or ? Any current waveforms - elg harmonics?

What is the PWM operating frequency of the inverters?

Conduit span between clamps?

For instance, if the objectionable noise is at say 180 or 540 Hz, one could infer in phse harmonics on the core wires leading to the magnetic forces already mentioned. A 540 Hz whine sure could drive folks out of an office.
 
Net Currents

Net Currents

Based on 2 jobs we have done several years ago, My gut feeling on this is that the root cause of the vibration and humming noise is due to net currents (as others have mentioned); currents that return to the source thru system grounding paths (conduits, building steel, metal enclosures, metal raceways, metal roofs, etc) instead of current carrying conductors. In both cases the symptoms were the same: screen jitter on the old CRT monitors of all the work stations. One we found using a Gauss Meter to measure the unusually high EMI in the affected areas. Also very high ground currents measured on the ground cables terminated on the ground bus of the main power panel. This one turned out to be loose bolt connections on the neutral bus (4000A service) bars at the outside utility pad mounted transformer. The other cause turned out to be the miss-match of the connectors between work station old and new partitions. During expansion, the new type partition sections caused the 120v hot conductors to connect to the ground conductors of the adjoining partitions resulting in net currents. If these currents are high enough then it could be the source of the humming since steel is not designed to carry current.
Just a thought....hope this helps.
 
Wow, thanks for all the thoughtful responses so far!

Wow, thanks for all the thoughtful responses so far!

It will take me a little time to digest all this and respond.

For what it's worth, the utility analysis was pretty comprehensive and they provided data on current and voltage waveforms at all three of the aforementioned service panels as well as THD on both I and V. I'd have included that info in my original post but ran up against the forum's character limit as it was. In any event nobody seemed to think there was much to go on from all that, although I did notice after we'd measured the odd ground currents that they also saw about 8A on the ground but didn't flag it. They did not, as far as I can tell, try measuring net current by capturing all the phases together to see if there was anything missing that must be running outside the pipe. I think that's a very good suggestion and I'll try to get that accomplished.

I'll see if I can roust up a Gauss meter too. It's almost too bad we don't still have the occasional CRT under a back table in some hoarder's office but I think we've past the point where any building less than 10 yrs old will not likely have ever housed a CRT to begin with.

More later, after the wife gets some Sunday chores out of me. Thanks again!
 
what size conductor?
what size conduit? what material? how as it supported?

the fact that it proportional to load is telling
 
It will take me a little time to digest all this and respond.

For what it's worth, the utility analysis was pretty comprehensive and they provided data on current and voltage waveforms at all three of the aforementioned service panels as well as THD on both I and V. I'd have included that info in my original post but ran up against the forum's character limit as it was. In any event nobody seemed to think there was much to go on from all that, although I did notice after we'd measured the odd ground currents that they also saw about 8A on the ground but didn't flag it. They did not, as far as I can tell, try measuring net current by capturing all the phases together to see if there was anything missing that must be running outside the pipe. I think that's a very good suggestion and I'll try to get that accomplished.

I'll see if I can roust up a Gauss meter too. It's almost too bad we don't still have the occasional CRT under a back table in some hoarder's office but I think we've past the point where any building less than 10 yrs old will not likely have ever housed a CRT to begin with.

More later, after the wife gets some Sunday chores out of me. Thanks again!

just addressing the sound, and not the cause, i've heard rattling of conductors in conduit twice
in my life. the loudest was at anaheim convention center, with a bolted fault on an 800 amp
trade show power panel in the overhead. it almost sounded like someone using a chipping hammer
against the conduit. it tripped a 3000 amp section.

i don't see an 8 amp current flow causing enough flux to make noice sufficient to drive people from
their offices nearby, no matter what type of leakage or flow it is.

i'm thinking harmonics, and a standing wave built up, resonating in the conduit. i'd get half a dozen
sandbags, fill them partly, so they can be tossed over the conduit(s) in question, and see if that
changes the frequency or intensity of the sound. think frets on a guitar string.

it's fast and easy. cheap too. see if they will dampen the sound.
 
  1. Going for the obvious problem first, I suggest they mitigate the heat issues identified with the FLIR survey. Lug connections had already been re-torqued before my arrival but obviously fixed nothing so I suggest removal and visual inspection with renewal of conductor endpoints with fresh application of Noalox if warranted. If upon re-inspection the combiner box main disconnect is still flagged under NEMA Thermal Severity Criteria I’ll recommend replacing that component. I'll be surprised if this eliminates the hum but it needs to be done regardless.

Or better yet, one of you geniuses who has read this far and hasn’t already decided I’m an idiot and not worth the trouble might already know exactly what’s going on here and can just tell me? :D

I thank you in advance for your indulgence and any thoughtful responses this might elicit.

if you have enough length on the aluminum conductors, my suggestion is to remove the setscrew lugs,
dress the ends of the cables, and use hypress lugs, with the appropriate dies. then seal the ferrules
with either 3M cold shrink, or Panduit heavy walled UL listed 600 volt heat shrink.

use grade 8 fine threaded bolts, washers, and nylock nuts on the connections.

that will elminiate any termination heat, and not require PM's other than checking the torque
on the nuts periodically.
 
Wire size

Wire size

what size conductor?
what size conduit? what material? how as it supported?

the fact that it proportional to load is telling

It's all aluminum conductor.

Phases are XHHW-2 AL 350 KCMIL.

Grounds are XHHW-2 AL #3/0 AWG.

The conduits are 4" EMT that runs above suspended ceiling within the building so I haven't seen most of it but I would assume suspended from unistrut or the like.
 
Did the original installer screw up and mis-wire the paralleled conductors in the conduits? If all three phases, neutral, and ground aren't each present in each conduit, currents will be induced in the conduit(s). This could be the source of the buzzing.


SceneryDriver
 
Did the original installer screw up and mis-wire the paralleled conductors in the conduits? If all three phases, neutral, and ground aren't each present in each conduit, currents will be induced in the conduit(s). This could be the source of the buzzing.


SceneryDriver

That seems like such an obvious question sitting here now but if I'm honest, I didn't take the time to painstakingly track each and every wire to make sure nothing was routed incorrectly. I just assumed the colored phase labeling tape was applied before it was pulled and nobody would make such an error but I will definitely check this next time I'm there.

That said, it seems such an error would have showed up in the phasor diagrams produced by the utility's monitoring report, which looked normal failed to identify any such anomaly.
 
Welcome to The Forum.

High currents develop high magnetic fields, enough to make the wires chatter in the conduits. If this noise and vibration was not present initially, I would look for loose hardware contributing to conduit movement and objectionable sound.

Depending on what and where the vibrations are, you may be able to dampen them with rubber sheeting or wedges.

The installing electrician claims to have already been through everything and checked torque on all connections. He didn't comment however regarding tightness of structural fasteners along the conduit. They did try the old spray foam in the conduit trick but does that ever work?
 
Comments, such as I can make:

- The 8amps on the solar ground could be from capacitive leakage currents, if the DC side of the solar system(s) are grounded. That is a common feature of such systems, and 8 amps is within reason for a 125kW system.
- The 2amps on the main premises GEC likely just represents building load having nothing to do with the solar.
- I'm somewhat doubtful that the parallel ground paths for the solar grounding is inherently contributing to the noise problem. I would more likely suspect insufficient bonding of some metal component to contribute to the problem; that is, trying to unbond certain grounding conductors and components and not others would, to my intuition, possible make the problem worse. It might be interesting to try to unbond and unground all components for a short (minutes) test to see what happens, but it would be pointless to advise on how to do that permanently since it would be non-compliant and significantly unsafe.
- Has the inverter manufacturer been contacted for any comment or advice?
- Your item (4) would probably be code compliant but is unlikely to make any difference. Ground rods really don't do much at those voltages.


Thanks for all these comments, which address many of my questions directly. I just wasn't sure whether the ground currents I was measuring were normal or not.

The inverter manufacturer was contacted months ago and claimed to have never encountered a situation like this. All the inverters appear to be operating normally and other than the noise issue there have been no indicators they are not performing well.
 
The easiest and most likely productive option would be to pull the wires out of the conduits, twist them tightly and pull them again (hopefully you will have enough extra length to do this, otherwise replace the wire.) This will both reduce extended magnetic fields and keep the wires from vibrating against the conduit and each other.

Sent from my XT1585 using Tapatalk

I'm not sure either the building owner or the installing electrician would consider this easy given the run is 250 feet through the middle of a pre-existing building, which required rounding a few corners to accomplish. In a worst case scenario I imagine we could be looking at replacing the wire but there's no way that will happen before we can both positively identify the root cause of the problem and exclude all other feasible mitigations that might be more cost effective.
 
A valid point. Assuming, for the moment, that there is some kind of problem with the wiring, I would probably put a clamp-on ammeter (amp-clamp) around the three phase conductors from each conduit to make sure that the net current is near zero. A relatively small extra termination resistance or other problem on just one wire could lead to very different currents between the two elements of what should be a parallel path. That imbalance could then cause non-zero net current in both conduits. Just one example of a potential problem that would not be seen by simple continuity checks.
My next step would be amp measurements on all of the individual conductors.

The utility already did exhaustive monitoring over several days, including current, voltage, and THD on phases and neutral conductors at the inverter combiner box, the AC Solar Disconnect, and the building's MSP. They provided a report with 26 figures displaying their collected data, none of which raised any red flags from those who've reviewed them. I might be able to share that report here but not before I get a release from appropriate stakeholders.

As far as I can tell they did not try wrapping a single CT around all three phases at the same time to make sure current zeroes out. That seems like a very good suggestion and one I will try to facilitate. One of these days I'm going to spring for a pro quality power monitor but for the time being all I've got is a Fluke T5.
 
Status
Not open for further replies.
Top