If this post gets a little gibberishy, I have a very good excuse. The best.
See, since I woke up around six ayem Saturday, I have not slept. Other than a quick nod in front of the TV while breakfast (canned corned beef hash) was sizzling, at least. I'm eating brekky now.
Why? It's a long story; I slept like a tired, tired thing, Friday-to-Saturday, 11 hours! --And I had to work an overnight shift, Saturday-to-Sunday. Alas -- of course -- there wasn't Nap One in me. I did manage a nice, long, relaxing and half-awake soak in the tub (if I had a hot tub or a lap pool, I'd be a semi-aquatic mammal) but that was all.
The work was one of those necessary but dreadful things: a live generator load test. That's live as in, "at oh-dark-thirty or a little earlier, we started up the megawatt generator and while the facility was in full, normal operation told the automatic switches to go kerchunking from perfectly good Power & Light power to our very own hand-knit power, with on a pathetic few tens of kilowatts of heaviliy-loaded UPSes to try to carry the building over the switch.
Naturally, madcap hijinks ensued; one gigabit ethernet switch simply curled up and bade this world farewell, a score of computers crashed (none super-critical but most of them darned annoying to lose), the opper-poppers began pipping instead of popping and in general, four, eventually five of us danced our way through a merry fire-drill of fail, reboot and try again.
It was serious. Matters stretched well past the cat's breakfast time and with Tam away at class, I could only imagine the mayhem Huck was committing, trying to get fed. Eventually we got it all done, or done enough, and I stumbled home, to feed the cats, open up a can of goodness (Mary Kitchen brand, still edible), pour a mug of tea and sit dizzily down here at the computer to relate the tale to you.
Really, all it's missing is QueeQueeg's coffin and a peg-leg sea captain dragged to the bottom of the sea by a dead or dying whale. Call me-- Tired. Or Sleepy. My breakfast is et and I'm ready for sleep.
BUILDING A 1:1 BALUN
4 years ago
10 comments:
So, it sounds like you were dealing with Break-Before-Make Switches?
I always preferred just spending the time to ensure everything was in up to speed and in sync and even though we had Make-Before Break Switching on the Subs, I always crossed my Fingers.
Sometimes, you just have to watch the Dials and Pray.
Sweet dreams there Ms. Roberta.
Of the non-madcap hijinks kind.
Yes, Les, we have to. The genset has to be able to go on line automatically, at any time, withough hanyone from my department being there to handhold it through the transition.
They make automatic synchronizers, and 99.99% of the time the darned things work. What haunts my nightmares is the 0.001% of the time, where you're explaining to the boss on one phone why so many important, major widgets are dead, and to Power & Light on the other phone about the blown-up stepdowns....
At one time, we were looking at a flywheel UPS with the genset on the same shift through a clutch. The problem there is that shutdowns to replace the bearings every two years tend to be traumatic -- and in-service bearing failures, even more so.
What we have is a compromise. Fairly frequent "glitch hunts" to find what should be UPSe and isn't are the key t lessening the impact -- and in that light, last night's activities were quite a success. What's the term? Pyrrhic victory? Too many more successes like that might doom us!
Rick, I don't remember a one of them. Possibly just as well.
Or you could come have fun with me: Things that should have UPSes but don't because they cost too much and need to be maintained. Things that should have generators and don't, same reason. Add a 24 hour weather-induced power outage, then stand back and watch the chaos. Finger-pointing and loud kvetching and moaning at no extra charge.
I spent a few years at the primary computer operations center for a company that makes many many things both farm and construction related. Shortly before my departure to another unit the facility upgraded the onsite generator system from 1 MW to 3 MW and upgraded the UPS as well. There was testing...oh was there ever testing. And wouldn't you know that in the decades since the original gen sets had been enshrined in their little building the damnedest things had been placed on circuits together. Temporary circuits had grown old and spawned children of their own even the resident electricians were none to sure about some of the breakers. So one day when we had all sorts of power available (two completely separate lines from local utility, the old generators, and the new generators) some sub sub sub contractor was load testing some little section of the project. There were several load banks in the parking lot by which the various generators were converting diesel into heat and fat cables stretched in the most annoying way possible along corridors. Sub^3 contractor flipped his breaker. My kitchen died. Everything important stayed alive but the kitchen. This was annoying. So I strolled out to ask the assembled electron wranglers if I could pretty please have a few sparks to at least keep the ice cream frozen. Blank looks. I very politely explained that if I didn't get at least a little juice for the vent hoods and a couple of lights that there would be no lunch today and if it went much longer then tomorrow came into grave doubt. More blank looks turning worried.
$15000 later I got a chance to scrub all sorts of places in the freezer I'd never managed to get before.
Sub^3 was never seen again. I fear the programmers may have lynched him.
So glad I'm out of that nut hatch.
BGM
One of our requirements is that receptacles and switches must be marked with panel and breaker ID, and panels have the same, with "fed from panel nnn, breaker mmm" right on the face of 'em. Conversely, the little list of what-breaker-feeds-what inside the door is supposed to be up to date. So you can, supposedly, trace from source to load or load to source without looking at a diagram or resorting to circuit-finders.
In practice, it only takes one guy too hurried or heedless to leave traps for the unwary, just because a couple of minutes with a pencil and Sharpie were too much effort.
I'm not even going to bring up the time we killed sixteen lead-acid batteries in a single 12-hour outage. (Can you say "leaky"? Sure you can.)
Sweet Dreams!
OUCH! Glad y'all were able to 'recover' most of the systems, and now it's time for YOU to recover the system...
Next time you talk to the guys at channel 20, ask them about St.Vincent North's load tests vs. solid-state transmitters.
OT- Hope Mom X is doing well.
Post a Comment