R.G. Keen's Blog

Remote Power Management, Part Four: A Safer Button Pusher

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on May 16, 2010

I had one of those “Doh!!”, forehead slapping moments.

In the first Button-Pusher article I talked quite a bit about safety issues and the need to know what you’re doing with AC power line stuff – or wind up electrocuted. That was all straightforward and true.
But I think of these things like an engineer, not a hacker. The hacker side of me finally pulled through.
You don’t need to do all the AC safety mess if you have some widget that encapsulates all that for you. And we do, and it’s cheap and easy. It’s an AC to DC power adapter, widely know as a “wall wart”.

Just do the same circuitry after the rectifier diodes to do the internal stuff. But make the input to the computer side of this be a barrel style DC power connector. Plug an AC to DC wall wart in the controlled-AC power source, and plug the wall wart power connector into the barrel-plug socket on the computer. All done. No need to mess with AC power line. That’s all enclosed inside the wall wart. All that comes out is DC, and the dangerous bits have been pre-designed and pre-tested for safety and other things so that you never have to endanger yourself and others – at least any more than you do now. ūüėé

Lights-Out Management For Home Servers

Posted in Opensolaris Server, Remote Management by rgkeen on May 15, 2010

I find that there is a term for what I’ve been developing in the back of my head: Lights-Out Management. This is the idea that you can reset, power on/off, and in general tinker with and get back on line your servers. It’s a big deal at the commercial data centers, of course.¬† I managed to back into the concept because I want a server in the attic and/or garage and am too lazy to climb/walk to it to reset/power/tinker with it.

I have a friend who is a professional data center administrator. He’s on call 24/7/365, and is always getting pages to fix a problem with the hundreds of servers in their center. To him, Lights-Out Management (or LOM¬† as it’s called in the data biz) means the difference between fixing the problem from his home office and driving an hour each way to fix it.

Full LOM for a server includes power on/off and resetting the server as I’ve bumbled into, as well as remote access to low-level/BIOS setup, remote console support, and other things. This is so valuable that most real servers in the server world have some degree of it built in, perhaps for a fee.

But us home hobbyists can have some of that. I’ve outlined some of it in the Remote Button Pushers article. I’ve dredged up some additional info.

There is a card designed to add quite a lot of the LOM functions to an ordinary PC. It’s the PC Weasel Card. This thing gives you much of the LOM stuff, but to my way of thinking it misses a couple of them. HP/Compaq makes a similar card (google “RILOE”) that may work in other PCs.

First and foremost, it misses LAN connectivity, and it seems to not have a way to cut off/turn on the AC power.¬† I think LAN manipulation is necessary (for me at least!) because I by definition already have LAN wires going to the server. I don’t necessarily also have RS232/phone wires going there. Sure, complete out-of-band control is nice, but it’s a home server and if I really had to, I could walk/climb to the server and mess with it even if the LAN is down.

In my mind, a good compromise for my home server would be to have a LAN connected card that could (a) “press” the reset button, (b) “press” the power button, (c) toggle AC power on/off if neither a nor b did what I needed. That’s how I stumbled into the IP Power thing – which by the way, works nicely.

There’s another option. I’ve recently found some ethernet-connected I/O servers.¬† These are widgets that have an ethernet port at one end and some combination of digital I/O lines, RS232, analog I/O, USB, etc. on the other side. Most have some kind of embedded minimal web server so you can get to them by http and read/write the I/Os. The only problem seems to be that the producers of this ilk seem to want about $300-$1000 for them, a price comparable to a whole new server.

However, there are some that are under $100 if you’re willing to hack your own hardware. Here are two examples.

To make the PC Weasel card really useful, I could get to the RS232 port by using an ethernet-to-232 converter. Several of these exist. I think the makers of the Weasel are missing a bet here. And an ethernet to 232 that had digital I/O could then control AC power locally via an optoisolator  controlling a relay, either magnetic or SSR.

I don’t really need the BIOS/frame buffer stuff that the Weasel has. I’m happy with remote power on/off and attachment to an SSH session. So what I need is a button pusher. I have one a suitable one on order.

A Korean company by the name of Wiznet makes an ethernet-to-232 card (WIZ107SR) and an ethernet-to-digital I/O card (WIZ220IO). What’s novel here is the low price: The WIZ220IO is $35 and the WIZ107SR is $22. I ordered the WIZ220IO and will be talking about how it works here. I don’t really need the serial interface, as I’d rather use the SSH session, but the serial interface would really neaten up the PC Weasel.

So, I’m hot on the trail of LOM for my home server. More later!

Curses – Opensolaris doesn’t support Wake On Lan!

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on May 12, 2010

Well, mostly not. After some more time on google, I find that to make WOL work, you need

  • A WOL-capable motherboard
  • Setting the BIOS to enable WOL, or in newer boards “S5 on PME” wakeup events
  • A WOL-capable LAN adapter; some add-in cards do this, some don’t; and some internal LAN adapters support it, some don’t
  • A network device driver that supports WOL

It’s that last which Opensolaris fails on. The drivers may or may not have WOL inside them, depending on their provenance. And the WOL that may happen to be there may or may not work, or -worse!- may not work correctly, or reliably.

As I read when doing this research, the guys who form the Opensolaris cognoscenti do not think much of WOL. The general idea is that it’s at least useless, possibly worse. They vastly prefer wake from sleep/standby. Obviously this is a very broad generalization, and it’s what I inferred from a couple of hours of reading. Individuals may take issue with it.

I’m after remote wakeup and shutdown, as I mentioned in the post on remotely managed AC power. I had stopped work on the managed AC power button pushers while I investigated WOL. I’m now glad I did the AC button pusher work. It looks like I’ll need it.

I also found another suitable class of wakeup/shutdown devices. These go under the heading of embedded ethernet I/O devices. Some of them contain a minimal web server and can be run by a remote web browser. All of them have their own LAN port, and let you twiddle the I/Os remotely.

The application is obvious. These things can do the low-power button pushing and could conceivably have an AC power relay added. I’ll do more digging and see if this gets me closer to the poor-man’s “Lights-Out Adapter”, which is what the industry terms remote management setups.

Remote Power Management, Part Three: Button Pusher Circuits

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on April 28, 2010

I’ve designed a couple of circuits which implement the button pusher circuits for remotely pressing the power-on and/or reset buttons on an ATX style machine from a LAN controlled AC power outlet.

The circuits are trivially easy to make if you’re an electronics hardware type. I’m doing PCB layouts of both circuits for myself and can make those available for you to use if there’s any interest.

Let’s do the warnings up front. These circuits necessarily involve connections to the AC power line because that’s how they work – they convert the AC power line coming on to an isolated contact inside a controlled machine. There’s no way around that. The circuits are easy but any time you even get physical access to AC power line wiring, there is danger of electrocution, electrical damage to property and starting fires. Living in an electrically-powered world like we do the rule for everyone has to be¬† that if you don’t know how to do it safely, don’t start. So read that safety warning in the center of the picture and follow it. It’s YOUR responsibility.

I apologize for the legalism, but in a lawsuit-mad society, I cannot be personally responsible for your actions. You have been warned. Act accordingly.

With that bit of nastiness out of the way, let’s get on with doing what we started out to do.

Circuit 1 is the no-transformer version. The amount of electrical power needed in faking a button activation is tiny. So the first version uses a capacitor (C1) to limit the amount of current the AC power line provides to the rest of the circuit, and it does it in a way that is very power-efficient. The rest of the circuit takes this little trickle of current, makes just enough DC voltage out of it to run the integrated circuit. The integrated circuit is a three pin device that does only one thing – it senses whether its power supply is big enough, and when it is, it pulls its output pin high.¬† High in this case is about 5Vdc, and that’s enough for the resistor R3 to allow enough current through the LED of the optoisolator ISO1.

This causes the isolated phototransistor on the isolator’s output to be fully saturated and look like closed contacts to the ATX power supply. If you connect the isolators output across the “power switch” contacts in the contacts of the motherboard you’re using, then it will look just like you’ve pressed the front panel switch to the motherboard, and this will activate the power supply on/off just like you pressing the button.

Two remote button pusher circuits

Button Pusher #2 does exactly the same thing on the computer side. What’s different is the way that the power supply for the output is generated. Instead of using resistors and capacitors to make the voltage needed by the reset IC, it uses a transformer to both isolate the AC power line and to change the power line voltage down to the few volts needed for the reset IC. This voltage is rectified, filtered, and then regulated by a tiny three terminal regulator. This circuit is bigger and heavier than Button Pusher #1, but not much. I was surprised to find that there are now tiny transformers which will do this job, and that they are not expensive.

I’m not going to clutter this posting up with parts lists and so on, because I suspect that very few people will ever actually build one of these. If you want to really build one of these, instead of just reading about it, leave a comment so we can get in touch one on one.

Remote Power Management, Part Two: A Button Pusher

Posted in Power Considerations, Remote Management by rgkeen on April 27, 2010

Having found a LAN-based controller that can turn on/off up to four AC power outlets, my quest for remote control resolved into needing a way to have an AC power line on/off be the equivalent of pressing the “reset” and “power on/off” buttons on an ATX computer.¬† Remember, the issue to be avoided is using only an AC power line for power off/on.¬† That’s necessary in some cases, but an extreme measure, and to be avoided if possible.

Here are a few ways to accomplish the task that came to mind:

  • Power a solenoid from a switched outlet, and have the solenoid plunger actually press the buttons. I discarded this as too clumsy and requiring too much mechanical fabrication. Likewise, other electro-mechanical approaches fall into the same category.
  • Power a relay with a 120Vac coil from the switched outlet, and connect the relay contacts in parallel with the front-panel switches. The power line outlet coming on causes the relay contacts to act like a front panel switch without being one. This one has some merit, and is conceptually easier to handle for non-electronically adept people.¬† It amounts to using a pre-manufactured solenoid plus extra switch contacts to do the first approach. I discarded it only when I thought about possible contact bounce and relay life issues.
  • Skip the fancy stuff and just use WOL. Nice enough, but my motherboard does not do this. A Whack-On-LAN card would work, probably. But I didn’t want to spend the time getting a possibly flakey card that is not otherwise used in the box to work reliably.
  • Power a small electronic circuit from the switched outlet, and have this circuit cause an electronic closure in parallel with the actual front-panel switches. Obviously this is inspired by the Whack-On-LAN device.¬† This is what I finally chose to do.

Having decided to design an AC-line to switch closure circuit, I sketched out my objectives. The final circuit has to short the contacts when the power line is on, release it when the power line is off. It has to do this cleanly, no funny wandering around between open and closed, and no bouncing open/closed/open/closed to confuse the power supply. Oh, yeah – it has to not electrocute me, burn down the house, or destroy the computer it’s controlling.

Update Notice:
The following is the kind of thinking that would go into designing a right-down-to-the-bare-metal device. I had a belated brainstorm that makes this kind of moot, as well as safer. If you’re going to do something like this read Part Three: A Safer Button Pusher first. Using a pre-existing wall wart relieves many safety concerns, as well as being quicker to actually get. Most Goodwill and used-computer stores have bins of wall warts at rock bottom prices.

This last is where all the complication is buried in this approach. Building a circuit which goes from the AC power line into a metal box that people can touch is a clearly dangerous situation. But there are rules for how to do this. Safety testing agencies have been specifying this for decades.  There are three Big Rules. They are:

  1. Provide physical spacing and testable 2500Vac to 5000Vac isolation between the AC line and metal that a person can touch.
  2. Make the circuit not start fires if any part fails in the worst possible way.
  3. Don’t generate any isolated voltage that is dangerous to people or property like the AC power line is.

We’re in luck on the first one. The electronics industry makes optical isolators which have an internal multi-kV isolation. They provide an output which is usable for the remote-shorted-switch operation, and are safety lab tested for the Big Rules. And they’re cheap. A suitable device is less than US$1.00 in almost all cases. I found one for US$0.30.¬† This is too good not to use.

The second task is to turn the isolator on and off. The input side of the isolators are universally an LED (Light Emitting Diode), which needs to be driven with about 5-10ma of current to make the output go on/shorted, and no current to make it turn off. I pondered designing a transistor circuit to do this, then an IC circuit (National Semi LM10), and finally found an IC which does the whole thing cleanly. This is a three pin IC which generates a clean, snap action on/off voltage in response to its power supply. The Microchip Technology TC54 comes in a TO-92 package and costs US$0.50. Perfect.

What remains is to make some kind of circuit to power up the TC54 when the AC power outlet goes on, and de-power it when the socket goes off.¬† The complication is that the socket provides 120Vac, and the TC54 wants a voltage in the range of 1 to 10 Vdc. I found this article by Robert Kollman and Brian King in back issues of EDN. It does the necessary job perfectly, and uses only light, small electronic components.¬† I’ve adapted it for the button pusher.

It is also possible to use a step-down transformer to make an isolated DC supply to power the TC54. In fact, in many ways this is a simpler approach, as the transformer itself encapsulates all the safety isolation considerations.

I’ll post both methods in an upcoming addition to this entry.

Remote Power Management for Home Servers

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on April 26, 2010

Shortly after I got my first Opensolaris server up and running, I discovered that I never wanted to see it again – servers should be usable but invisible as well as silent!

Back to google. A long series of searches led me to some conclusions and solutions.

  1. LAN managed power cycling is a Very Good Thing if you are not able to walk over to your server to restart/reboot it. It will be needed. My neighbor makes his living doing system admin for a commercial data center with hundreds of systems in racks. He has to drive in to work in the middle of the night often enough even with industrial-strength remote management setups as it is.
  2. Hard power-off is potentially life-threatening to your data. Soft power off is much kinder. Hard power off should be reserved for situations where the system is no longer listening to your instructions and there is no option. Again, my neighbor who lives and breathes remote support confirms that he’ll do pretty much anything to avoid hard power-off on a critical server. His opinion is that hard power-off forces you to consider server CPR as a real possibility.
  3. LAN managed power cycling is moderately expensive. And I’m – well, I prefer the term “economically prudent” to cheap.

I can hear you wondering why I didn’t just set up Wake On LAN (WOL). That would actually be a great thing to do except for two issues.¬† First, not all motherboards support WOL. My first motherboard, the M3A78-CM which is otherwise great, does not. Worse yet, it does not clearly tell you that in the manual. I had to send a tech support request to the maker to find that out unambiguously.

Then again, WOL is for when things are all working OK. I can always tell my machine to shut down if it’s listening to me. But if it’s too busy contemplating its own navel and re-counting its toes to be bothered to listen, there ought to be some way to make it listen. I think I found it.

A short series of conclusions came up in the searches.

There is a sweet spot for LAN managed AC power. It’s the IP Power 9258 series. This is a small box with an ethernet port, four managed AC outlets, and an internal web server to flip the managed outlets’ states. Here’s a copy of the manual.

The current manufacturing state of the unit seems to be in question. A google search turns up many places which say out of stock and no longer available. However, there are also many places which have them, including a seller on ebay for $80. It’s in the mail to me as I write this, so we’ll know soon.

I found another technique called “Whack-on-LAN”, which uses a modified ethernet NIC which has a WOL connector to do what amounts to¬† pressing the reset button over the LAN. The article describing this is here. In essence, you convert a cheap LAN card to a remote-controlled button pusher and hit the reset button. The only issues with this trick is that the LAN card lives inside the server and is dependent on the server power supply and BIOS/operating system making it happy so it can do the LAN remote control trick.

So there’s some interesting stuff. In my mind, it plays out like this:

  • I want to be able to tell a non-WOL system to fire up. A LAN-controlled button pusher would be great for that.
  • Any system that’s working properly will shut down on its own.
  • Any system that’s not working properly would have to be hard-reset.
  • If the hard-reset doesn’t work, it needs hard power-off cycled as a last resort.
  • There are advantages to an integrated power management system even for a board that supports WOL.

Here’s the solution I came up with. Use the 9258 to control AC power. I really only need one of the controlled AC outlets to run the system. That leaves me three other outlets which can be viewed as a form of LAN controlled logic outputs – that is, AC power = logical true, no AC power = logical false. What is needed to get all of what I need for deluxe remote power management for my server is to make those AC power logic levels into isolated contact closures, which then can interface directly into the normal circuitry of the industry-standard ATX form factor personal computer stackup.

Did I mention that I spent many years doing analog and digital circuit design? The two obvious answers are either to use an AC-activated relay with a power-line rated primary run from a controlled outlet, or an electronic circuit which senses AC power and does an isolated “contact closure” similar to the transistor switch use in “Whack-on-LAN”. There are many optoisolators available with 4kV isolation from the LED driver on the primary side to a transistor output on the secondary side which will safely do this job.

I’ve done a quick design of both of these approaches, and I’ll document those here in a future post.