R.G. Keen's Blog

Remote Power Management, Part Four: A Safer Button Pusher

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on May 16, 2010

I had one of those “Doh!!”, forehead slapping moments.

In the first Button-Pusher article I talked quite a bit about safety issues and the need to know what you’re doing with AC power line stuff – or wind up electrocuted. That was all straightforward and true.
But I think of these things like an engineer, not a hacker. The hacker side of me finally pulled through.
You don’t need to do all the AC safety mess if you have some widget that encapsulates all that for you. And we do, and it’s cheap and easy. It’s an AC to DC power adapter, widely know as a “wall wart”.

Just do the same circuitry after the rectifier diodes to do the internal stuff. But make the input to the computer side of this be a barrel style DC power connector. Plug an AC to DC wall wart in the controlled-AC power source, and plug the wall wart power connector into the barrel-plug socket on the computer. All done. No need to mess with AC power line. That’s all enclosed inside the wall wart. All that comes out is DC, and the dangerous bits have been pre-designed and pre-tested for safety and other things so that you never have to endanger yourself and others – at least any more than you do now. 😎

Advertisements

Curses – Opensolaris doesn’t support Wake On Lan!

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on May 12, 2010

Well, mostly not. After some more time on google, I find that to make WOL work, you need

  • A WOL-capable motherboard
  • Setting the BIOS to enable WOL, or in newer boards “S5 on PME” wakeup events
  • A WOL-capable LAN adapter; some add-in cards do this, some don’t; and some internal LAN adapters support it, some don’t
  • A network device driver that supports WOL

It’s that last which Opensolaris fails on. The drivers may or may not have WOL inside them, depending on their provenance. And the WOL that may happen to be there may or may not work, or -worse!- may not work correctly, or reliably.

As I read when doing this research, the guys who form the Opensolaris cognoscenti do not think much of WOL. The general idea is that it’s at least useless, possibly worse. They vastly prefer wake from sleep/standby. Obviously this is a very broad generalization, and it’s what I inferred from a couple of hours of reading. Individuals may take issue with it.

I’m after remote wakeup and shutdown, as I mentioned in the post on remotely managed AC power. I had stopped work on the managed AC power button pushers while I investigated WOL. I’m now glad I did the AC button pusher work. It looks like I’ll need it.

I also found another suitable class of wakeup/shutdown devices. These go under the heading of embedded ethernet I/O devices. Some of them contain a minimal web server and can be run by a remote web browser. All of them have their own LAN port, and let you twiddle the I/Os remotely.

The application is obvious. These things can do the low-power button pushing and could conceivably have an AC power relay added. I’ll do more digging and see if this gets me closer to the poor-man’s “Lights-Out Adapter”, which is what the industry terms remote management setups.

Staggered Disk Spin-up for Home Servers

Posted in Opensolaris Server, Power Considerations by rgkeen on May 2, 2010

You can spin up your disks one at a time if your hardware allows it. Making your hardware allow it can be tricky, but worthwhile. You can use a much smaller power supply.

Low power is a virtue in a home server, just like it is for a commercial site server. When a disk spins up, it temporarily uses as much as five to ten times as much power as it does when it’s operating normally.

That presents a problem when you turn on the power supply. I have eight disks in my server at the moment, six 750GB 3.5″ devices and two 40GB 2.5″ devices.  The main array is the six 3.5″ devices, which get most of their power from +12V. At start up they can pull as much as 3A (36W) of +12V power each. That’s 18A of +12V – 216W.  After spinning up, the disks settle to 6W each, or three amps of +12V for the whole array.

The rest of the system uses +12V to power the processor, memory, and other things, so booting the entire system takes even more. When the system gets past this power-on-peak, it settles to about 100W total. There is a big pulse of power to get the mess started.

No problem, just buy a big power supply right? Actually, that is a problem. Power supplies waste power making the power you use. The power supply efficiency is the ratio of the power you get out of it to the power you put into it from the AC power line. It’s only recently that you could get power supplies which stated, and lately, certify their efficiency. The best are rated for “80-plus” which stands for “better than 80 percent”. These especially-good power supplies eat 100W of AC power line power for every 80W of power they deliver to your computer.

What that doesn’t tell you is that the losses are usually fixed, or at least do not decrease much with lower power out.  A 100W-rated 80+ power supply “eats” 125W from the AC power line to make 100W of  output power, the 25W being the efficiency penalty. A 500W rated 80+ power supply wastes five times as much, 125W, to make 500W, and it probably wastes 100W all the time, even if the load is only getting 100W. And the 80-plus units are the good guys. Lower power supply efficiencies only make the wasted power worse.

There is therefore a big plus in terms of less wasted electricity and heat generation if you can buy a smaller power supply for your server. But we already figured out that we have to buy a power supply to withstand the power-on peak in power needs. The power supply probably loafs after that.

Staggered spin-up is one trick to getting the peak power down, letting you buy a smaller, less wasteful power supply. If you could power up only one disk at a time, you could start your system with a much smaller power supply. In my system, the measured idle power from the AC line is about 105W. Each disk pulls 6W at idle, but 36W at spinup.

Theoretically, I could get the system to start up successfully with a power supply rated at 105W plus 30W for the peak for one disk drive to spin up if i could ensure that only one disk would start at a time.  A 150W power supply would be fine for that! Not only cheaper, more economical to run and “greener”.

To get to staggered spin up, you have to enable both hardware and software to make it happen. Opensolaris is great for that, because you can enable staggered spinup. But how about the hardware?

It turns out that some SATA drives (notably from WD) have a jumper on the drive for enabling and disabling staggered spin up. Others (notably Seagate) use pin 11 of the SATA power connector for enabling staggered spinup. Disabling staggered spin up happens when pin 11 is grounded, enabling happens when it’s open. It took me a few days to find that out.

When I could not make staggered spin up work, I dug in and found out that my power supply cable connectors have pin 11 internally connected to pins 10 and 12 – they are hard wired to make Seagate drives not do staggered spin up! The only way to make Seagates do staggered spin up was to commit surgery on either the disks or the power connectors to isolate pin 11. Then they spun up only when the software told them to.

As usual, some of the more useful bits are buried quite deeply. But if you look, you can sift out some power and money savings in your server.

Remote Power Management, Part Three: Button Pusher Circuits

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on April 28, 2010

I’ve designed a couple of circuits which implement the button pusher circuits for remotely pressing the power-on and/or reset buttons on an ATX style machine from a LAN controlled AC power outlet.

The circuits are trivially easy to make if you’re an electronics hardware type. I’m doing PCB layouts of both circuits for myself and can make those available for you to use if there’s any interest.

Let’s do the warnings up front. These circuits necessarily involve connections to the AC power line because that’s how they work – they convert the AC power line coming on to an isolated contact inside a controlled machine. There’s no way around that. The circuits are easy but any time you even get physical access to AC power line wiring, there is danger of electrocution, electrical damage to property and starting fires. Living in an electrically-powered world like we do the rule for everyone has to be  that if you don’t know how to do it safely, don’t start. So read that safety warning in the center of the picture and follow it. It’s YOUR responsibility.

I apologize for the legalism, but in a lawsuit-mad society, I cannot be personally responsible for your actions. You have been warned. Act accordingly.

With that bit of nastiness out of the way, let’s get on with doing what we started out to do.

Circuit 1 is the no-transformer version. The amount of electrical power needed in faking a button activation is tiny. So the first version uses a capacitor (C1) to limit the amount of current the AC power line provides to the rest of the circuit, and it does it in a way that is very power-efficient. The rest of the circuit takes this little trickle of current, makes just enough DC voltage out of it to run the integrated circuit. The integrated circuit is a three pin device that does only one thing – it senses whether its power supply is big enough, and when it is, it pulls its output pin high.  High in this case is about 5Vdc, and that’s enough for the resistor R3 to allow enough current through the LED of the optoisolator ISO1.

This causes the isolated phototransistor on the isolator’s output to be fully saturated and look like closed contacts to the ATX power supply. If you connect the isolators output across the “power switch” contacts in the contacts of the motherboard you’re using, then it will look just like you’ve pressed the front panel switch to the motherboard, and this will activate the power supply on/off just like you pressing the button.

Two remote button pusher circuits

Button Pusher #2 does exactly the same thing on the computer side. What’s different is the way that the power supply for the output is generated. Instead of using resistors and capacitors to make the voltage needed by the reset IC, it uses a transformer to both isolate the AC power line and to change the power line voltage down to the few volts needed for the reset IC. This voltage is rectified, filtered, and then regulated by a tiny three terminal regulator. This circuit is bigger and heavier than Button Pusher #1, but not much. I was surprised to find that there are now tiny transformers which will do this job, and that they are not expensive.

I’m not going to clutter this posting up with parts lists and so on, because I suspect that very few people will ever actually build one of these. If you want to really build one of these, instead of just reading about it, leave a comment so we can get in touch one on one.

Remote Power Management, Part Two: A Button Pusher

Posted in Power Considerations, Remote Management by rgkeen on April 27, 2010

Having found a LAN-based controller that can turn on/off up to four AC power outlets, my quest for remote control resolved into needing a way to have an AC power line on/off be the equivalent of pressing the “reset” and “power on/off” buttons on an ATX computer.  Remember, the issue to be avoided is using only an AC power line for power off/on.  That’s necessary in some cases, but an extreme measure, and to be avoided if possible.

Here are a few ways to accomplish the task that came to mind:

  • Power a solenoid from a switched outlet, and have the solenoid plunger actually press the buttons. I discarded this as too clumsy and requiring too much mechanical fabrication. Likewise, other electro-mechanical approaches fall into the same category.
  • Power a relay with a 120Vac coil from the switched outlet, and connect the relay contacts in parallel with the front-panel switches. The power line outlet coming on causes the relay contacts to act like a front panel switch without being one. This one has some merit, and is conceptually easier to handle for non-electronically adept people.  It amounts to using a pre-manufactured solenoid plus extra switch contacts to do the first approach. I discarded it only when I thought about possible contact bounce and relay life issues.
  • Skip the fancy stuff and just use WOL. Nice enough, but my motherboard does not do this. A Whack-On-LAN card would work, probably. But I didn’t want to spend the time getting a possibly flakey card that is not otherwise used in the box to work reliably.
  • Power a small electronic circuit from the switched outlet, and have this circuit cause an electronic closure in parallel with the actual front-panel switches. Obviously this is inspired by the Whack-On-LAN device.  This is what I finally chose to do.

Having decided to design an AC-line to switch closure circuit, I sketched out my objectives. The final circuit has to short the contacts when the power line is on, release it when the power line is off. It has to do this cleanly, no funny wandering around between open and closed, and no bouncing open/closed/open/closed to confuse the power supply. Oh, yeah – it has to not electrocute me, burn down the house, or destroy the computer it’s controlling.

Update Notice:
The following is the kind of thinking that would go into designing a right-down-to-the-bare-metal device. I had a belated brainstorm that makes this kind of moot, as well as safer. If you’re going to do something like this read Part Three: A Safer Button Pusher first. Using a pre-existing wall wart relieves many safety concerns, as well as being quicker to actually get. Most Goodwill and used-computer stores have bins of wall warts at rock bottom prices.

This last is where all the complication is buried in this approach. Building a circuit which goes from the AC power line into a metal box that people can touch is a clearly dangerous situation. But there are rules for how to do this. Safety testing agencies have been specifying this for decades.  There are three Big Rules. They are:

  1. Provide physical spacing and testable 2500Vac to 5000Vac isolation between the AC line and metal that a person can touch.
  2. Make the circuit not start fires if any part fails in the worst possible way.
  3. Don’t generate any isolated voltage that is dangerous to people or property like the AC power line is.

We’re in luck on the first one. The electronics industry makes optical isolators which have an internal multi-kV isolation. They provide an output which is usable for the remote-shorted-switch operation, and are safety lab tested for the Big Rules. And they’re cheap. A suitable device is less than US$1.00 in almost all cases. I found one for US$0.30.  This is too good not to use.

The second task is to turn the isolator on and off. The input side of the isolators are universally an LED (Light Emitting Diode), which needs to be driven with about 5-10ma of current to make the output go on/shorted, and no current to make it turn off. I pondered designing a transistor circuit to do this, then an IC circuit (National Semi LM10), and finally found an IC which does the whole thing cleanly. This is a three pin IC which generates a clean, snap action on/off voltage in response to its power supply. The Microchip Technology TC54 comes in a TO-92 package and costs US$0.50. Perfect.

What remains is to make some kind of circuit to power up the TC54 when the AC power outlet goes on, and de-power it when the socket goes off.  The complication is that the socket provides 120Vac, and the TC54 wants a voltage in the range of 1 to 10 Vdc. I found this article by Robert Kollman and Brian King in back issues of EDN. It does the necessary job perfectly, and uses only light, small electronic components.  I’ve adapted it for the button pusher.

It is also possible to use a step-down transformer to make an isolated DC supply to power the TC54. In fact, in many ways this is a simpler approach, as the transformer itself encapsulates all the safety isolation considerations.

I’ll post both methods in an upcoming addition to this entry.

Remote Power Management for Home Servers

Posted in Opensolaris Server, Power Considerations, Remote Management by rgkeen on April 26, 2010

Shortly after I got my first Opensolaris server up and running, I discovered that I never wanted to see it again – servers should be usable but invisible as well as silent!

Back to google. A long series of searches led me to some conclusions and solutions.

  1. LAN managed power cycling is a Very Good Thing if you are not able to walk over to your server to restart/reboot it. It will be needed. My neighbor makes his living doing system admin for a commercial data center with hundreds of systems in racks. He has to drive in to work in the middle of the night often enough even with industrial-strength remote management setups as it is.
  2. Hard power-off is potentially life-threatening to your data. Soft power off is much kinder. Hard power off should be reserved for situations where the system is no longer listening to your instructions and there is no option. Again, my neighbor who lives and breathes remote support confirms that he’ll do pretty much anything to avoid hard power-off on a critical server. His opinion is that hard power-off forces you to consider server CPR as a real possibility.
  3. LAN managed power cycling is moderately expensive. And I’m – well, I prefer the term “economically prudent” to cheap.

I can hear you wondering why I didn’t just set up Wake On LAN (WOL). That would actually be a great thing to do except for two issues.  First, not all motherboards support WOL. My first motherboard, the M3A78-CM which is otherwise great, does not. Worse yet, it does not clearly tell you that in the manual. I had to send a tech support request to the maker to find that out unambiguously.

Then again, WOL is for when things are all working OK. I can always tell my machine to shut down if it’s listening to me. But if it’s too busy contemplating its own navel and re-counting its toes to be bothered to listen, there ought to be some way to make it listen. I think I found it.

A short series of conclusions came up in the searches.

There is a sweet spot for LAN managed AC power. It’s the IP Power 9258 series. This is a small box with an ethernet port, four managed AC outlets, and an internal web server to flip the managed outlets’ states. Here’s a copy of the manual.

The current manufacturing state of the unit seems to be in question. A google search turns up many places which say out of stock and no longer available. However, there are also many places which have them, including a seller on ebay for $80. It’s in the mail to me as I write this, so we’ll know soon.

I found another technique called “Whack-on-LAN”, which uses a modified ethernet NIC which has a WOL connector to do what amounts to  pressing the reset button over the LAN. The article describing this is here. In essence, you convert a cheap LAN card to a remote-controlled button pusher and hit the reset button. The only issues with this trick is that the LAN card lives inside the server and is dependent on the server power supply and BIOS/operating system making it happy so it can do the LAN remote control trick.

So there’s some interesting stuff. In my mind, it plays out like this:

  • I want to be able to tell a non-WOL system to fire up. A LAN-controlled button pusher would be great for that.
  • Any system that’s working properly will shut down on its own.
  • Any system that’s not working properly would have to be hard-reset.
  • If the hard-reset doesn’t work, it needs hard power-off cycled as a last resort.
  • There are advantages to an integrated power management system even for a board that supports WOL.

Here’s the solution I came up with. Use the 9258 to control AC power. I really only need one of the controlled AC outlets to run the system. That leaves me three other outlets which can be viewed as a form of LAN controlled logic outputs – that is, AC power = logical true, no AC power = logical false. What is needed to get all of what I need for deluxe remote power management for my server is to make those AC power logic levels into isolated contact closures, which then can interface directly into the normal circuitry of the industry-standard ATX form factor personal computer stackup.

Did I mention that I spent many years doing analog and digital circuit design? The two obvious answers are either to use an AC-activated relay with a power-line rated primary run from a controlled outlet, or an electronic circuit which senses AC power and does an isolated “contact closure” similar to the transistor switch use in “Whack-on-LAN”. There are many optoisolators available with 4kV isolation from the LED driver on the primary side to a transistor output on the secondary side which will safely do this job.

I’ve done a quick design of both of these approaches, and I’ll document those here in a future post.