How is power usage calculated in a data center?

tdean

just got an email for a job, heres one line from the email..

Datacenter practices: cable routing, calculating power usage, and other standard practices in datacenters

is calculating power usage a standard practice for net admins? i've never heard of anyone really doing it....

anyone here know how its done?

laidbackfreak

I've done it in the past and in my current role. Doing it from scratch can be a pain if you have no documentation to work from. As it means doing an inventory which is no bad thing.

Started off making a list of the kit already installed and then trawled the vendor sites to get the figures needed some are much better than others. As well as probes\monitoring software etc all good fun

I've never looked around for any software I put together a spread sheet years ago that I use which works for my needs.

tdean

but what is the process? elec bill - power supplies? i know it isnt that simple, but where would i even start?

Kaminsky

There's usually on site electricians that measure the actual usage. This is different from what the clients get billed as they are typically only billed for 50% or so of the potential maximum of their devices based on the fact that it is unlikely their devices will be operating at maximum power specified in the data sheet for that device due to load on the device.

Is a whole big calculation thing based on whatever factors. But for the actual usage, that's an electricians job. Actual power of the one I work at is roughly the same as that of a city with 250,000 people apparantly. Data centres go through power in a very big way. Green wise, probably pull the same amount of carbon as a small airline.


Whats the job and I can give you a rough idea of duties

jeanathan

That is true the IT job usually stops at the power plug in the wall and moves responsibility to the electrician. I guess you mean power consumption optimization though.

I haven't done this at the data center scale, but these are some resources I've used on a small scale.

MEASURING ENERGY USAGE
UPS Selector Sizing Applications

bellhead

Contact the power company, they will likely have an engineer, "a real EE engineer" who does nothing but this for a living.

Forsaken_GA

Yeah, alot of factors go into it. Cooling is another issue. You can't just take into consideration how much power each device is going to use, you have to consider how many BTU's it puts out, because that also raises the amount of cooling you need to do, which in turn also increases your power requirements.

Our senior network engineer is an EE as well. When we were building out our new facility, it was amusing as hell to watch our facilities manager (who had final say on how things were built out), the senior engineer, and the contractor's mechanical engineer all sit around and debate how we were going to handle the cooling to the network cage

Fugazi1000

Power can be measured in a number of ways. VA ratings are often used to calculate maximum load on specific devices (UPS/PDUs etc).

Power consumption is usually measured in Kwh or Kilowatt Hours. In the UK the cost of powering a 1000 Watt device for 1 hour is typically around £0.06p. This is a commercial negotiation with the electricity supplier or co-lo provider.

A Server may have a 'plate rating' of 1400 Watts but the actual draw in use, is 1000 Watts. There are various ways to measure this from PDU based sensors to the manual use of a Clamp style Ammeter.. So assume our example server is powered 24x7 for 1 year. It will have a power cost of £525.60. Then count the number of servers and the costs rocket! It's now that you need to factor in the cooling. BTUs (British Thermal Units) are the norm here. A good rule of thumb is to account for double the power into the server to account for AirCon/CRAC units etc. This does vary a lot however, as this is about efficiency in the room. Hot Aisles/Cold Aisles, External ambients etc. Cooling kit in Texas is not the same as cooling it in Iceland!

When you do the maths on all the kit in a typical DC - the waste is often huge!!! IT is generally very inefficient and it's where the business case to switch to virtualisation can often be made. Get more efficient Power Supplies and the losses to heat go down dramatically. So less to cool. I seem to recall a statistic that there is over a 98% loss from the power station to the CPU. Green indeed!

bellhead

I talked to one of the business marketing reps "I work at a major power company" and they confirmed what I thought. During the build of a facility an engineer and marketing rep would work with the architect and other project managers to help get the facility going. This is because the power company will need to know how much power you are going to require so the outside plant infrastructure can be put into place. I have been around long enough to know that they want to insure everything is done correctly the first time.

DevilWAH

each of our racks has a power distribution unit, this has a network interface and you can use SNMP, or the compinies own soft ware to read current/history usage for each connected device. So its in in line power monitor.

The server room ups also has a large UPS?Generate back up so this can give energy stat's on the over all usage of each of the circuites in the room.

each rack is fed by two separate circuits so if one fails there is redundancy, and there are separate circutites for lights and Air con. So we can if needed get a over all view of the total use of energy in the server room.

And we are just on the edge of it being built in to servers, Cisco already had "energy Wise" which allows you to take energy reading directly of the network equipment through SNMP, and this year Dell and I think some of the other manafacturs are adding energy monitoring directly on to the server (the PSU and mother board have energy monitoring), so you will be able to see energy usage from with in the OS. With the "green policies" becoming such big business, this on board monitoring is going to only get bigger and more wide spread.

For a data centre power monitoring will be built in to the infrastructure of the electricity system. so generally total usage for a building is not hard to find out, but many compinies now monitor the usage at the rack level. and it is not incommon these days with new instulations to see systems in place that can monitor single servers, and even subsystems on those server, all fed back to a central management console.

If you dont have it built in there are plenty of "clamps" around that measure current passing through a cable, (OWL | Wireless Electricity Monitor Designed To Save You Money & The Environment - Home is a home version), but these are generally used for a one of mesurment and on small instulations,

But yes seeing how a large data centre may be spending in excess of £500,000, on energy, in fact for large data centers it may be the largest chunk of costs, It become vitaly important to know how much is being used and how it can be reduced.


Just one other point, before we got the rack monitoring we worked out usage from looking at the servers we have, the dell site and APC UPS site, to try to get an idea of what they where pulling. In every case the figure we came up with was incorrect compared to the real figures we have now, in most cases we over estimated by about 30 to 40% So much depends on how the server is being used that really the figures you get from server specs will give you a worst case figure where the server is maxed out.

Forsaken_GA

bellhead wrote: »

I talked to one of the business marketing reps "I work at a major power company" and they confirmed what I thought. During the build of a facility an engineer and marketing rep would work with the architect and other project managers to help get the facility going. This is because the power company will need to know how much power you are going to require so the outside plant infrastructure can be put into place. I have been around long enough to know that they want to insure everything is done correctly the first time.

What's *really* fun is when you find out that your backup generator doesn't failover properly. That kind of fun ruins a large number of peoples weekends.

DevilWAH

Forsaken_GA wrote: »

What's *really* fun is when you find out that your backup generator doesn't failover properly. That kind of fun ruins a large number of peoples weekends.

lol oh yer,

Talking about power failerers we noticed a strange thing the other day. Although Servers have ups's to cover any spikes caused durign fail over to generators, Sone of the buildings on site with users in don't. So generally a power fail causes PC's to reboot.

But we noticed PC's with an external Power supply, such as dell small form factors, the powerpack retains enough to cover power out of about a second. where as bigger PC with internal power supplies reboot...