Modest hardware updates
CPU
Find out if you can do reasonably-priced upgrades. If you are able to buy a CPU, which uses less power, but provides the same performance – do so. New CPUs pop up all the time. One good preemptive action against fast changing technologies is to purchase a good board. Spend more on it and it will pay off with better CPU support. One example is the Phenom II x4. Mid-range priced AM3 motherboards supported all x4 processors. Currently the same motherboards support x6 CPUs. You might spend a bit more on the x6 CPU, but second hand prices for x4 are still very good, so here are another 2 core out of nothing.
HDD/SSD
Despite what you may think, Solid State Drives are not a good upgrade when it comes down to power consumption. The price tag on those is pretty high too. SSD drives consume just as much power any modern HDD. If you think of switching solely for power saving then don’t. If you are into better performance, while not being bothered with the smaller size, then go ahead.
PSU
More efficient Power Supply Units is probably the most overlooked topic in energy efficiency. Higher efficiency PSU means that more power goes into your machine and less power dissipates in heat. There is a pretty good comparison over at SilentPCReview.com. Spend more on an 80 PLUS Power Supply. They come at different specifications, but the 80 PLUS Bronze seems reasonably priced.
GPU & Other Hardware
Amateur projects and simpler installations can’t utilize the GPU as a vital source of computation power. Removing any video hardware is highly advisable. If you don’t feel like removing the graphics card once you are able establish SSH connection, then consider purchasing a motherboard with integrated GPU. If you have any hardware which you do not use, take it out (sound cards, DVD drives, etc.).
Cooling
Fanless designs in such cases are rate, but if you can go without the CPU fan in favor of a better heat sink and good chassis ventilation, then do so. It won’t save much, but efficiency freaks will have a blast out of the several watts saved. There are many advantages to fanless designs in terms of system stability, noise levels, amortization and maintenance.
Consolidate Servers
A lot of CPUs and motherboards support virtualization. Why not consolidate multiple servers into one machine. If your have a bunch of servers with non significant load, then put them on a single machine.
Track Down Unwanted Services and Applications
A machine consuming less power usually means a machine producing less heat. This prolongs the life of your components and with fewer components in a server there is less to go wrong. You might also want to get a Kill-A-Watt meter to measure your progress.