Purpose:

• Raise team awareness of the power of Big CPU folding
  • These systems are capable of posting 200K - 800K PPD.
  • Provide some organization to the wealth of good information, guides, and my lessons learned.
• My personal goals were to push myself into the Top 20 Producers for Folding@EVGA, maybe motivate my teammates to step up as well, force myself to learn about -BigAdv, Server hardware, and learn to spell Ubuntu.
• I'm certainly not an expert in this area. What work for me, may not work for you or your hardware. This represents about a month of nightly information gathering on the various source. I hope this save you alot of research time.
• Please send me a private message for any correction, updates, rewording, rewrites, and any additional information you think we should include. (It going to need it, because it was written way too late at night)

Table of Contents

Hardware and Cost
Hardware Decisions and Discussions

- CPUs
- Motherboards
- Memory
- Heat sink / CPU Coolers
- Thermal Paste
- Drives
- Power Supply Unit (PSU)
- Case / Mounting

System Pictures (Everyone love hardware)
BIOS Settings

------------------------------------------ Under Construction Still 
Software

- Ubuntu Installation
- Folding Configuration and Utilities
- Hardware Monitor Tools
- Useful Ubuntu Stuff

References
 

Change Log:

6/15/12 - Initial Post
7/14/12 - Added BIOS, PSU update and minor edits
8/8/12 - Added Power Advisory on CPU/EPS adapters
9/1/12 - Added Referrence to Improving Opteron 6200 performance
4/25/13 - Update Vendors CPU Pricing

Hardware and Cost 

AMD 4P Build Cost

$1,360 - 4x AMD Opteron 6166HE 1.8Ghz CPUs (Used, w/ 1yr warranty $340 per)   
$663 - Supermicro H8QGi+-F AMD Quad G34 Socket Motherboard (New) 
$240 - 4x G.SKILL Ripjaws X Series 8GB (4 x 2GB) DDR3 1333 (Cas 7 7-7-7-21) Desktop Memory Model F3-10666CL7Q-8GBXH (New $60 per) 
$120 - EVGA Classified SR-2 100-PS-1200-GR 1200W (New $120 on Close Out Special)
$48 - 4x Cooler Master Hyper 212 Plus (Refurb $12 per) 
$15 - Seagate 40GB Sata (From the local used PC store)
$0 - LG DVD Drive (From the local used PC store, removed after install)
$10 -  Zalman’s Super Thermal Grease (STG) 
$20 - Misc Hardware for board & CPU Cooler mounting
------------
$ 2,476 - My out of pocket cost

At stock speed this rig is producing (~243K PPD w/ P8101, ~276K PPD w/ P6901, ~380K PPD w/ P6903/4) with an electrical draw of 460W.
 
AMD 2P Build Cost

$700 - 2x AMD Opteron 6176SE 2.3Ghz CPUs (Used Ebay $350 per)    
$250 - Supermicro H8DGU-F AMD Dual G34 Socket Motherboard (Used Ebay)  
$120 - 4x Crucial Ballistix 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1600 (Cas 8 8-8-8-24) Desktop Memory Model BLT2KIT2G3D1608DT1TX0 (New $30 per)

$0 - OCZ ModXStream Pro 700W Modular High Performance PSU (had sitting Idle)  
$26 - 2x Cooler Master Hyper 212 Plus (Refurb $12 per)  
$15 - Seagate 40GB Sata (From the local used PC store) 
$10 -  Zalman’s Super Thermal Grease (STG)  
$15 - Misc Hardware for board & CPU Cooler mounting 
------------ 
$ 1,136 - My out of pocket cost 

At stock speed this rig is producing (~125K PPD w/ P8101) with an electrical draw of 340W.
 
Power and PPD Comparison to others:
 - Stock i7 920 w/ GTX 580@920, GTX 580@900, GTX 260@735 produced ~75K PPD with an electrical draw of 830W
 - Stock i7 920 w/ GTX 580@920, GTX 260@735 produced ~40K PPD with  an electrical draw of 560W
 - Stock i7 920 w/ 3x GTS 450@950 - 900 produced ~40K PPD with  an electrical draw of 560W
 - Stock Intel Quad Q6600 w/ GTX 460@900 produced ~20K PPD with an electrical draw of 313W
 - Stock Intel Dual E4400 w/ GTS 8800 produced ~10K PPD with an electrical draw of 235W
 - Mitsubishi 65in DLP TV has an electrical draw of 208W

 

Hardware Decisions and Discussions - CPUs

Intel vs AMD. I was certainly bias to Intel and believed you only folder on just Intel and NVIDIA GPUs. I definitely thought I was looking at a Dual Xeon setup with an SR-2 motherboard. Xeon CPUs are changing over from the older LGA-1366 socket to the new LGA-2011 socket, the older can't measure up and I didn't want to put money into SMP 16 core system at would struggle to meet the new -BigAdv deadlines. The LGA-2011 CPU are very expensive ie $2,000 per processor and no overclocking motherboard at this point (June 2012).
 
I looked at AMD 6200 Series is the latest and greatest with up to 16 cores (June 2012) & 6100 Series is the last generation model with 8 and 12 cores (You'll also seen them as Magny-Cours or MC)
The several debates about the two series, I believe the general thoughts are 6200 are newer, faster, have more cores and more expensive BUT don't fold as well as the older 6100 series core for core. The 6100s are starting to be life cycled out of the big high end servers, thus makes for a larger supply on the used markets like Ebay. More details AMD 6100 vs 6200 Debate reference.
*EDIT 9/1/12* QuickZ's thread on Improve Opteron 6200's performance (and PPD) by using TurionPowerControl is new information since I looked at this decision.
 
Here is how I thought about it. I could get an AMD Opteron 6166HE with 12 cores  for $340. I couldn't even get a Intel (because I think Intel in the PC world) Hexa / Heca / 6 cores CPU for that price. Intel i7 (970, 980, 990) all costing over $400 on eBay market. I'm sure we could debate of HyperThreading compared to AMD native cores, but the deciding fact was you can't run dual or multi socket motherboards with the Intel i7 (970, 980, 990) only Xeons.
 
In the AMD 6100 series (12 core) looks like

6180SE @ 2.5Ghz -140w
6176SE @ 2.3Ghz -140w
6176 @ 2.3Ghz -115w
6174 @ 2.2Ghz -115w
6172 @ 2.1Ghz -115w
6168 @ 1.9Ghz -115w
6166HE @ 1.8Ghz -85w
6176HE @ 1.7Ghz -85w

In the AMD 6100 series (8 core) looks like

6140 @ 2.6Ghz -115w
6138 @ 2.5Ghz -115w
6136 @ 2.4Ghz -115w
6134 @ 2.2Ghz -115w
6132HE @ 2.1Ghz -85w
6128HE @ 2.0Ghz -85w
6124HE @ 1.8Ghz -85w

SE - Top of the Line
HE - Energy Efficient
EE - Engineering Samples (These are test and evaluation units, not sold to the public, but yet on eBay hmm)
 
I selected the AMD Opteron 6166HE with 12 cores for $340 (May 2012)
 
Here is a great reference to see what -BigAdv numbers are being posted by these and other big CPU / server hardware rigs. https://docs.google.com/s...XWVZ2S2xvejBDcHc#gid=0
 
Where to Buy: (~$340)
Ebay.com 
IT Creations, Sales Manager (Gal Lis - gal@itcreations.com) and telling him your with "Stanford's Folding Project".

12 core:
 
AMD 6172, 97 in stock, $225 each
AMD 6176, 85 in stock, $375 each
AMD 6168, 66 in stock, $435 each
AMD 6176 SE, 38 in stock, $425 each
AMD 6180 SE, 37 in stock, $550 each
AMD 6174, 31 in stock, $575 each
AMD 6164 HE, 29 in stock, $225 each
AMD 6234, 25 in stock, $325 each
AMD 6238, 22 in stock, $345 each
AMD 6344, 18 in stock, $400 each
 
16 core:
 
AMD 6276, 150 in stock, $500 each
AMD 6284 SE, 46 in stock, $1,025 each
AMD 6274, 41 in stock, $575 each
AMD 6272, 58 in stock, $435 each
AMD 6282 SE, 21 in stock, $775 each
AMD 6376, 18 in stock, $650 each

IT Creations offers a 1yr warranty and Free Shipping with these price. Please remember prices do fluctuate depending on the market and there inventory, but these should be good for the near future. (Updated April 2013)

Hardware Decisions and Discussions - Motherboards

In the 4 Processors (4P) / Quad Socket / MultiProcessor (MP) Motherboard, there not a lot of choices the two main ones are TYAN S8812 (S8812WGM3NR) and SuperMicro H8QGi / H8QGL / H8QG6 series.
 
I selected the Supermicro because the folding team [H]orde / [H]ardOCP created a BIOS upgrade that could allow you 
to overclock the CPUs, I saw posted claims 30% overclocks or better. I picked SM H8QGi+-F because of the price and CPU socket placement, I felt it would be easy to cool than the cheaper SM H8QGL. I did not consider the H8QG6 series due to the added cost of SAS controller that I didn't need. Additional information on the different board series.
 
SM H8QGi+-F Benefits
 
- Socket placement was good for cooling.
- Embedded graphics card (no need to buy one, also add a video card could hamper cooler by blocking the CPU cooler).
- Motherboard works with just one cpu and would allow you to continue to upgrade by adding addition cpus over time. This was a big advantage over buying a 2P (2 Processor / Dual Socket) motherboard at 65% of a the cost of a Quad socket board.
- Although I don't recommend this, but you can theoretically use different / mismatched CPUs on the board. It will just drop the overall CPU speed down to the lowest common denominator.
- Biggest benefit is the ability to get great support for the folks at the [H]ardForum.
 
 
 
Where to Buy: (~$660)
WiredZone.com
Provantage.com 
Ebay.com 
 

Hardware Decisions and Discussions - Memory 

The AMD 6100 series CPUs has an integrated quad-channel DDR3 SDRAM Memory controller. This translates for optimal performance each CPU needs 4 separate memory sticks. You need to check your motherboard manual, but Error-Correcting Code (ECC) memory is common place on server motherboard. ECC memory is more expensive and slower than the traditional non-ECC memory common to desktop computing. In this case both the Tyan and Supermicro can be used with non-ECC memory. You also have to look at of the memory voltage, once again defer to your motherboard manual, you need the 1.5v memory for the Supermicro. You'll also want memory with XMP profile.

If you think, you would like to overclock in the future, you're going to need fast memory. If you don't overclock you could save some dollars here.

Recommendation are: Detailed Reference
G.SKILL Ripjaws X Series 8GB (4 x 2GB) DDR3 1333 (Cas 7 7-7-7-21) Desktop Memory Model F3-10666CL7Q-8GBXH 

For high end overclock you need DDR3 1600 Cas 7 with XMP profiles and 1.5v Remember to stay with 1.5v memory, watch out for the high-end memory it tends to jump up to 1.65v
 
Special notes unless you are doing other things than folding on the system, 1GB sticks are all you need. Fold runs well in under 6GB for Dual and Quad sockets systems. If you're interested in BOINC Crunching in the future the recommendation is to use 2GB sticks. <Need a Cruncher to send me a PM with a good refer>

Where to Buy: ($60)
G.SKILL Ripjaws X Series 8GB (4 x 2GB) DDR3 1333 (Cas 7 7-7-7-21) Desktop Memory Model F3-10666CL7Q-8GBXH  

Hardware Decisions and Discussions - Heat sink / CPU Coolers

Background: Server boards are design more or less for Data Centers, where cool air is rapidly circular thought the case, with the smallest height and foot print as possible. Rack mounted Data Center devices are generally avoided in the home environment for several reasons. Most folks don't have a server rack at home, they can take up alot of space, but the main reason I believe is the noise levels of the fans. These Data Center devices fans are design to pull high volumes of cool air thru the device and are very loud and typically not user adjustable. How Loud? Take your vacuum cleaner in the closet and turn it on and close the door. If you and significant others are OK with the sound volume, ignore this warning.

You basically have 3 options, with various tradeoffs of Cost / Time / and Cooling Effectiveness.

1. Use standard server heat sinks. Typically what is used in Data Center deployed servers and normally consist of a copper block heat sinks (~$50 per CPU).
2. Upgrade to server heat sinks with 70 - 80mm fans in a low-profile setup (~$100 per CPU)
3. Custom Modify the Motherboard or PC CPU Cooler to fit (large range of cost, but starting ~$20).

 

- Option A - Modify PC CPU Cooler fan mount. This will require change a ex. 1366 / 1155 mount from a 4 post setup to match the G34 socket of 2 post. Detailed reference here Kendrak's 2P and 4P system guide
- Option B - Custom create an adapter for a 4 post CPU Cooler to the 2 post mounting on the G34 socket. Detail reference Musky's CM Hyper 212+ - Alternative G34 Mounting Method

Both Options A & B requires some work to use standard #6-32 threaded screws for the 2 post mounting on the G34 socket. I've seen basically two options to do this:

- Option 1 - Modify the motherboard (and Yes I understand it cost over $650 dollars) with a #6-32 Tap tool to cutting the threads thru the motherboard post inserts, thus enable you to use standard #6-32 x 1.5in bolts to be screwed in from the bottom side of the motherboard.

- Option 2 - Take #6-32 x 2in bolt an cut off the heads (creating a threaded rod) an insert the by hand screwing it into the motherboard post inserts.

I chose PC CPU Coolers route and selected Refurbished Cooler Master Hyper 212+, that where recommended for overclock and good value ($12 per CPU). I went with Option B and Option 1, where I modified the motherboard tapping the thread thru the post and used #6-32 x 1.5in bolt screwed in from the bottom of the board. I followed Musky's CM Hyper 212+ - Alternative G34 Mounting Method. The only exception, I didn't use the springs on 2 post to the motherboard, but just a #6 lock washer an #6-32 standard nut. Due to the space constraints you can only turn the nuts about a 3rd of a turn with a wench. I didn't use the nylon threaded nuts here for that reason.

Special note - These CPU Coolers are tall and you'll need to take that in to consideration if you want to use a case. Some folks have voice concerns over the CPU Cooler's combine weight on the motherboard in a vertical / upright position, I personally would think it fine, but again wouldn't recommend using the springs on the mounting to the motherboard post.

Where to Buy: (~$12)
Cooler Master Store - Hyper 212 Plus Universal Direct Contact Heat-Pipe CPU Cooler - Refurbished  Cooler Master Store 
Geeks.com - Cooler Master Hyper 212 Plus Aluminum Heat Sink & 4.72" Fan w/Copper Heatpipes & 4-Pin Connector for Intel & AMD CPUs
 

Hardware Decisions and Discussions - Thermal Paste

I use Zalman’s Super Thermal Grease (STG) that is brushed on like a nail polish. I'm done with the tube / paste and using a credit card to smooth it, I never seem to get it just right, it's either too thick or too thin. Just brush it on the heat sink and CPU and it has been very consistent.

Special note - AMD G34 / Operton CPU are significantly larger than the standard PC Intel chips, the heat sink doesn't cover the whole ship, so there no need to cover the whole chip. I was able to do the install of all 4 CPU with a single bottle, but your mileage may vary.

Where to Buy: (~$10)
Zalman’s Super Thermal Grease (STG) 

Hardware Decisions and Discussions - Drives

I picked up a small used hard drive, because it what I know and cheap $15 at the local used computer store. Theoretically you could run the whole system off something like a 8GB USB thumb drive, but I had enough challenges already, maybe a project for a rainy day. I use a DVD drive for the installation of Ubuntu only then disconnected it. I'm thinking a USB external DVD drive could be useful for future system builds.

Special note - You could play with the IGMI management interface (which is a BIOS supported KVM over IP) and virtually attach to a DVD drive across the network for the installation. Detailed reference Penn7's post Supermicro IPMIview Review – Remote Server Monitoring and Management and KVM-over-IP. I wasn't aware of this at the time of my installation, but probably would have stuck with my physical attached DVD.

Where to Buy: (~$15)
Any store near you.
Ebay.com 

great write up

Hardware Decisions and Discussions - Power Suppy Unit (PSU)

I had an OCZ ModXStream Pro 700W on hand, so didn't put much thought into this decision. At stock CPU speeds and no dedicated graphics card I'm pulling 440W - 460W from the wall. Overclocking 10% pushed to 480W of consumption. I've seen post with folks using AMD 6180 CPUs and large overclocks pulling 800W - 850W. If you buying a PSU I'd recommend one that is EPS12V 2.92 version, I believe all 850W and above units come standard with 2 8-pin CPU / EPS connectors. Alot of smaller PSU come with 1 8-pin & 1 4-pin ATX12V / EPS. The 4-pin (inserted in the motherboard 8-pin connector) provides enough power to boot the system and do the installation, but you're not going to be able to fold under that setup. Other options would be to use an adapter converts 2 4-pin molex connectors to feed the 8-pin CPU / EPS connector NOT RECOMMENDED or a 4 Pin Male to 8 Pin Female EPS Power Cable Adapter or a PCI-E (6+2) to 8-pin EPS. I used the adaptor (2x 4-pin Molex to 8-pin CPU / EPS connecter) on my system and it seems fine for stock speeds, but I've had alot of difficult with overclock and I'm looking into this as a possible problem. EDIT We are definitely seeing reports that the CPU / EPS sockets are capable of pulling more power than the molex wires can handle. For more details read Patriot's post on 4p Power advisory.
 
EDIT After upgrade the PSU to a EVGA 1200W (that has 2 CPU / EPS connectors, I've had no problems with the overclocking. My 6166HE default clock is 1.8 GHz now running 2.3 GHz. Detail reference Bowlinra's post AMD 4P Burning and Prep for OC? 
 
Power Draw References:
SuperMicro H8QGi+-f with 4x 6166HE (85W CPUs) stock is 1.8Ghz
Stock 200@465W
OC 226 @ 465W
OC 236 @ 530W <- Change VCore to 1.05
OC 246 @ 542W
OC 251 @ 543W
 
SuperMicro H8DGU-F with 2x 6176SE (140W CPUs) stock is 2.3Ghz
Stock 200 @ 340W

Special note - Consider going with Gold or better rated PSU, this could reduce the electric cost by 10 - 20% for me at a relative low rate of .10 per KWh that's an $40 - $80 annual savings.

Where to Buy: 
Name Brand 850w PSU EPS12V 2.92 version
Molex to 8-pin EPS connector (~$7)
4 Pin Male to 8 Pin Female EPS Power Cable Adapter ($.54)
PCI-E (6+2) to 8-pin EPS (~$9)

Hardware Decisions and Discussions - Case / Mounting

Case or run Naked (w/o a case). That is the question. I still have mixed feelings on this one. I guess my visions of granduir lead me to think, I might have more than one of these setups, needing some form of a wooden homemade rack. (EDIT 7/20/2012: I did build a homemade rack guide on a separate thread here Home Rack Garage Build - Beating the Folding Heat of Summer). I also had concerns with the weight of the CPU Coolers on a vertical mounted motherboard. I'm not so concern with this anymore (see note in the Heat sink / CPU Coolers section). I also thought the heat displacement would be better horizontally placed. 
If you wanted a case the #1 issue is the size of the motherboard is just huge (16.48" x 13" (41.9cm x 33.0cm)) it NOT going to physically fit in most cases, virtually all case will have to be modified somehow. The Supermicro H8QGi motherboards have 13 screws to properly attach to the case, I would guess 6 of those screw in standoffs will have to be drilled and tapped. Even large case like the popular full case Cooler Master HAF 932 will need move fans to the outside of the case for the side, top and back. Details reference ChelseaOilman's New 4P Build 

Still Under Construction - Check Back for Updates

For addition Pics look in the Pictures section 
Musky's Mounting PC's on wood instead of cases 
Bowlinra's Home Rack Garage Build - Beating the Folding Heat of Summer
Spotswood's Tech Tray

System Pictures (Everyone love hardware)

R-Type post on 2P and 4P system guide
chaandan's Muskymod
Linden's G34 4P Supermicro Pics
tuigi69's 4p systems - Cases and Case Mods 
Norcalsteve's The Thing 
harlam357's Project Quadteron 
Bobc36's Something good happened! (Scroll down for pics)
Barbarossa's 4P & 2P Systems
NerdGZ's 2P AMD
Kazenagi's My Waterlogged Folding 4Pie Goodness (WIP)
Texinga's Mint-4P photos of my finished case
Tanner2's ARG..... not another one of hese posts!!
David12857's New member of the EVGA 2P family.
 
Please send me a PM, if you have additional good pictures post to include.

Message Place Holder / Still Under Construction

BIOS Settings
 
For OverClocking BIOS on the Supermicro boards, the reference is Tear's post Supermicro H8QGi/6 and H8QGL Next Generation OC BIOS 

These recommendations apply only to non-OC BIOSes (e.g. stock SM or Tyan). Reference to Musky's post Multiple Processor G34 Checklist

BIOS settings to check
- PowerNow set to Disabled
- NUMA/SRAT set to Enabled
- Node Interleaving set to Disabled
- Bank/channel interleaving set to Auto or Enabled
- Tyan boards - Ungang DCTs set to Always

Message Place Holder / Still Under Construction

Software
- Ubuntu Installation
- Folding Configuration and Utilities
- Hardware Monitor Tools
- Useful Ubuntu Stuff

 

Message Place Holder / Still Under Construction

Software 
- Ubuntu Installation - I use Musky's Ubuntu Desktop Installation Guide for F@H

   I know there several available, please PM me any links, if you have found other useful as well. 

Message Place Holder / Still Under Construction

Software 
- Folding Configuration and Utilities 

 

Message Place Holder / Still Under Construction

Software 
- Hardware Monitor Tools 

 

well for a standered case you can use the xigamatek elyisum http://www.newegg.com/Product/Product.aspx?Item=N82E16811815013
if your running more than one server you could use a cabinet and use either a piece of would or plexiglass as a mobo base like this one http://www.newegg.com/Product/Product.aspx?Item=N82E16816133032

Message Place Holder / Still Under Construction

Software 
- Useful Ubuntu Stuff 
 

Message Place Holder / Still Under Construction

References

Then you want to read these: 
Musky's multiple processor g34 checklist 
Tear's Overclock BIOS 
Musky's Ubuntu guide
Musky's 212 mounting guide
Jmtyra's ** Team 32: 4P Folding Mega Thread ** 

Sweeeet Build/Guide Bowlinra!!! 
 
Definitely will be an excellent reference for our Team and thank-you for putting it together for us! 

excellent write up Bowlinra
here are some pics of mine
watercooled in a Lian  Li PCA77 (EOL and has been replaced by the PCA 77F which doesn´t allow for the PSU to go in the top)

 
caseless with Noctua NH U9DO A3

 
and a dual setup with the Noctua NH U12DO A3

 
on the Noctuas I used their NH T1, don´t remember what i used on the waterblocks

you can add the mod i use on the 2P :


 
and the solution implemented on the Urlauber 2P

Attachment(s)

Attachments are not available: Download requirements not met

I thought they made server racks with multiple server trays in them, so you could run anywhere from 2 to 16 servers?
 
Do they make a smaller version of a server rack that can maybe hold 4 or so server trays?

In my area space is limited. So If I were to do a 4p build it would have to go in a case and be verticle. So the water cooler option would probably be the best for me. But then I'd need a new case to fit such a beast. We need to compile a list of any cases that will fit these monster boards with little to no modification if possible.
 
Very nice job so far bowlinra.
 
@NerdGZ, what kind of themps do the stock Phenom coolers give you?
Also @ Barbarossa, what water blocks are you using on your board?
And one final side note, Operon 6128 8 core CPU's are currently going for $165 or so on fleabay. For those of us that are wanting to do this on a faily tight budget. And yes I realize that budget and 4p machine don't go together so well.

@zophar 40ºCelsius with a 20mm 800rpm splitted over the heatsinks memory and with a 80mm fan on the bare metal behind hdd connectors.
This solution is noisy aprox 3600 rpm fans from the 2 Phenom II X4/x& are spinning.
Note to add to the post these Opteron CPU dies at 70ºC and over 54 -55ºC the CPU autoprotects them and does a shutdown/reboot.
I have seen one at 50ºC but after removing the cooler fans and do a clean i was at this temperature.
 

This deffinately needs to moved to the Help and Guides section. Great work.

NerdGZ

you can add the mod i use on the 2P : and the solution implemented on the Urlauber 2P

 
NerdGZ, I'll be glad to add it, Could you add some word explaining what you did, why and how for example.. It's not clear how and what you did from the pics, maybe because I'm not familiar with the stock version of the Urlauber 2P.

zophar

We need to compile a list of any cases that will fit these monster boards with little to no modification if possible. 

I have might have some more information, but most of it posted, just not organized well for cases information. I've mounted my to a board and will be adding the write up after some vacation.
 

zophar
 
Also @ Barbarossa, what water blocks are you using on your board?

I haven't looked into water setups at all, maybe Barbarossa  or someone else can work up a "How To" and some cost numbers.

NerdGZ

Note to add to the post these Opteron CPU dies at 70ºC and over 54 -55ºC the CPU autoprotects them and does a shutdown/reboot.

Could you post a reference link?
I certainly haven't heard this with the Opteron 6100 series, I understand they start down clocking at 70ºC. 
 
Also what kinda of PPD are you getting out of the 2P setup?