I came across something interesting a few weeks back regarding the x79 board and the use of PCI-E 3 for DC that I thought might be of interest to other crunchers here. I initially had a pair of 680s setup via a Rampage IV board for running the Einstein GPU app which uses a lot of PCI-E bandwidth. One day, I installed a PCI-E 2.0 card in slot 2a which is an x8 slot on the Rampage IV board. As soon as I did that, I began to see a performance improvement on the 680 cards which were running at 8 GT/s via the x16 slots. The improvement was approximately 16.5%. I had already seen a 15% increase from running the cards at 8 GT/s over 5 GT/s prior to installing the PCI-E 2.0 card. This came as a surprise to me as the PCI-E 2.0 card is sitting completely idle and should not have any effect over the other cards. The only thing I can think of is that by having this card installed, something happens with the PCI-E controller that results in a further increase in bandwidth. Interestingly enough, if I install a PCI-E 1.1 card or another PCI-E 3.0 card, I do not get the performance boost.
At the time, I thought perhaps this was something related to the board, BIOS, or CPU. This week I decided to start testing out the 680s via my EVGA x79 FTW board to see what would happen if I added the PCI-E 2.0 card on this board. Bare in mind that this board has a different CPU from the other board. I tested with and without the PCI-E 2.0 card installed and sure enough, I see a 16.5% increase in performance again with the PCI-E 2.0 card installed and the 680 running at 8 GT/s. I had the 2.0 card installed in both slot 3 (x4) and slot 4 (x16) and saw the increase in both cases. So it appears that this interesting side effect is not tied to a specific board, CPU, or slot. To date I do not know for sure why this happens. I need to see if I can find an application that can measure the bandwidth to see if the performance boost can be attributed to change in bandwidth.
As a result of all this, I ended up picking up a Zotac 9800 GT card for $40 on E-bay that is short enough to not impede airflow for the 680 sitting above it. The card is just short enough to make it so the fan on the card above is completely open and is sitting by idle. Like with my other 2.0 card, I get the boost on the PCI-E 3 cards. I thought I would share my latest evil experiment for anyone else here running x79 and PCI-E 3 cards for DC. I am not sure what other projects out there can take advantage of the PCI-E bandwidth but if you happen to have a similar setup, it might be worth a try to see if you can get an boost in GPU performance. Now I am thinking of picking up an Ivy Bridge processor at some point to see how its PCI-E 3 controller handles compared to my other CPUs. Unfortunately that CPU is limited to 16-lanes which will limit my card options. Sorry for the wall of text.
<message edited by linuxrouter on Wednesday, July 04, 2012 8:39 PM>