Well, the results of my PCI Express 2.0 versus 3.0 on my 4-way SLI GTX 680 FW900 Surround setup are in. The results are so incredible I had to start the tests over from scratch and run them multiple times for confirmation! :eek:
Test setup: 
3960X @ 5.0 GHz (temp slow speed)Asus Rampage IV Extreme with PCI-E slots running 16x/8x/8x/8x(4) EVGA GTX 680's running 1191MHz core, 3402 MHz MemorynVidia Driver 301.10 with PCI-E 3.0 registry adjustment turned on and off for each applicable tesGPU-Z  0.6.0


After PCI-E settings changed, confirmed with GPU-Z:



All settings in nVidia control panel, in-game and in benchmark, EVGA precision are all UNTOUCHED between benchmark runs. The only setting adjusted is the PCI-E 2.0 to 3.0 and back and forth for confirmation (Reboots obviously for registry edit).



I kid you not, that is how much PCI-E 2.0 running at 16x/8x/8x/8x versus PCI-E 3.0 bottlenecks BF3 and Heaven 2.5 at these resolutions. I attribute this to the massive bandwidth being transferred over the PCI-E bus. We are talking 4-way SLI at up to 10-megapixels in alternate frame rendering. Entire frames at high FPS are being swapped and PCI-E 2.0 falls on it's face. 
The interesting part was that while running PCI-E 2.0, the GPU utilization dropped way down as would be typically seen if you are "CPU" limited. In this instance I am not CPU limited, nor GPU limited. We are really at a point now that you can be PCI-E limited unless you go PCI-E 3.0 8x (16x PCI-E 2.0) or faster on all GPU's in the system.  GPU utilization dropped down into the ~50% range due to PCI-E 2.0 choking them to death. As soon as I enabled PCI-E 3.0, the GPU utilization skyrocketed to 95+% on all cores. I was going to run more benchmarks and games but the results are such blow-outs it seems pretty pointless to do any more. It may interest some of those out there running these new PCI-E 3.0 GPU's in which they think they are CPU limited (below 95% GPU utilization) yet might have PCI-E bandwidth issues. 
Down to the nitty gritty; if you run a single GPU, yes; a single 16x speed PCI-E 2.0 slot will be fine. When you start to run multiple GPU's and/or run these new cards at 8x speed, especially in Surround/Eyefinity, make sure to get PCI-E 3.0. ;)
 
More on my system: http://www.overclock.net/...-computer-edition-2012

Wow! I'm starting to wonder if my 2-way SLI is bottlenecked by having x8 2.0 for each card then... it looks possible (2560x1600). My GPU utilization is high (90%+) at almost all times in games though. Regardless, looking like a day-1 buy for Ivy Bridge for me to enable PCI-E 3.0 on my Z68 Gen3 board!
Is there any chance I could trouble you for a 2-way SLI run at 3600x1920 (PCI-E 2.0 vs 3.0)? Pretty please?  That's most comparable to my resolution... though still far greater (4mp vs 7mp).

Is there any ETA from nvidia on fixing their driver issues with PCIE3.0 on X79 yet ?

I am considering getting a 3rd 680, but after looking at your results, I won't bother until they enable PCIE3.0 on X79.

callsignvega


Well, the results of my PCI Express 2.0 versus 3.0 on my 4-way SLI GTX 680 FW900 Surround setup are in. The results are so incredible I had to start the tests over from scratch and run them multiple times for confirmation! :eek:

 
@EVGA MODS: PETITION! BLUE RIBBON FOR CALLSIGNVEGA PLEASE!
 
Fantastic work, respect.

 
Would be nice If You could insert the percentage gains over PCIe 2.0. Roughly I would say 30% to 50% higher.
I remember quite well some guys here, defending tooth and nail that PCIe 3.0 is not even worth a try. Contrarily I used driver 300.83 from the beginning where PCIe 3.0 is enabled.
 

 
PCIe 3.0: Enable PCIe 3.0 on GTX680/X79 through Japanese registry hack?
http://www.evga.com/forums/tm.aspx?m=1509861
 

Firstly I know stuff all about SLI.
 
Looking at your GPU-Z pics, is it right that the first pic with PCI-E 2.0 shows that the nVidia SLI is disabled?

Raistlin Majere


Firstly I know stuff all about SLI.

Looking at your GPU-Z pics, is it right that the first pic with PCI-E 2.0 shows that the nVidia SLI is disabled?

 
Yes, That doesn't look proper..  Is it possible that SLI wasnt enabled when doing your benchmarks?

Nice post on the PCI 3.0
 
 
 
 
 
MSI Big Bang Xpower
I7950 @ 4.0GHz
24g G.Skill Ripjaw
EVGA GTX 580 SLI @ 850/1700/2004
OCZ Agility 3 120GB Raid 0
Raptor 10000K 300GB Raid 0
Thermaltake Speedo Case
UltraX 1200W PSU
H70 Water Cooled Push Pull

How ironic,  just Thursday a buddy of mine sent me all these links on pics of your system and some videos on youtube.  He was going on and on about those CRTs.  
 
totally awesome man, those speakers that look like crystals are pretty sweet too, i went to look at your specs to findout more about them but they werent listed :(   
 
on topic tho, thats pretty jawdropping results there, certainly wouldnt have ever guessed that big of gains.   
 
BR for sure!!!! 
 
 
 

Im still new at this, how you post that pic at the bottom of your message. The Sig.

omg There is so different between pcie 2 vs pcie 3

Very nice ! Good info ,wonder how much difference there is on a single card? 

AB Was Here Per Request... 

@OP... These results are a tad off others I have seen. Would you be kind enough to find the correct CPU-z pic with SLI enabled or run the PCI-E 2.0 again/correctly?

Max Silencio

callsignvega


Well, the results of my PCI Express 2.0 versus 3.0 on my 4-way SLI GTX 680 FW900 Surround setup are in. The results are so incredible I had to start the tests over from scratch and run them multiple times for confirmation! :eek:

@EVGA MODS: PETITION! BLUE RIBBON FOR CALLSIGNVEGA PLEASE!

Fantastic work, respect.


Would be nice If You could insert the percentage gains over PCIe 2.0. Roughly I would say 30% to 50% higher.
I remember quite well some guys here, defending tooth and nail that PCIe 3.0 is not even worth a try. Contrarily I used driver 300.83 from the beginning where PCIe 3.0 is enabled.

PCIe 3.0: Enable PCIe 3.0 on GTX680/X79 through Japanese registry hack?
http://www.evga.com/forums/tm.aspx?m=1509861

I did this registry hack, but it only made my first card in SLi run at PCI-e 3.0 and my other card was still running at the PCI-e 2.0 speed. I sure would like to know how the op got all 4 cards to run in PCI-e 3.0 speeds.

Afterburner


@OP... These results are a tad off others I have seen. Would you be kind enough to find the correct CPU-z pic with SLI enabled or run the PCI-E 2.0 again/correctly?

 
That particular GPU-Z screenshot was taken when I was re-arranging the monitors. In order to do that, you must disable SLI and then re-enable SLI/Surround mode and it will pick up my CRT monitors again. With three CRT's it is impossible to run these Surround resolutions without being in SLI, and due to each GTX 680 having a maximum 1 analog connection to the RAMDAC of each card, you need a minimum of three GTX 680s.
 
I can understand some people being skeptical, heck I was at first results! I knew there would be a nice difference, but now THIS much. Remember I am pushing cards and resolutions at FPS's that have never been tested comparing PCI-E 2.0 vs 3.0 before. The only other 2.0 vs 3.0 benchmarks I have seen are either single card and/or single monitor. 

goldentiger


Wow! I'm starting to wonder if my 2-way SLI is bottlenecked by having x8 2.0 for each card then... it looks possible (2560x1600). My GPU utilization is high (90%+) at almost all times in games though. Regardless, looking like a day-1 buy for Ivy Bridge for me to enable PCI-E 3.0 on my Z68 Gen3 board!
Is there any chance I could trouble you for a 2-way SLI run at 3600x1920 (PCI-E 2.0 vs 3.0)? Pretty please?  That's most comparable to my resolution... though still far greater (4mp vs 7mp).

 
Sorry my Surround setup needs a minimum of three GTX 680's to run 

sourtimes9


Is there any ETA from nvidia on fixing their driver issues with PCIE3.0 on X79 yet ?

I am considering getting a 3rd 680, but after looking at your results, I won't bother until they enable PCIE3.0 on X79.

 
Please show your support here:
 
http://forums.nvidia.com/index.php?showtopic=226569&st=0&gopid=1392983&#entry1392983
 
ManuelG from nVidia is reading the thread. The squeaky wheel gets the oil!

Max Silencio

callsignvega


Well, the results of my PCI Express 2.0 versus 3.0 on my 4-way SLI GTX 680 FW900 Surround setup are in. The results are so incredible I had to start the tests over from scratch and run them multiple times for confirmation! :eek:

@EVGA MODS: PETITION! BLUE RIBBON FOR CALLSIGNVEGA PLEASE!

Fantastic work, respect.


Would be nice If You could insert the percentage gains over PCIe 2.0. Roughly I would say 30% to 50% higher.
I remember quite well some guys here, defending tooth and nail that PCIe 3.0 is not even worth a try. Contrarily I used driver 300.83 from the beginning where PCIe 3.0 is enabled.

PCIe 3.0: Enable PCIe 3.0 on GTX680/X79 through Japanese registry hack?
http://www.evga.com/forums/tm.aspx?m=1509861

Yup, that's the registry mod!

I did this registry hack, but it only made my first card in SLi run at PCI-e 3.0 and my other card was still running at the PCI-e 2.0 speed. I sure would like to know how the op got all 4 cards to run in PCI-e 3.0 speeds.

 
I ran into the same problem but I found the solution! When all cards are in non-Surround mode, they each have their own VID ID. You  must apply the registry edit to the 0000 folder of EACH GTX 680 entry. Then once that is done, go into SLI/Surround mode and then apply to the SLI/Surround VID ID 0000 folder. It combines them and treats them as one in SLI! Reboot and now you are good. ;)

/me wonders why he even bothers sticking with his dreaded Phenom II/790GX setup when results are really this clear. 
 
Might as well build a whole new rig around the GTX 680 if this is the case.  Seems like just doubling the RAM and the GPU upgrade won't be enough alone it sounds like to me.

callsignvega

I did this registry hack, but it only made my first card in SLi run at PCI-e 3.0 and my other card was still running at the PCI-e 2.0 speed. I sure would like to know how the op got all 4 cards to run in PCI-e 3.0 speeds.

I ran into the same problem but I found the solution! When all cards are in non-Surround mode, they each have their own VID ID. You  must apply the registry edit to the 0000 folder of EACH GTX 680 entry. Then once that is done, go into SLI/Surround mode and then apply to the SLI/Surround VID ID 0000 folder. It combines them and treats them as one in SLI! Reboot and now you are good. ;)

I'm not following this part as I don't have a SLI/Surround folder in my registry.
 
"Then once that is done, go into SLI/Surround mode and then apply to the SLI/Surround VID ID 0000 folder. It combines them and treats them as one in SLI! Reboot and now you are good. ;)"
 
Can you be more specific on this please?
 
EDIT: Also I did not instal any of the 3d surround drivers since I don't use surround.

OK I got it done now. I have to apply the pci-e 3.0 reg hack while the cards are out of SLi mode. This gives me folders to enter the RMPcieLinkSpeed for each card. Once that is done I turn SLi back on and both cards are running at the pci-e 3.0 speed.
 
Thanks.

kram36


OK I got it done now. I have to apply the pci-e 3.0 reg hack while the cards are out of SLi mode. This gives me folders to inter the RMPcieLinkSpeed for each card. Once that is done I turn SLi back on and both cards are running at the pci-e 3.0 speed.

Thanks.

 
What cards/monitors are you running? Now that you can switch between PCI-E 3.0 and 2.0 at will, maybe do some benchmark comparisons?

CallsignVega,
I spent my morning looking over your 2012 build 
http://www.overclock.net/t/1220962/vegas-heavyweight-display-and-computer-edition-2012
Amazing "outside the box" thinking, sweet computer pron!
Your willingness to share is second to none, hats off to you my friend. 
Great work, keep it up, thanks for sharing!!!!

callsignvega

kram36


OK I got it done now. I have to apply the pci-e 3.0 reg hack while the cards are out of SLi mode. This gives me folders to inter the RMPcieLinkSpeed for each card. Once that is done I turn SLi back on and both cards are running at the pci-e 3.0 speed.

Thanks.

What cards/monitors are you running? Now that you can switch between PCI-E 3.0 and 2.0 at will, maybe do some benchmark comparisons?

I'll give 3DMark 11 a go and see if it improves any there.
 
EDIT: Looks like I got a bump with this. Exact same system settings, just the pci-e 3.0 hack done.
 
Before http://3dmark.com/3dm11/3136887 Score 19,206
 

 
After http://3dmark.com/3dm11/3141931 Score 19,417
 

 
 

Could this disparity be more a result of the other cards running at x8 rather than x16.?  I am just wondering because I wanted to keep my e762 x16/x16/x16/x16 for at least another year or two but I do want to run 3 - 4 GK110's for a 3D Vision surround setup.  I would seriously reconsider upgrading to Ivy if it will be like buying a ferrari to only drive it 20MPH.

LOL... great... more to tinker with. :)
 
Folks, y'all gotta stop giving me reasons to not write papers and watch the Masters.

englehmj


Could this disparity be more a result of the other cards running at x8 rather than x16.?  I am just wondering because I wanted to keep my e762 x16/x16/x16/x16 for at least another year or two but I do want to run 3 - 4 GK110's for a 3D Vision surround setup.  I would seriously reconsider upgrading to Ivy if it will be like buying a ferrari to only drive it 20MPH.

Hm, that is debatable. Since that board runs 2x NF200's it should help out with multi-GPU/Surround gaming versus native X58 PCI-E lane allocation. It would be interesting to see if the increased PCI-E bandwidth of 3.0 to the CPU would be a boon over the NF200's communication to the CPU via PCI-E 2.0. I would lean toward PCI-E 3.0 being overall better but NF200 quad 16x being much better than without the NF200's. 

Hitting VRAM limit and PCI-E 2.0 vs 3.0 battle video tests. Make sure to watch in 480P instead of lower (I forgot to set the camera back to 720P and I don't feel like recording everything all over again lol). Sorry about the video quality but it is still view-able. 
http://www.youtube.com/wa...jBhzZ8mtcWAKgkFdz3LBo=
http://www.youtube.com/wa...LsTk3pZxTxt4tbGt5H9Yk=

The only reason there is such a large increase is because the resolutions are so high that he ran out of VRAM, so then PCI-E traffic increased dramatically to access textures in the main memory. If you had 4GB cards I guarantee that you'd only see around a 1% improvement like you saw with 3dm11.

Khenglish


The only reason there is such a large increase is because the resolutions are so high that he ran out of VRAM, so then PCI-E traffic increased dramatically to access textures in the main memory. If you had 4GB cards I guarantee that you'd only see around a 1% improvement like you saw with 3dm11.

 
Wow, this is wrong on so many levels. Did you even watch the videos?

Does it really even matter what the % increase is as long as we get an increase with it?

kram36


Does it really even matter what the % increase is as long as we get an increase with it?

 
Ya, even if on a single GPU with a single monitor if you only get 2-5% performance increase. Isn't that a good thing? Silly people treat PCI-E 3.0 likes it's something bad. PCI-E 3.0 may only provide small gains in the single GPU/single monitor realm but can create phenomenal gains in the multi-GPU/multi-display world. There is basically no reason not to use PCI-E 3.0 whenever possible.