What to Expect with the EVGA Killer Xeno Pro - Updated

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (Arrakiv)

Thu, 25 Feb 2010 10:07:00 GMT

I honestly have no idea why JPR did what they did, or their reasons behind it, in general - we weren't involved in the testing process. Actually, offloading would be a neat test to see done in a report like this, though.

I can only assume that they didn't test file transfer speeds because they were only looking at gaming very specifically (note the white paper's title). Gamers do use their networks for other things though, which is accurate, and I think a totally valid point. You're also right that the Xeno at the time had pretty fail transfer speeds, as we had to sacrifice throughput for the better latency. That said, our transfer speeds have improved an awful lot since then, and should be pretty comparable these days - basically we were able to get the benefit in latency without the sacrifice. So I guess I wouldn't mind seeing that in a report, either. :-P

Such a report might happen, too. The JPR one is pretty aged at this point and getting some fresh ones showing off the current performance would be nice.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (MSim)

Wed, 24 Feb 2010 11:52:33 GMT

I don't know who paid for JPR to test the Xeno Pro, but do you know why JPR didn't do a full test on this product?
They tested gaming and downloading while gaming but failed to test the offloading or large file transfers on a home network. They talk about multimedia server in the Conclusion but didn't take the time to do any tests. Gamers who go to lan parties will share files with friends. with this product being targeted at gamers shouldn't that be part of the test too. We have seen reviews that test it and show its one area where the xeno pro doesn't perform that well.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (Arrakiv)

Wed, 24 Feb 2010 09:32:43 GMT

MSim

The load test was a joke, what gamer in their right mind would download a big file while trying to play. its common sense you don't do that without a way to limit the bandwidth. They don't even show the ping without the load (downloading a big file).

Personally, I rather like being able to stream a nice HD video file off Netflix for some background noise (depending on what I'm playing), or downloading a linux ISO while gaming. With a Xeno and Bandwidth control, I can do that. And I do, frequently. Just because you won't normally do that with a standard NIC, doesn't mean that there is anything wrong in the test, more so when it becomes possible to do that.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (MSim)

Wed, 24 Feb 2010 07:04:39 GMT

I think the JPR study is skewed too. it looks more like a sugar coated study. The spike at the front of each graphic is misleading. every game when you first connect has a higher ping. it does look like they used that to show a big increase of fps and lower ping. Players are not going to jump into battle the second they connect to a server.

The load test was a joke, what gamer in their right mind would download a big file while trying to play. its common sense you don't do that without a way to limit the bandwidth. They don't even show the ping without the load (downloading a big file). Without the extra load both NIC's would have a steady ping. It's strange they don't show any graphs on the file transfer speed during this "study".

If you want to see real world results just google/yahoo/bing Xeno Pro review, checkout what the top review sites on the net have to say.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (shaolin95)

Wed, 06 Jan 2010 16:50:11 GMT

Well, as I am want to like testing things instead of going by what others think, I a getting this on Friday.
Is this Profiles idea something you are really working on as of right now or have this been canceled?

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (Arrakiv)

Fri, 20 Nov 2009 08:31:53 GMT

You are correct, different games do use different systems for hit detection.

Currently, we are focused on raw game networking performance, since all of these methods are reliant on low latency. However, it is our plan to begin making inroads into profiling games and making sure our optimizations improve specific titles - basically like NVIDIA, yes. We will also make this automatic, so that users don't have to make use of a utility just to switch games, which wouldn't be friendly for many people.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (Balthazor)

Thu, 19 Nov 2009 10:57:21 GMT

The fact that different games use different methods for 'hit prediction' is a good argument why Bigfoot should have a profile-based system for how the card handles net traffic for different games, similar to how graphics card companies worked around the fact that different game developers code their graphics engines differently and require profiles to maximize performance.

Unless Bigfoot already does this, but it is hidden from the end user; if so, I think it would might be good to expose this level of detail so that we can adjust settings for particular applications that appear to have problems with the Xeno.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (BigfootSean)

Thu, 19 Nov 2009 07:34:10 GMT

First – thanks for your comments and questions! They show an excellent understanding of the methods and complexities of testing performance in online games. Let’s address your points:

-          From what we (Bigfoot Networks) know, Peddie Research followed either the procedure we recommended or adapted our procedure to fit their own methods. So, in the case of a game like TF2, we recommend getting into a game, setting up your characters, making sure both testers are on the same side and carry the same weapon. (Weapons are important, not only in total poly count, which can affect framerate, but also in the sort of traffic a player might send over a period of time – for example, a Heavy can theoretically send more traffic in a given period of time than a Soldier.) After a period of time, usually when the players are fully deployed and out of the locker room at the beginning of the round, framerate and ping tests start. After ten minutes of the same map / same game / same characters, we stop Fraps and either finish out the round or start another test. A full run usually involves a full hour of gameplay in ten-minute increments over 90-120 minutes. More runs are conducted when connecting to different servers over different days.
We assume John Peddie Research followed these procedures or something close to it, as they took pains to assure us that the PCs they used were identical, and connected to the same Internet connection through the same router. However, their procedures and their raw data belong to them.
-          Consistent pings ARE more important, if not to the game, at least to the player. Discarding the fact that different games use different hit-prediction methods (good point!,) the player needs to be assured a fairly standard level of playability during their game if it’s latency-sensitive at all.
This is where JPR’s testing results differed from our own internal tests. In all the Xeno Pro testing we completed, we consistently got lower standard deviations in in-game ping than a standard NIC. By bypassing the Windows network stack, the steadier traffic flow is less likely to amplify any network latency inconsistencies and deliver steadier traffic.
But we could publish our own results all day long, right? They’d only mean anything if they got decent third-party verification.
-          Which gets us to your next point – skewed studies do nobody any good. Totally true. We picked JPR to do this study because we knew of their reputation throughout the PC hardware industry. When you submit your hardware to a third party for analysis and testing, you have to live with the results or go back to the drawing board. This report is simply the independent confirmation of our performance story – a part of the whole package, if you will.
-          Finally, we do read and pay attention to our forums and those of our partners, and are especially eager to respond to posts like these that not only typify the curiosity and intelligence of our audience, but reward other forum members by explaining new concepts. We’ll try to refrain from saying that “it’s all in the feel,” because you’re not the first person that’s pointed out the placebo effect. We work more every day to not only increase the performance of Xeno Pro (check out our latest Beta driver release!) but to find ways to quantify its performance in ways that are meaningful to our customers. The JPR report is one step, we hope to have more for everybody soon.
Thanks,

Sean

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (arcsign)

Sat, 14 Nov 2009 17:46:47 GMT

(Note: I've actually got one of these cards, and as far as I can tell it seems to work pretty well. I just don't like crappy studies and misleading data.)

Eh, correct me if I'm wrong, but I'm fairly certain that isn't a performance improvement. The lag compensation/prediction system used in most games relies on a stable, but not necessarily low ping; a 10ms drop in ping time, even if it's an average 10ms (not like the CS graph up top), is utterly useless if the standard deviation (and I guess by extension, the ping jitter) is significantly greater than other network cards.

I ran some quick numbers for the TF2 and CS:S data in the white paper, which is something that I really shouldn't have had to do; the paper presents itself as a scientific study, and utterly fails in that regard. The methods and test beds sections are alright, but they give no indication of what procedure they used, which should have included some sort of information about the sample rate (What unit is used along the bottom of the graph? Minutes? Seconds? Inches? Obviously some unit of time, but still), and when they started taking said samples. It looks to me like they started taking samples as soon as the game entered the server, which makes sense given the spike at the beginning of each graph (first 2 units, in all cases). This could be construed as either bias, poor testing practices, etc, as in all cases shown, the greatest difference between the control and the Xeno (and from what I can tell by eyeballing, is enough to skew the results as an outlier in the CS:S no load, TF2 no load, and L4D under load tests.)

Some data, just because I ran it through the calculator and didn't feel like throwing it out.

Counter-Strike : Source
CONTROL
mean = 30
std.dev. = 1.69967

XENO
mean = 25.6
std.dev. = 2.87518

Just under a 70% (69.16%) increase in ping range/variability on the Xeno in exchange for a 4.6ms drop in average ping. That's really not that great of a deal.

TF2
CONTROL
mean = 43
std.dev = 2.08

XENO
mean =42.5
std.dev. = 1.71

1.011% ping improvement (interesting that the Peddie study reports 1.2%... either they have more data to play with, or they're doing some creative rounding there), and a 21.6% decrease in ping range/variability for the Xeno. These results are a bit better, but as I mentioned earlier, they don't say when they started sampling, and if my suspicions are correct, the large spike in the first 2 units is what's responsible for the difference.

Conclusions and comments...

I don't like misleading, badly done studies that appear to be biased in favor of the products under review and lack crucial bits of information that make large differences in the experimental outcomes. After reading carefully through the study and thinking a bit, I'm actually considering ditching the Xeno in favor of the onboard nic, as the ping stability on a standard card appears to be better than that of the Xeno Pro, and from what (admittedly somewhat limited) understanding I have of lag compensation and prediction, improved ping stability/consistency (less jitter, lower standard deviation) is far more important than marginally decreased latency (the prediction system relies on a stable ping in order to... predict properly. Wildly fluctuating ping is much, much worse than a stable ping even 20-30ms higher). I could really care less about how it performs if I'm downloading torrents or large files in the background, because to me that's just bizarre in a "why on earth would I want to do that?" sort of way, so I'll leave that alone (on the bright side, except for the horribly skewed L4D, that part of the card appears to work quite well).

Takeaway Points:

Lack of a detailed testing procedure calls into question the results, especially given the consistent skew caused by the first 2 unknown units on the graph. My guess is they fired up the game and start taking measurements as soon as the level loaded. The Xeno may have an edge for that period of time, but it's misleading, and not a real world situation. (I'm assuming that the Windows network stack delays a bit before giving the game network priority or something to that effect; something the Xeno doesn't have to deal with as it's already giving a higher priority to UDP packets.)
Consistent pings are more important, especially for hit registration in FPS games, than are low pings. This should be examined a bit more by the guys over at Bigfoot Networks. The variability on a large scale also makes me wonder what it's like on shorter time scales, say millisecond/sub-millisecond range, which is really what all of this network stack offloading and such is supposed to be about. (Aside: Fire up Quake Live and take a look at the netgraph output. It flickers back and forth between a lower and higher number (around 10ms apart) so quickly that it starts to blur @ 120hz. That does not bode well.)
Skewed studies are good for no one. Even if the big letters on the graphs and simplified bits make the Xeno look better, once somebody takes a closer look and sees that (a) it looks like someone paid for the results (note results, not study. funding a study and paying for results are 2 entirely different things.) and (b) the results actually make the card look bad (is it? I'm not sure anymore, and quite obviously can't rely on feel, as one's perceptions are easily biased after spending 100$ on a network card). Instead fund high-quality research (either in-house or outsourced) that, in the event the card actually makes things worse, will tell you what happened, and give you a starting point for figuring out the why part. (In fact, you're more than welcome to hire me. Seriously.)
I'd love to hear some feedback from the developers/pr guys/whatever, but if I hear, anywhere in the response that "the numbers don't show it, you have to rely on how it feels when you play," I'm going to have to assume that the only reasonable explanation is that the study was a joke, the card does very little, and the company (Bigfoot most likely, but you never know) is trying to hide it. (Which, by the way, directly contradicts this whole thread and study thing. I'd love to quote the times I've seen it said on the forums, but everything seems to have disappeared...)

-Matt

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (Vectrexer)

Thu, 12 Nov 2009 01:18:58 GMT

Thanks for the info.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (djester55)

Mon, 09 Nov 2009 08:17:17 GMT

i agree i bought this card about 3 months ago. i didnt see any difference with the card so i mailed it back to the egg to get my money back.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (onikyatto)

Fri, 06 Nov 2009 17:45:24 GMT

what u can expect is a waste of money with no refund. You can expect alot of bsods before u pull this pos outa ur comp and use an intel or onboard lan adapter. DONT BUY THE XENO

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (drowsiness)

Fri, 06 Nov 2009 08:05:11 GMT

SocioPC

Wouldn't the program need resources to do the bypassing real time though?

Yes. Yes it does need to use resources to bypass the Windows networking stack. The hardware NEEDS drivers to function. Drivers NEED to communicate with the OS kernel in order to operate. The OS kernel NEEDS to use the CPU.

You get where I am going.

SocioPC

Honestly I had good results with this product until I upgraded to windows 7, now all it does is blue screen as soon as I enter multiplayer on games no matter what I do.

Blue screens usually are a very poorly written driver problem. Sometimes it's hardware itself, but that is actually more rare than bad drivers.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (drowsiness)

Fri, 06 Nov 2009 08:02:04 GMT

sirmasterboy

I'm not really sold on the Xeno. With the integrated NIC in my 680i I already get a ping of 5-10ms in CS:S or TF2. I highly doubt the Xeno can make any difference. People say it can lower your bings by 5-10ms. If it did that I would have a 0 ping? That is not possible.

If your ping is much higher, then it's not your LAN that's adding the latency. Its your ISP which you have no control over. Just do a Traceroute and see the hops through your LAN.

The TCP/IP stack in Windows is alreay highly optimized and there is plenty of extra CPU power to do all this work especially since quad cores and hyperthreading are coming back into mainstream CPUs.

That's my 0.02¢

I've been saying this on the forums for a while now... no one will listen. The people who bought it have perceived benefits from using this piece of hardware and nothing in this world can convince someone otherwise, apparently.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (Arrakiv)

Mon, 02 Nov 2009 08:52:21 GMT

SocioPC: Have you tried contacting our customer support?

http://bigfootnetworks.com

We've done plenty of testing in Windows 7 (I also run Win7 on my home machine with a Xeno with no issues), so there might well be something wrong that is entirely fixable.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (SocioPC)

Sat, 31 Oct 2009 15:11:36 GMT

Wouldn't the program need resources to do the bypassing real time though?

Honestly I had good results with this product until I upgraded to windows 7, now all it does is blue screen as soon as I enter multiplayer on games no matter what I do.

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (pgmoney)

Sat, 31 Oct 2009 13:00:56 GMT

Bypassing the default Windows Network Stack. (by default the Windows Network Stack is really not suited for gaming)

LOL!
I can do this manually and already have. Someone can just run a simple program that does the same thing and is free, there is many out there on filehippo and other tweak system tune software sites. google people google

notice the word default? yes you can change this without this card!

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (doorules)

Sat, 31 Oct 2009 04:22:45 GMT

so teamspeak 3 beta release date has been announced for Dec 19/09....http://www.gameservers.co...te/article.php?id=1361

does this mean that the xeno will be able to be offload TS3 to the card ?

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (doorules)

Sat, 31 Oct 2009 03:31:56 GMT

just go have a read in the older forums, lots of feedback there

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (spitfire5792)

Fri, 30 Oct 2009 13:55:54 GMT

I know about the benchmarks and everything but in everyone's own opinion is this really worth it?

Re:What to Expect with the EVGA Killer Xeno Pro - Updated (sirmasterboy)

Fri, 30 Oct 2009 10:42:59 GMT

I'm not really sold on the Xeno. With the integrated NIC in my 680i I already get a ping of 5-10ms in CS:S or TF2. I highly doubt the Xeno can make any difference. People say it can lower your bings by 5-10ms. If it did that I would have a 0 ping? That is not possible.

If your ping is much higher, then it's not your LAN that's adding the latency. Its your ISP which you have no control over. Just do a Traceroute and see the hops through your LAN.

The TCP/IP stack in Windows is alreay highly optimized and there is plenty of extra CPU power to do all this work especially since quad cores and hyperthreading are coming back into mainstream CPUs.

That's my 0.02¢

What to Expect with the EVGA Killer Xeno Pro - Updated (Delirious.)

Fri, 02 Oct 2009 09:47:45 GMT

original post by JacobF http://www.evga.com/forums/tm.asp?m=100734097&mpage=1&key=?

Now we have final 3rd party benchmarks that show what you can expect with the EVGA Killer Xeno Pro.

Full benchmarks located here:
[link=http://www.jonpeddie.com/special/WhitePapers/NetworkGamingAcceleration.zip]http://www.jonpeddie.com/...GamingAcceleration.zip[/link]

Here is CS Source, first without huge load on the network:

Now with a load on the network:

quote:

Killer Xeno Pro provides a clear performance benefit if the user encounters a heavy online gaming load, like multiple enemies or friends in the same area. If the user wants to download or upload files while gaming with no impact on performance, the Killer Xeno Pro excels at this task as well. Finally, if the user communicates with a voice chat program while gaming, Killer Technology and its Hardware Bandwidth Control provide a clear advantage to the gamer.

Sow where is the performance improvement coming from? Well the short answer is:

Bypassing the default Windows Network Stack. (by default the Windows Network Stack is really not suited for gaming)
400MHz NPU to handle the traffic.
128MB of onboard DDR2 for Killer applications.
Bandwidth control to prioritize game traffic.
Hardware Accelerated voice chat.

This below illustration may help as well:

I would suggest checking out the full evaluation here!

[link=http://www.jonpeddie.com/special/WhitePapers/NetworkGamingAcceleration.zip]http://www.jonpeddie.com/...GamingAcceleration.zip[/link]