After much consternation, swearing, and the odd reboot it finally happened. I upgraded my PC. I squeezed every penny together I had spare, got new components, prayed to Zommoros that whatever collection of trash I could afford would spit out at least 60FPS, and I started Guild Wars 2.
The result, as you will guess, is a much more complex question than throwing money at a screen and waiting for more power to push past the 60 barrier. MMORPG performance is a multi-faceted issue. It covers various types of technology, distributed computer systems, as well as a multitude of hardware and software layers. So, when one of the community asked what was bottlenecking their game we really had to split the whole question down into several chunks. Going forward I’ll be taking a high-level overview of a number of these technologies, and discussing the systems particular to Guild Wars 2. We might even get some buy-in from ArenaNet on the squirely black magic that holds the Mists together. For now, however, we are going to look at issues that directly impact the local client as much as possible.
To do this I’m going to have to frame this discussion. This means that, in this instance, we will be looking at three indicators of local client performance. These are frames per second, load, and render times. This might seem like a very limited range of tests but these should give us a good indication of the resulting player experience. Other factors may influence the overall performance but these simple checks are solid and measurable indicators of in-game performance.
The Test Hardware
I am lucky enough to have a number of systems available to me. To this end we will be outlining them here
System Config 1
- Ryzen 5 1600 at stock
- 16 GB DDR4 at 2400Mhz
- Crucial BX500 250GB SSD
- MSI Overclocked 1070Ti
- Asus Prime B350M-A
- Gigabit Ethernet
System Config 2
- Ryzen 5 1600 at stock
- 16 GB DDR4 at 2400Mhz
- Samsung Spinpoint F1 1TB HDD
- MSI Overclocked 1070Ti
- Asus Prime B350M-A
- Gigabit Ethernet
System Config 3
- Shadow Desktop
- 12GB of RAM. DDR4 at 2400MHz
- 259GB virtualized data center storage
- GTX 1080
- Gigabit connectivity
System Config 4
- Acer VN7-592G
- Intel i5 6300HQ @ 2.3GHz
- WD Blue 3D NAND SATA III SSD
- 8GB DDR4 RAM
- GTX 960M
- 5Ghz Wireless
System Config 5
- Acer VN7-592G
- Intel i5 6300HQ @ 2.3GHz
- WD Blue 3D NAND SATA III SSD
- 8GB DDR4 RAM
- Intel HD 530 Graphics
- 5Ghz Wireless
Limitations
Before we get to the methodology, we need to be clear that everything has limitations. With time constraints involved, we decided to use a range of real-life hardware rather than virtualize all the testing. With a virtualized system we would find definite granular control of CPU, memory, and disk resources but it does have a few downsides. The ease with which we can change out the underlying hardware, throttle GPU utilization, and hot-swap hard drives made a compelling case for using local hardware. We might very well see some perception issues due to the incredibly complex network configuration involved in delivering an MMORPG. Bearing all this in mind let’s get onto the plan of attack.
)
The Tests
We’re going to take a relatively high level and simple approach to testing local client and proceed to dive deeper as we progress. Our initial test will primarily consist of two basic checks.
FPS and Performance Testing
FPS testing in Guild Wars 2 is problematic at the best of times. Anybody jumping into WvW can attest to this. We needed to pick a way that would drastically reduce the variables involved while still ensuring that we were being realistic in our test. Logging tests on each of the available systems, we chose to idle in a spot overlooking the Mystic Forge. This was a good compromise as it has a level footfall of users that tend to spend a good length of time in front of the forge. It also demands that any system render a number of effects spouting out of the Forge. In essence, the mystic toilet is about as consistent as any populated environment in Tyria is going to be.
Measuring the frame rate is possible in multiple systems. We utilized ArcDPS to give us a note of the frame rate and confirm that our response times to the game servers were not fluctuating while MSI’s afterburner was used to generate an output over time. This has the added benefit of capturing CPU core usage, memory utilization and giving a clear indication of server latency during testing. This allows for easy identification of any unexpected bottlenecks. Each of the following tests was run using the best appearance and best performance settings.
Load and Render times
Load and render times are a little more problematic to test then frame rates. Despite this, we don’t need a system that can measure down to milliseconds. We simply need to pinpoint pain issues. To this end, we took to the Path of Fire for this test. We performed a waypoint jump from Lion’s Arch to the city of Amnoon and timed the time taken from clicking the Amnoon waypoint until the cityscape finished rendering. This was repeated multiple times to ensure no issues or errant problems with the load times and compared,again, to the time to travel back to Lion’s Arch. The Number you will see here is a rounded average. Remember, we aren’t looking for the greatest precision, simply an indicator of performance bottlenecks.
In both instances, these tests were performed with both best performance and best appearance presets in the game’s graphics settings.
)
The Fast And The Furious
The numbers here aren’t all that surprising in their distilled form. The laptop running on best appearance, without a discrete GPU had the hardest time of all, idling at 5 frames per second as the system tried to crawl through the particle effects being thrown at it, while the configurations with the Nvidia 1070Ti and 1080 GPU (config 1 and 3) had the easiest time chewing through at 20, 25, and 20 FPS. Notably, Configuration 2 actually fared better than the Shadow desktop, suggesting that once we got things up and running, the storage medium wasn’t of any real consequence.
While the drop-down in frame rate between the 1070Ti and 960M is significant, the rough drop of up to 50 % still comes up trumps compared to the 66.6% reduction when switching from the 960M to the Intel HD 530 in configuration 5.
It’s fair to say that when it comes to pumping out frames, getting a graphics card is a must. The difference in CPU load on our laptop when switching between the 960M and Intel graphics is visible and consistent. The right-hand change shows the uptick when we turned off the graphics card.
)
That said, all cores of the laptop’s Intel i5 6300HQ, even with boosting available, were still put under some continuous pressure when set to best appearance, and this was just while idling in Lion’s Arch. At a base clock of 2.3Ghz, we still found that this was consistent across all our systems. Processor utilization isn’t everything that we need to talk about when it comes to Guild Wars 2 but it is always going to be an obvious constraint, even in light activities. To show you just how egregious Guild Wars 2’s CPU hunger can be I dropped a toe into WvW, taking config 3 up against a zerg at best appearance and watch the CPU rocket up, maxing out on several occasions and FPS falling through the floor, despite a 1080 and data center grade internet connection that is simply unparalleled by most home PC standards.
)
When we get into WvW or other very busy situations the complexities of MMO systems start to rear their heads, with more and more information to crunch through. SImply put it gets to this. If 8 Xeon 2620 threads can come near to maxing out, then we can’t do much more than point out that Guild Wars 2’s CPU efficiency will always be problematic. Note this and move on or check out this comment on Reddit if you want to delve a little deeper.
Lock and Load
Time to Amnoon might not seem like much but the difference in time to load and render is pronounced enough when you look at the results. Specifically, load and render times are a bit of a mixed bag. The biggest change comes between config 1 and 2. Running Guild Wars 2 on an SSD has significant advantages over a traditional Hard Disk Drive Load times. Figures between Lion’s Arch and Amnoon fluctuate by around a third and this behavior seems to be mirrored, although not as heavily, by the same test when running on best performance mode.
What did surprise us was the Shadow desktop load times. What has to be said here is that we don’t know exactly what sort of storage medium is being used by Shadow. We can guess but it wouldn’t be fair to make that assumption at this point.
The other thing that you might not have twigged is that everything doesn’t come down to the hard drive. When we kicked the graphics processing over onto our Intel HD 530, it seems that stressing the processor slowed everything down. Data transfer in your computer is managed by your CPU to some extent or another. To this end, if it’s too busy trying to shuffle pixels, then you’ll find a slow data transfer rate in spite of a pretty fast SSD. Compare the similar tests on the SSDs for our desktop on the left and laptop on the right, then consider the difference in load times. Significant, right?
So what did we notice?
It’s really quite difficult to give a definitive description of what component gets the majority of the load from the Guild Wars 2 client. Well, actually it isn’t. It’s clear that the game is incredibly CPU demanding and most of the community already know this. We also found that even in WvW the game was happily chewing through around 3.5GB of system RAM so make sure you’ve got a good 8GB of memory and a x64 bit operating system to use it all.
Really, the way in which the frame rate and load times succumbed to the removal of any discrete GPU support tells its own tale. Guild Wars 2 might be known in the community as a processor intensive game but past a certain point, we saw little difference with even the shadow desktop slowing off. If your processor is not maxed out then there are a ton of other limitations that are going to cause slowness. Slow load times are subject to a number of components and low frame rates also seem to be impacted summarily, but yet pound for pound the most efficient change that impacted multiple areas of performance was switching on a mid-tier graphics card. Like I said earlier, MMOs are complex beasts and taming their performance is hard, and we have even talked about networking yet.
Next time we take a look I’ll be going deeper, looking at why WvW destroys our frame rate and talking a bit more about what makes MMORPG so uniquely difficult. In the meantime, what’s your rig running? Are you finding the same thing we did or are you just obliterating our results?
