Last time, I talked generally about the trend of game memory usage. Today, I thought I’d expand a little on some of the specific issues around memory usage on 32-bit editions of Windows. This happens to have been highlighted recently by APB: if you run the game on 32-bit Vista or Win 7, you will be forced to run with noticeably lower quality graphical settings; 32-bit XP and 64-bit editions of Windows do not suffer from these problems. What’s up with that?
The short answer is that these various editions of Windows provide different amounts of virtual address space. 64-bit editions of Windows provide the most, followed by 32-bit XP, with the least virtual memory available on 32-bit Vista and Win 7. APB is so close to the limit on those last two platforms that it was necessary to reduce memory usage in a few ways (such as reduced texture resolution) to avoid having the game run out of memory and crash. Most graphical settings affect only the framerate, so gamers are free to choose whatever level they like – but on this one, the decision is forced in code, to prevent crashes.
The long answer
Let’s start with some virtual address space basics. There’s an excellent article about virtual memory on Windows here – as a quick summary of the points most relevant to this post:
- While a 32-bit application can theoretically address 4GB of memory, Windows traditionally reserves the upper 2GB for itself.
- At some point Microsoft introduced a /3GB boot option to Windows, allowing an application to have 3GB (with Windows reserving just 1GB), provided the application declares itself as “large address aware”. The reason they require applications to opt in to this is that some applications make hard-coded assumptions about all pointers being in the lower 2GB (for example, setting the high bit in a pointer as a flag for their own usage). The /3GB option is obviously very little use to a game wanting to use more memory, as it requires users to boot their machine differently.
- On 64-bit Windows, 32-bit apps are still limited to 2GB by default, but those “large address aware” apps will now get the full 4GB – Windows doesn’t actually need to reserve any of the address space. Again, this is opt-in to avoid exposing problems in applications that have made a (dodgy) 2GB assumption.
Based on just this, you can understand why APB has more virtual memory available on 64-bit Windows (yes, it is “large image aware”), but it’s not clear why the 32-bit editions differ. Why is XP different than Vista and Win 7?
Some simple experiments
Firstly, I thought I’d do a few simple experiments to demonstrate how XP is different. I started out with a simple “Hello World” console application, and used the performance counters API (PdhGetRawCounterValue) to find out how much virtual memory had been used by the application. Turns out, this trivial application uses 60MB of virtual memory on Vista, compared to 32MB on XP. Clearly, Vista is substantially more memory-hungry for even the simplest of applications!
Supposing we’re interested in games, not just a console app, then the obvious next step is to take the game equivalent of “Hello World” – I chose this very basic Direct3D tutorial that draws a triangle on screen. Here’s what I found:
- On the first line of the programme, before initialising D3D or doing anything else, the virtual memory usage had already increased: 39MB on XP, 67MB on Vista. That’s a 7MB cost on each platform, essentially down to having the additional DirectX DLLs loaded into the address space.
- Immediately after initialising D3D (specifically, creating a window, calling Direct3DCreate9 and CreateDevice), virtual memory usage jumped again: to 43MB on XP and 110MB on Vista. Those are increases of 4MB and 43MB respectively.
It’s this last increase that is most revealing. On Vista, D3D usage causes significant graphics driver allocations from your virtual address space; on XP, the same operations seem almost free (presumably, the same allocations are made from the reserved upper 2GB of the address space). That’s 43MB just to initialise the graphics system – we’ve not created any hardware resources yet.
The final experiment is on … APB itself! I’ll save myself the work here and refer to this insightful comment on my previous post, from APB’s Technical Lead, Rob Anderberg:
It’s worth mentioning that when running under 32 bit Vista and Win 7 APB has about 300MB less virtual memory available to the process after we initialise the engine when compared to 32 bit XP.
Clearly, in a full game, the graphics driver has to do a lot more allocation – internal data structures for managing shaders, textures, vertex buffers and other hardware resources. And in APB’s case, that comes to 300MB. About 15% of your address space gone, just like that. And all because …
They got rid of the blue screen of death
Well, amongst other things. And, er, not quite. Ok, they reduced it by 20%.
Vista introduced a new driver model. This was, in several ways, a big step forwards, for example:
- It allows the GPU to be used by all desktop windows, and allows multiple processes to share GPU time and video memory cleanly. The fancy new “Aero” UI is reliant on this.
- Much driver code now runs in user mode, not kernel mode, meaning that driver crashes crash the application, rather than causing a blue screen of death. Graphics drivers were supposedly responsible for 20% of all BSODs, previously, so this is a very Good Thing.
(better support for DRM is also quoted as a ‘benefit’ on the MS page). All that goodness comes at the price of eating away at the 2GB address space limit. Maybe they felt this was a good tradeoff in its own right; or maybe they assumed that 64 bit editions would take off much faster from Vista onwards? Either way, memory-hungry games like APB are left with some tough choices to make on 32-bit Vista and Win 7.
Customers don’t care
Going back to Rob’s comment again,
When players give feedback on APB running under 32 bit Vista and Win 7 they tell us two things; first of all that they don’t like the compromises we had to make to ensure that the 32bit Vista and Win 7 clients don’t run out of virtual memory and crash. And secondly that they don’t see why the 32 and 64 bit versions should look different.
It’s always really tough to explain away deficiencies in an experience that are down to technical problems. Customers just want the game to be awesome, and rightly so. And proper technical explanations sound like “blah, blah, blah” to a lot of people. We may be about to enter a time when more games have to make compromises like this, and consumers come to understand the limitations of a 32-bit OS, but over the last few weeks, it’s been APB taking it on the chin.