32-bit Windows and APB

Last time, I talked generally about the trend of game memory usage.  Today, I thought I’d expand a little on some of the specific issues around memory usage on 32-bit editions of Windows.  This happens to have been highlighted recently by APB: if you run the game on 32-bit Vista or Win 7, you will be forced to run with noticeably lower quality graphical settings; 32-bit XP and 64-bit editions of Windows do not suffer from these problems.  What’s up with that?

The short answer is that these various editions of Windows provide different amounts of virtual address space.  64-bit editions of Windows provide the most, followed by 32-bit XP, with the least virtual memory available on 32-bit Vista and Win 7.  APB is so close to the limit on those last two platforms that it was necessary to reduce memory usage in a few ways (such as reduced texture resolution) to avoid having the game run out of memory and crash.  Most graphical settings affect only the framerate, so gamers are free to choose whatever level they like – but on this one, the decision is forced in code, to prevent crashes.

The long answer

Let’s start with some virtual address space basics.  There’s an excellent article about virtual memory on Windows here – as a quick summary of the points most relevant to this post:

  • While a 32-bit application can theoretically address 4GB of memory, Windows traditionally reserves the upper 2GB for itself.
  • At some point Microsoft introduced a /3GB boot option to Windows, allowing an application to have 3GB (with Windows reserving just 1GB), provided the application declares itself as “large address aware”.  The reason they require applications to opt in to this is that some applications make hard-coded assumptions about all pointers being in the lower 2GB (for example, setting the high bit in a pointer as a flag for their own usage).  The /3GB option is obviously very little use to a game wanting to use more memory, as it requires users to boot their machine differently.
  • On 64-bit Windows, 32-bit apps are still limited to 2GB by default, but those “large address aware” apps will now get the full 4GB – Windows doesn’t actually need to reserve any of the address space.  Again, this is opt-in to avoid exposing problems in applications that have made a (dodgy) 2GB assumption.

Based on just this, you can understand why APB has more virtual memory available on 64-bit Windows (yes, it is “large image aware”), but it’s not clear why the 32-bit editions differ.  Why is XP different than Vista and Win 7?

Some simple experiments

Firstly, I thought I’d do a few simple experiments to demonstrate how XP is different.  I started out with a simple “Hello World” console application, and used the performance counters API (PdhGetRawCounterValue) to find out how much virtual memory had been used by the application.  Turns out, this trivial application uses 60MB of virtual memory on Vista, compared to 32MB on XP.  Clearly, Vista is substantially more memory-hungry for even the simplest of applications!

Supposing we’re interested in games, not just a console app, then the obvious next step is to take the game equivalent of “Hello World” – I chose this very basic Direct3D tutorial that draws a triangle on screen.  Here’s what I found:

  • On the first line of the programme, before initialising D3D or doing anything else, the virtual memory usage had already increased: 39MB on XP, 67MB on Vista.  That’s a 7MB cost on each platform, essentially down to having the additional DirectX DLLs loaded into the address space.
  • Immediately after initialising D3D (specifically, creating a window, calling Direct3DCreate9 and CreateDevice), virtual memory usage jumped again: to 43MB on XP and 110MB on Vista.  Those are increases of 4MB and 43MB respectively.

It’s this last increase that is most revealing.  On Vista, D3D usage causes significant graphics driver allocations from your virtual address space; on XP, the same operations seem almost free (presumably, the same allocations are made from the reserved upper 2GB of the address space).  That’s 43MB just to initialise the graphics system – we’ve not created any hardware resources yet.

The final experiment is on … APB itself!  I’ll save myself the work here and refer to this insightful comment on my previous post, from APB’s Technical Lead, Rob Anderberg:

It’s worth mentioning that when running under 32 bit Vista and Win 7 APB has about 300MB less virtual memory available to the process after we initialise the engine when compared to 32 bit XP.

Clearly, in a full game, the graphics driver has to do a lot more allocation – internal data structures for managing shaders, textures, vertex buffers and other hardware resources.  And in APB’s case, that comes to 300MB.  About 15% of your address space gone, just like that.  And all because …

They got rid of the blue screen of death

Well, amongst other things.  And, er, not quite.  Ok, they reduced it by 20%.

Vista introduced a new driver model.  This was, in several ways, a big step forwards, for example:

  • It allows the GPU to be used by all desktop windows, and allows multiple processes to share GPU time and video memory cleanly.  The fancy new “Aero” UI is reliant on this.
  • Much driver code now runs in user mode, not kernel mode, meaning that driver crashes crash the application, rather than causing a blue screen of death.  Graphics drivers were supposedly responsible for 20% of all BSODs, previously, so this is a very Good Thing.

(better support for DRM is also quoted as a ‘benefit’ on the MS page).  All that goodness comes at the price of eating away at the 2GB address space limit.  Maybe they felt this was a good tradeoff in its own right; or maybe they assumed that 64 bit editions would take off much faster from Vista onwards?  Either way, memory-hungry games like APB are left with some tough choices to make on 32-bit Vista and Win 7.

Customers don’t care

Going back to Rob’s comment again,

When players give feedback on APB running under 32 bit Vista and Win 7 they tell us two things; first of all that they don’t like the compromises we had to make to ensure that the 32bit Vista and Win 7 clients don’t run out of virtual memory and crash. And secondly that they don’t see why the 32 and 64 bit versions should look different.

It’s always really tough to explain away deficiencies in an experience that are down to technical problems.  Customers just want the game to be awesome, and rightly so.  And proper technical explanations sound like “blah, blah, blah” to a lot of people.  We may be about to enter a time when more games have to make compromises like this, and consumers come to understand the limitations of a 32-bit OS, but over the last few weeks, it’s been APB taking it on the chin.

This entry was posted in Software development. Bookmark the permalink.

3 Responses to 32-bit Windows and APB

  1. Dundee Dan says:

    I completely understand the issues you’ve had on this but Vista was available to developers from Mid 2005 and actually at the end of of 2006.

    What I’m getting at is this issue couldn’t have been a surprise and something you guys must have been aware of for the majority (if not all) of APB’s development.

    Generally a good way to approach PC dev is target having a great experience in the likeliest mainstream spec set up which from an OS pov would be Vista 32bit. From the end results it almost seems like the team did the opposite and developed for the fat address space of a 64 bit OS.

    I can sort of understand your problem on memory usage, player models seem to have 512×512 textures with 10 mip levels and (I’m guessing here) 8 channels which is over 2mb per character. When you have a 100 of those in the game as well any car modifications I can see it soon adds up.

    That said customisation has been pushed as one of the core pillars of APB and the restrictions on 32bit OSes completely destroys any benefit you get from time spent in the excellent customisation too.

    It’s a shame but is there any argument that if the game was strictly targetted for 32bit Vista during dev you would have ended up with a much better experience for those players?

    Interested to hear your comments

  2. lukehalliwell says:

    I’m sure we could have done things to make things better for 32-bit Vista players during development, yes. But as you know, on any large project, there are many competing things you could spend your time on. We opted to reduce texture quality on Vista and spend that time on other things instead. With hindsight, I’m sure you could make an argument that was a mistake, but right now all we can do is try to improve the situation.

    You’re wrong about the machine spec we targeted, though. We developed primarily on 32-bit XP (APB’s development pre-dated Vista!), and certainly never targeted 64-bit OS’s as our main spec. It’s worth reiterating that 32-bit XP is still the dominant, mainstream machine spec – see my previous post – with 64-bit Win7 second. 32-bit Vista is about 15% of the market, so while it’s significant, it’s definitely not the mainstream.

  3. Dundee Dan says:

    Sorry, all of the public press has had APB under development for 5 years now and Vista has been available since 2005 so I got that wrong.

    Yep I can see 32 Bit Vista is 15% but combined with 32 Bit W7 it’s actually 26% of your market.

    It’s not an inconsequential amount. Not trying to be horrible but you can’t be happy about the level of support for one of your games core pillars (customisation) you offer to 1 in 4 of your customers.

    The way it comes across here and on the forums is a as a finger pointing exercise at the operating system where the problem is described in terms of deficiencies in Windows.

    My point of view is the memory restrictions of 32bit Windows (post XP) have been known and understood for the last 5 years.

    Okay that may not be all of APB’s development time but it is a majority.

    Out of interest why does running out of memory for something mean that the game has to crash?

    Can’t you offer something where players can prioritise where they want gfx detail spent and the game just tries to deliver the best it can with the memory available. I can imagine feeling a little better about this if at least my player character looked nice after spending time customising it.

    Anyway, that’s my 2p🙂

Comments are closed.