A common pattern I’ve noticed is that the games industry is frequently a step or two behind the rest of the software development world – more or less as you’d expect given our rapid growth – we’re like the fast-growing child whose trousers don’t reach their ankles. Use (or misuse) of source control is a classic symptom: winding time back a little, when I started out in 2001, I cringe to think how pleased we were just to be using source control at all – it was like, we’re all up to speed and “doing things properly”. Never mind that we were using SourceSafe and only storing the code in it. Looking back, it’s both remarkable and embarrassing that development books of the time (such as this one from 2000) felt the need to explain what source control was and why it was important. Look what Microsoft were doing at the time – 1400 developers, 16 depots, branching, …
But that was long ago
I can hear you already … yeah yeah, that was a long time ago, we’ve moved on since then, everyone’s using source control now. True. But the story doesn’t end there. When I first moved to Realtime, we were using Alienbrain, and seemed pretty delighted with how much better than SourceSafe it was. Here are a couple of my memories:
- Archives of old file revisions were stored in “buckets” which were conveniently CD-sized. The idea was that rather than buying server disk space, you just copied your old buckets to CD. Wow, was this annoying and stupid! Did you check the price of disk space recently? Suppose you had so much data that you couldn’t fit it on a hard drive – do you know how long it would take to transfer that data onto CDs!? Best of all, it led to one of the most entertaining dialog boxes ever … upon trying to view an old revision of a file, you were told “please insert the CD for bucket 578” … so off you went to get the filing cabinet key, find CD 578, insert into server, back to your desk … only in the games industry!
- Branches were a new “feature” for the version we were using (version 7, I seem to remember) … but when we tried to branch some files as a test, it corrupted the entire database and we had to restore from backups. We contacted support and were told yeah, don’t use branching. But it was still a “feature” in their marketing materials!
Truly, the only good things that spring to mind were (1) integration into common content production tools such as 3DS Max, Photoshop etc, and (2) a reasonably slick UI with thumbnails of content assets in the repository browser etc – both useful, but not at the expense of having the basics working. Maybe it’s all fixed now (we were using version 7 and they’re up to 8.1 now) but come on! How can you be on version 7 of your software and have such fundamental problems as branching that corrupts the whole repository!? I’m not the only one pointing out how much it sucked.
The Alienbrain experience has made me fundamentally suspicious of any software which ought to be widely applicable, but is marketed specifically to game developers. It’s like they know we’re suckers! Obviously, some software is specific to game development and has to be marketed as such, but it really seems as though source control is a pretty general-purpose product. Credit to Alienbrain for coming up with a smart niche marketing position in a crowded product space (“Asset Management for Artists”), and I do think they’re genuinely trying to help artists (source control products tend to be written for programmers, by programmers) but you have to back it up with a decent product: this stuff is mission-critical to us.
We then moved to perforce – definitely a good product: Microsoft and Google are pretty good customers to have, it has plugins for content production tools, it’s rock solid and scales to lots of big binary files at high speeds. Success at last? Well, eventually … there’s still been resistance in some places to basic concepts such as branching, but by now I reckon we’re reasonably caught-up at RTW.
We keep all our assets in perforce so that we have true versioning of everything that goes into making the game. We use hierarchical branches to parallelise development, isolate risky changes, stabilise releases for testing and produce demos with minimal disruption. We integrate source control with bug tracking, run overnight builds and continuous integration off it, and our in-house tools come with perforce integration. Oh, and we back it up. I think we’re in good shape, but it’s shameful that this stuff is even worth mentioning in the year 2008. And I can’t help wondering, in what new ways are we a step behind now?😦