Why not to build in-house tools

Part 1 of “Building World-class Art Tools” from Develop 08

Before I talk about how to build great in-house tools, I want to talk about why to build in-house tools, or looked at another way, why not!  Understanding the motivation and reasons behind doing your work, whatever that work may be, is always of critical importance.  You can be doing the best work in the world, but if your motivation is wrong in some way, that work could just be a waste of time.

The reason I want to look at tools from the “why not” angle is that there are many powerful reasons against building in-house tools.  If you’ve worked in games for a while, think back over the in-house tools you’ve seen, whether you built them, used them or simply saw them.  I bet you can remember seeing some lousy software.  I know I can.  Nobody sets out to build mediocre tools, nor are the tools programmers any less capable than the rest of the team.  Why are the results frequently so poor?

Today I present four key reasons why in-house tools often suck.  Understanding these should give you some insight into whether to actually build those tools at all; should you decide to go ahead and build, it will give you insight into the pitfalls you need to avoid.  Several of these reasons have nothing to do with games, or art tools.  The term “in-house software” is a dirty term the world over; here’s what Joel Spolsky has to say about it:

if you’re not very, very careful when you graduate, you might find yourself working on in-house software, by accident, and let me tell you, it can drain the life out of you.

Lock-in

When you buy commercial software, you make a rational choice between products in the market, with plenty of information to compare them.  This basic market effect in turn forces the products to compete with one another, driving the quality up.  Many aspects of our modern quality of life are in large part thanks to the power of this mechanism: do not under-estimate it.  In contrast, when you use in-house software, your users are not exercising a choice (they will generally be forced to use it even if it’s poor quality), or do not have all the information they need to choose (at the point that you decide to start building in-house, you don’t know how your in-house software will turn out, how much it will cost, or when it will be ready).  Furthermore, there are no market effects on your developers, which makes them lazy.  If their software isn’t quite good enough, they’re less likely to be out of a job than they would be if their customers were paying for it.  This is the lock-in problem.

Giraffe feeding at zoo

You can’t eliminate the lock-in problem.  Well I guess logically, you might think you could, you just have to sell your tools publicly, right?  Well that’s definitely not something I’m going to advocate.  You’re a games company, and you don’t want to distract yourself from that!  You’d no longer be making in-house software, so you might as well have been a tools company in the first place.  There are isolated examples of making this work – game developers selling their engines, and of course Pixar selling Renderman – but they are the exception rather than the rule (and many of those engines are hardly success stories).

If you can’t eliminate the lock-in problem but you want to make great tools, you have to fight its symptoms with everything you’ve got.  Find developers who are passionate about making the best software they can, and keep them that way: cultivate a culture of caring deeply about your content creators.  Foster close communication so that artists are involved deeply in the design of the tools.  Apply best practices in your development, with rigour: pretend you’re making professional, commercial software, even if it’s “just” for the guys down the hall.  This stuff is not rocket science, and there is plenty of information in books and online.  Use branching in version control to allow maintenance releases of older versions.  Provide backwards compatibility on file formats.  Provide help, both reference and tutorials.  Use a bug database.  Use a professional release process with an installer, release notes and QA.  Do you have dedicated testers and customer support for your tools?

Good enough

The next problem is even more serious: in-house tools are often not directly related to how your company makes money.  You are not working directly on the consumer product (the game) – you are providing an internal service.  This means that it doesn’t matter too much how great the tools are; it means that once they’re “good enough”, there’s no business case for making them better.  There is no benefit to going the extra mile.  If you polish your game a little bit, it should translate into extra sales; if you put the same investment into your tools, the return on investment is unlikely to be the same.  In business terms, you’re equivalent to other internal services, such as administration, accounts, cleaning.

Mop bucket

Unfortunately, building serious software is orders of magnitude more difficult than these other internal service functions. We all know that the last 10%, the last layer of polish on software, can cost almost as much as the rest of development.  Reaching the stratosphere is expensive, and in-house tools can seldom justify this expense.  Sadly, software that’s simply “good enough” is unlikely to feel that way.  It’s more likely to appear lousy to its users, who are accustomed to polish.

More work than you think

When you start building in-house tools, do you really know what you’re letting yourself in for?  We’ve found that we’ve spent 3 times as much effort on the tools as on the technology underlying them, and that ratio is rising as we continue to polish the tools (Ubisoft quoted a 4-to-1 ratio for their procedural environment tools on Far Cry 2).  We’ve amassed half a million lines of code on the tools we use for APB, and we’ve been going for nearly 6 years.

An iceberg

If this amount of work is heavily under-estimated (and it usually is), your initial decision on whether to build in-house tools may be heavily misguided.  The danger is that you prototype some cool technique in pre-production, think “let’s go into production now, we can easily slap a UI on that”, and find you’re still working on that UI years later.

Parallel pain

Finally, there’s the issue of parallel development.  If you develop in-house tools in parallel with the game project that’s using them, you could be letting yourself in for trouble.  In-development software has all kinds of problems, from bugs to missing functionality.  It also suffers from slippage like any other software, so you have another risk to your main project cost and schedule.

Spot the cat

Spot the cat

The trouble is, it’s hardly ever possible to build tools in isolation before starting work on your game.  The very nature and purpose of the tools is likely to be bound up with the project itself.  Furthermore, until the tools are being used in anger, you don’t even know when they’re done.  We addressed this problem by building our APB toolset in parallel with Crackdown.  We were able to use some of the tools in “moderate anger” on Crackdown, thus getting a clear picture of how complete they were, but without making Crackdown entirely dependent on them, so the project risk was eliminated.  Once we started APB, the tools already had a certain level of maturity.  There was still plenty of parallel development, and this has caused some pain at times, but I think we managed to reduce it a lot.

Just don’t do it

I’ve described four problems, and they’re not just minor issues; they are powerful underlying forces, which combine to reduce the quality of in-house tools and cause problems during their development.  If your content creators are to make significant use of these tools, you could be in for big problems.  The efficiency of your content creators may have a big impact on your development costs.  But most of all, their creativity will have a big impact on the quality of your game.  If they spend their days fighting with inferior software, they are not going to do their best work.  They will be demoralised, and they may iterate on their work less to avoid painful parts of the process.

Commercial tools are likely to be high quality (provided you choose carefully), cheaper than building yourself and certainly less risky.  They will be ready from the start of your project, and your artists may already know how to use them, so there will be less time spent training.

So my first piece of advice has to be: don’t build in-house tools without very good reasons.  Your first option should always be to buy a commercial product, a known entity whose functionality and costs you can evaluate up-front.  Your second option is probably to extend a commercial product – assuming they have some kind of scripting or plug-in mechanism built in.  In many cases, this may heavily reduce the amount of custom work you do.

Next time, in part 2, I’ll turn to a more positive outlook, and examine some positive reasons for building in-house software.

This entry was posted in Develop 08, Software development, Tools. Bookmark the permalink.