Feeds:
Posts
Comments

My to-do list

I’m a bit of a to-do list nut, always looking for new and better ways to manage the list. It sounds trivial, and in some ways it is, but I think it’s also very important for a manager.  One of the distinguishing features of management work compared to engineering is that you have so many more things in flight at once, and being well-organised about them is important to success. Unlike some of the other keys to success, this one seems fairly easy – there shouldn’t really be any excuses for being bad at it.

I’ve been through a number of different systems.  I’m not sure whether that’s because my job changes, because I find better systems, or because I just get bored of one system after a while.  My current systems do not involve a computer, which feels somehow both obvious and impossible to explain.  Last week, I added a new element, which I find oddly exciting.  But let’s start at the beginning.

Everything starts with the notepad: my portable recording device.  It’s a bit sad, but I can hardly go anywhere without it any more.  I imagine myself stuck with the habit when I retire, unable to leave the house without it, doddering round town clutching a notepad.

For a while, I used to keep the to-do list itself on the notepad. There’s a convenience to that, but I got fed up of it. After I’d crossed out two-thirds of the list, it would start to look messy and I’d feel compelled to copy the remaining items onto a new page. Over the years, the rate of turnover on my list increased to the point where this happened every day or two, and just got annoying.  So now, the notepad is temporary, and when I get back to my desk, it all gets uploaded to the wall:

todo_stickies

Although I’ve been using this system for a while, it still feels new enough for me to be in love with it. I love that each item can be moved around independently, so I can keep the list visually prioritised, with urgent items at the top.  I also like the stack of ‘done’ items (which sits on my desk) – it can be satisfying to flick back through them sometimes.

For a while, I used a few different colours to categorise items – but that’s been less useful recently so I’ve stopped (but I still have the multicoloured pack of post-its).

Trickles

The post-its are all about ticking things off.  The goal is to get items off the wall, onto the ‘done’ stack.  But not everything works this way.  It’s not how you get fit.  It’s not how you improve your people skills.  The stickies are tactical and one-off, and I’ve been realising for a while that I need something for strategic and recurring items.  So here’s the newest part of my system:

todo_trickle

It’s taken straight from here, and I’m rather excited about it :)

For now it’s on the whiteboard by my desk, which is nice for visibility, but I’m almost certainly going to move it to paper so I can look back at previous weeks.

Recently we chose Jira as our new bug database, for reasons mostly explained in my previous post (at least, those were my reasons – other people were involved, no doubt with their own reasons). We were on Bugzilla before, which did us proud for many years, but we needed something a little more fully-featured in order to rely on the bug database as a more complete project management system. I don’t think Jira is the only choice that fits those criteria; there are several other good products out there (Fogbugz would have had a much better chance if version 7 had been out when we were deciding).  I don’t want to spend a lot of time talking about Jira specifically – their website does a pretty good job of selling their product already – instead, I wanted to mention how important the plugin system is to us. I’ve touched briefly before on how useful plugin systems are; in Jira’s case, it’s a fantastic system that makes all the difference between a great product that we’d have to bend ourselves slightly to fit, and a product that can do exactly what we want. It’s probably the first time I’ve used a plugin system in anger, and I’ve been delighted.

You can get a good glimpse of how cool it is by looking at their plugin page, and you can gain a lot just by grabbing some of the existing ones, but it’s also very easy to create your own.  Here are some of the customisations we’ve made in a short space of time.

Custom fields

You can define custom fields to do all kinds of interesting things:

  • Calculated fields don’t store any data in the back-end database, but calculate their value (in a Java function) based on all the other fields in the issue.  For example, we created a “reopen count” field that just looks back at the bug’s history and counts how many times it’s been reopened.  Once you’ve created this field, you can search and sort bugs to find problematic issues that have been reopened a lot.  Similarly, we have a “last status change” date that helps you search for bugs that are a in a given state and entered it recently, for example.  We’ve created a field that automatically hyperlinks into some custom web-based in-house QA software.  And we have a calculated field that simply mirrors data from the parent issue (Jira has a parent/child issue hierarchy), allowing you to use the search interface to search for issues based on data in their parent, which it doesn’t ordinarily do.
  • We’ve created some other fields that present different user interfaces to the built-in choices. The built-in “select” field lets you choose “none” as an option (effectively storing NULL in the database) – we wanted certain fields to always have a value, so we’ve just customised the UI to remove the “none” option.  We also created a “combo box” field that supports select-list UI as well as typing your own value (with nice autocomplete).  We just used scriptaculous to do all the hard work :)
  • Perhaps my favourite one is a “trick” implementation of issue locking.  Here’s the backstory: imagine that two developers go to edit the same bug, around the same time.  One of them changes the priority and submits their change.  The other developer changes the description and submits their change.  We have a conflict!  In Bugzilla, the second developer gets a “mid-air collision” warning, with the choice of overwriting the other person’s change, or backing out of their own change.  Jira doesn’t do this … the second developer quietly overwrites the first, which isn’t ideal.  We managed to recreate Bugzilla’s behaviour by creating a clever custom field: it calculates its value to be the “last-updated” timestamp of the issue, stores this value as a hidden element on the HTML form, so the user can’t edit it but it gets uploaded when they submit, and we then override the field validation hook to cause the user’s submission to fail (with a “mid-air collision” error message) if the submitted value is older than the issue’s current “last-updated” timestamp.  Phew!

Fake searching with reports

Jira’s built-in search functionality didn’t do as good a job as we’d have liked.  As it happens, they’re fixing all of that in 4.0, but in the meantime, we’re making do with a couple of reports.  The plugin system lets you write your own reports, so we just added reports that present a search interface for the “report configuration” step, perform the search and just show the resulting issue list as the report output.  It’s not as good as using the built-in search system, because you miss out on standard functions of searches (the ability to save and share them, subscribe to them, and the ability to perform a bulk edit on the resulting list of issues), but it does perform the primary goal of a search: to find a set of issues!  The specific search functions we added this way are:

  • Specifying a list of issue keys – this is something that Bugzilla always supported (separated by spaces), and that people relied on, so we just added it.  To make it even easier to access, we created a portlet so people could access this directly from their dashboard.
  • Specifying a boolean combination of searches – again, something that Bugzilla supported.  In our report, you have to create saved searches as the predicates, and then you can type a freeform boolean expression of these predicates.  The report reuses the underlying search system to find the issues efficiently, and then performs the boolean on those results (term negation leads to a fun little programming problem that’s worth thinking about if you’re curious … given that the scope of a negation on its own is unclear, with a Jira instance consisting of multiple projects, components and so on).

Workflow customisations

Jira uses a workflow system for modelling the lifecycle of an issue.  That’s a bit of an abstract mouthful, but it basically means that you have a little state machine to control the status of an issue, and the possible transitions between different status values.  I’m not a huge fan of over-using this feature to create wacky state machines (the standard, built-in workflow has had some good thought behind it), but there are some useful hooks to customise on the transitions.  On each workflow transition, you have 3 possible points of customisation:

  • A workflow condition, which is a snippet of Java code that evaluates whether the transition should even be presented as an option to the user.  I don’t think we’ve actually used this much – its main purpose seems to be for permissions, and the conditions that ship with Jira fill that need completely.
  • A workflow validator, which checks that any user-entered data (a transition can present a user interface to ask for certain data to be entered) is valid.  We’ve added a couple here – requiring the comment field to be non-empty, for example, or checking that the text entered in a text box is in fact a valid issue key.
  • A workflow function, which is a snippet of Java code that fires after the workflow transition has been executed.  We’ve created a function that sets a given field to the current user (effectively allowing us to keep track of the user who performed a particular action last, and hence search based on this), and a bunch of functions to auto-assign a bug based on some special rules of our own (Jira’s built-in assignment is purely component-based but we sometimes want to do some other things, like assign certain bugs based on the branch they’re reported in).

Management reporting

Reporting on the data in Jira is a pretty obvious use of the plugin system.  We’ve got a load of different reports, and no doubt many more to come, that slice and dice this data and present high-level summaries, such as a grid breaking down work for each sub-team, for each upcoming version, together with their projected finish date, critical path team member, and a hyperlink (using the search system) to the list of work they have left.  It even takes into account projected future hires for the team.  I think we’ll have a lot more of this kind of stuff to add … for example, I’ve been thinking about whether a simple version of EBS could be possible :)

Interactive planning

Last time I mentioned the idea of a “swim-lane” chart to show a team’s schedule.  We now have this in Jira, with each block being a draggable item: drag within a developer’s row to re-order their work, and drag between developers to reassign work.  This was actually really easy to do – ended up using the Yahoo UI library to handle all the dirty details of drag-and-drop in the browser, and communicating the edits with Jira was really straightforward.

swimlane

For me, this is all an incredibly good illustration of the power of a decent plugin system.  Interestingly, using the plugin system has also revealed how it can never do everything you want – there’s always a plugin customisation point where you just want one extra parameter the author didn’t think to provide, or a place in the application that they didn’t let you hook into.  In Jira’s case, you’re ok as you get the source code – and we’ve managed to do some other cool things that way – but using the plugin system is always the first choice, because it’s so much lower-maintenance when they release updates.

Last time I said I’d talk a bit about my preferred tools for software project management.  Right now, I’m a huge believer in just using the bug database.  Pick a good one, a web-based one, and I can’t see any better tools out there today.

It has to be on the web

Firstly, I believe firmly in using a web-based system. Think about some of their key characteristics:

  • Collaboration. Web-based software inherently means simultaneous concurrent access; no longer does your schedule need to be checked into source control and locked by one person at a time.
  • Distributed.  I believe in letting the most detailed level of scheduling being done by the people closest to the work; overly-centralised scheduling will not do for me.
  • Integration. Think about all the other tools you use: IM, email, wiki, … web-based software integrates smoothly, all through the beauty of the hyperlink.
  • Accessibility. Very few developers will want to install, let alone use, dedicated desktop software for project management. But they’ll have a browser open all day long; opening an extra tab is nothing, the web UI is familiar, and hyperlinks draw them into the system effortlessly.  A web-based system is the only way you’re going to involve and engage your developers in the schedule.

Of course, merely being on the web is not enough. You wouldn’t manage your project on Facebook!

Mixing bugs and tasks

The first games I worked on were split into 2 phases: pure feature development followed by pure bug fixing, with dedicated tools (MS Project followed by a bug database) for each phase. I think a lot of games must have been made this way.  It’s always bothered me:

  • It’s not good from an engineering standpoint; even though rule 5 of the Joel Test is not to be taken literally, the spirit is right: deferring bug fixing to the end is inefficient, demoralising, risky, and detrimental to eventual quality.
  • It’s not good from a management standpoint, because even projects following this model inevitably do bug fixing throughout – and if they’re managed with Project, this isn’t reflected in the tools being used. These projects are either in denial about the reality of what’s going on, or they’re duck-taping the two systems together, say by having a fixed percentage allocation of time to bug fixing in Project. Either way, the tool is not supporting what’s actually happening in the project.

What is a bug anyway? There are plenty of clear-cut cases, but also a big grey area between bug, improvement, usability enhancement, and new feature. Why try to classify them? Who cares? To me, what matters most is that I have a list of things to do, I have an idea of how long they’ll take, I have an idea of how important they are to the product, and I work in the most sensible order based on that information – in other words, I prioritise. If a new feature is higher priority than fixing a bug, I should do that next. If the bug is more important, I should do that next.

Sweets

I think categorising work as bug or feature is only useful to the extent that it helps you quickly sort and slice the data when prioritising a lot of stuff. Once you mix bug work with new feature work, and stop caring too much about neatly classifying work as one or the other, you really need a tool that provides a unified schedule, allowing you to prioritise all your work in one list.

If this sounds a lot like Agile, that’s because it is. Despite my issues, this is something I totally agree with: managing projects ultimately boils down to managing lists of stuff to do. Bug trackers have emerged as fantastic tools for managing lists of bugs; it turns out they’re just fantastic for managing lists of work, and it doesn’t much matter whether that work is bugs or not.

Maturity

Bug databases have known all of this for a while. Jira doesn’t just use the “bug” word: it’s also an “issue tracker … from software bugs and help-desk tickets to project tasks and change requests”. Fogbugz “makes it simple to track your projects”, with Scrum support and innovative scheduling tools. Bugzilla describes itself as “server software designed to help you manage software development”. And so on.

Bug databases have a bit of a bad image for some – it’s not long since they were smelly VB/MFC thin wrappers around a crappy database crammed with boring fields, a horrible UI, and forever associated in developers’ minds with an unpleasant end-of-project crunch. But they were born to be on the web, and have really excelled as a market segment since web applications took off. Look at the world’s best bug databases today and you see incredibly high-quality, good-value, powerful, pleasant-to-use software.

Does it work?

Personally, I’ve done this on a complete project once – on our in-house tools team over about 4 years with 5-10 developers. Right now, I’m helping to put it into practice on a much bigger scale on APB with its 100+ developers – so I’ll have many more data points, and plenty of interesting experience to talk about, by some time next year :)

When I first started managing a small software team, I was expected to use MS Project and its Gantt charts as the primary means of planning the team’s work.  For a while now, I’ve believed that this is a terrible idea.  Today I’m just going to focus on why I don’t like the Gantt chart itself.

For starters, it’s a gigantic waste of space.  Suppose for the sake of argument that I’m managing a team of 5 engineers, that a typical task (in terms of the planning granularity) takes them about 2-3 days, and I want to look at their next month’s work.  Here’s what I get in Gantt form:

Gantt chart

Gantt chart (maximised laptop window)

It’s an unbelievably bad use of the space on the page, from the point of view of density of information.  Nearly the entire chart is blank, information-less space.  If the project grows much at all, this turns into a crippling problem, where you can’t see what’s going on without lots of scrolling.  It seems obvious that the same information would be better displayed with a row per person, not per task:

Same information, less space

Same information, in a tiny fraction of the space

Now you can see a fair bit of work at a glance.  You can fit at least 10 people on the screen in this view, and you can make full use of the time axis for each person – unlike the Gantt chart, where looking further into the future uses up more vertical space and reduces the number of people that can be visualised.  I was initially very disappointed that Project didn’t support this view and allow reordering and reassigning tasks with simple drag and drop operations (maybe it’s because of it’s super-general resource data model which means it can’t just view a resource as a person that does a list of tasks in order?)

The second problem with the Gantt chart is more subtle, and applies to the “swim-lane” chart too.  The question is whether plotting the underlying information is a good thing to do in the first place.  A couple of comments on this (incredibly long) Edward Tufte forum thread are worth pulling out.

Firstly, the misleading sense of accuracy they give:

One thing that always fascinates me about these process diagrams is that they reflect a rather poignant human desire to predict the future, to capture the chaos of the unknown and render it in a coherent, logical framework as way to assuage fear. In this way they are almost more talismanic than practical. Unlike statistical charts, they try to project what might be as opposed to what has already transpired.

[Daniel Spock]

Looking at a schedule, it’s tempting to be tricked into thinking that the future will play out according to this nice plan.  The Gantt chart does not begin to show the consequences of a task slipping, work being completed out of order, an employee being off sick, or new requirements being added.  All of these things will happen on a normal project.

Secondly, there’s the unhelpful notion that it’s useful for one central person to map out exactly what everyone does, in what order:

Many problems in project management (PM), including PM graphics, are the result of blending — and thereby confusing — planned outcomes with planned actions. Planned outcomes are the desired ends, including interim results expected along the way. The purpose of defining planned outcomes is to provide direction, stability and control. Planned actions are far more chaotic and more detailed, but they are the only way to provide forward motion. Planned outcomes are the best way to describe and organize project scope and they can remain relatively unchanged. Outcomes are the rudder and actions are the oars. Blending outcomes and actions is an error that causes miscommunication — the most common cause of project failure.

[Garry L. Booker]

It’s very hard for a project manager to plan out all the actions involved.  Personally, I think they’re much better off focussing on outcomes, on higher-level views, and leaving the low-level stuff to be done in an organic, distributed, bottom-up fashion.  There’s simply no way one person can know all the details of the project and their interactions.  That’s why as a programmer I always used to hate those conversations with my producer where he was “updating my schedule”:

Producer: “How much time have you spent on ‘AI behaviour second pass’?”
Me: “Could you be more specific about what that means?”
Producer: “It’s what’s written in your schedule!”
Me (thinking): “It’s only there because you mangle, mis-spell, and randomly re-interpret what I tell you I’ll do, which is itself simplified and dumbed-down for your sake”
Me (aloud): “I’ve been adding a representation of an NPC’s perception of world state.  It means the AI can be fooled, surprised and disappointed.”
Producer: “Shall I just say the second pass is complete?”
Me (thinking): “Whatever keeps you happy”
Me (aloud): “Why don’t we just delete it and create some tasks that describe what I’m actually working on?”
Producer: “We can’t delete it, it’s part of the milestone!”

Aaargh!

Next time, I’ll get more into talking about some things I like.

Question: when hiring engineers, do you look for positive reasons to hire someone (they’ve done something good, have a good capability, a particular strength or good potential), or do you look for reasons not to hire someone (not enough experience, no degree, failed a question you asked)?

On some logical level, it seems a meaningless distinction.  People are either good enough for you or not, and whether you find the good ones directly, or implicitly by eliminating the bad ones, wouldn’t seem to matter.  In practice, however, there’s a huge difference, and something I believe strongly:

While positive reasons are important, elimination is the primary basis of good hiring

Why?

#1 Comparisons

Positive reasons to hire someone tend to be unique to the individual.  Evidence of high performance in their past is clearly very specific to them.  When you look at people’s strengths, they have unique capabilities or combinations thereof.

This matters because evaluating someone accurately in a short space of time is hard, and one of the most powerful tools at your disposal is relative evaluation.  Choosing the best of 4 candidates allows you to be far more confident than deciding whether an individual is good enough in isolation.  Performing relative comparisons is best done using standard criteria, and once you have those it becomes easy to eliminate based on them.

#2 Efficiency

Looking for positive reasons to hire someone takes more time.  To discover how good a contribution someone made to a past project, you need to spend a lot of time digging (asking them questions, chasing up colleagues, or looking at the code if it was open source).

Fight against time

Evaluating submitted demos takes time most game studios don’t have.  Asking open-ended questions that give people a chance to excel, and judging their performance on these questions, takes much longer than setting them problems with right/wrong answers.  Attractive places to work get overwhelmed with applicants, and hiring managers have lots of other work to do!

An efficient elimination-based approach involves starting with a large pool of applicants and finding the simplest possible relevant test that can eliminate a large proportion of them.  Repeat until left with a small number.  At each stage, you can afford to spend a bit more time evaluating the candidates.  Eventually, you get down to few enough that you can afford to spend real time to look at their positive qualities, and your tests have heavily reduced the chance of them wasting your time.

#3 Bad hires are disastrous

Hiring the wrong person is like bringing a cancer or a poison into your team.  It takes a huge amount of time and effort to fix the problem and they drain energy from your existing good employees in the meantime.  Given the choice between two types of hiring mistake: (1) eliminating a candidate that would have been a great hire, and (2) hiring a bad candidate, mistake (1) is far better.  Mistake (1) can be fixed, because there are always more candidates of sufficient quality.  Your project might be delayed, but it’s generally not disastrous.

All of this means that a good interview process has to be very sensitive to possible bad attributes.  You accept that it sometimes eliminates good people, provided it doesn’t let bad people through.  Setting up multiple independent elimination hurdles is a great way to do this.  Let’s say you have a test that a bad person could pass 10% of the time.  That’s no good on its own.  But put 6 independent tests in a row with the same 10% failure rate, and the chance of a bad individual getting through them all is 1 in a million.  That’s more like it (actually, it’s unrealistically high … but you get the idea) :)

Just for the sake of argument, suppose that same test tends to be passed by good candidates 90% of the time.  Their chance of getting through all 6 is then about 50%.  So you will eliminate quite a few perfectly good candidates, but if you start with a few good applicants, you’ll make a hire.

Cheer up

You often see job applicants get disheartened by this. The classic situation is where someone works their heart out to do something great, advertises this boldly on their CV, and can’t understand why they get eliminated for seemingly niggly reasons.  Take this tale for example.

Don’t take elimination personally.  Keep the above reasons in mind to understand why the elimination approach is important to the employer.  It doesn’t necessarily mean you’re not good enough.  There are plenty of jobs to apply for, and most companies won’t have a problem with you re-applying after a while.  If you’re good, you’ll get past those elimination stages soon enough, and then you’ll have a chance to show your strengths :)

Although I don’t write much code any more, I still think of myself as a programmer; I still have programmer tendencies in everyday life, like trying to “optimise” the world around me. When I was in hospital just last week, I was quietly infuriated when they did the scan before the blood test, forcing me to wait for the lab to analyse the blood.  I was going to point out to the nurse how much more efficient it would have been to analyse the blood while I was having the scan; luckily another of my programmer tendencies (avoidance of social interaction) saved her from my grumpiness!

Yes, “we” (programmers) are an odd bunch, and one of the fascinating things about game development is the number of other disciplines that must interact closely with us during development.  This post is for you.

Maybe you’re an artist.  You might be dependent on programmers to make your tools better, allowing you to work more efficiently; perhaps you need your art to look better, or perhaps you need it compressing so you can fit more in.  Never forget: we love you really.  If it wasn’t for you, we’d have “art” like this, and we know it:

programmer_art

Of course, it can be a bit of a love/hate relationship at times.  You drive us bonkers with your wacky organisation of source assets, mixing multiple logical changes in a single source control check-in, or going over your budgets.  I’m sure we inflict just as much frustration in return!

Or maybe you’re a designer of game mechanics – someone who theorises about how the game should work, and requires programmers to implement your concepts.  I think you may have one of the toughest jobs on the team.  Unlike the artists, your specialism isn’t so obviously expert, because everyone’s an armchair designer with an opinion on your work.  But programmers aren’t just everyone.  We’re liable to say exactly what we think of a design in a brutally honest way; we may be upset if there’s something even slightly illogical about it, and from time to time, yes, we may appear to imply that we could do your job better than you.  I feel guilty just writing about it.  Sorry!

Whatever your role on the team, I offer two tips to get the best out of your programmers.  As a small disclaimer, I certainly don’t mean to suggest that it’s your problem to make the relationship better – but I hope you find something of use here.

#1 Give me a problem

Programmers live to solve problems: there’s really nothing we love more.  We don’t just love finding solutions: we love finding the best, most elegant solution.  So there’s nothing we find more infuriating than being handed a solution to a problem and told to go do it – particularly when the request feels inelegant or arduous. We’re going to be immediately suspicious that we’re wasting our energy, that there’s a better way to solve your problem.  If you ask a programmer to go do something and they react with doubt, negativity or a grudging manner, this is often why – and they may not even realise it themselves.  [As a programmer, it's important to become self-aware in this regard so you can respond with "hey, what are you really trying to achieve with this thing you've just asked me to do?" rather than being grumpy]

I’ve seen some artists achieve a certain degree of success by getting closely involved and specifying very precisely how they’d like their tools to work.  That’s fine, but the best artists I’ve known (best in terms of getting what they need from programmers) took the time to explain their job to the programmers: explain the challenges they face, explain what it’s like to be in an artist’s shoes using the software.  They always got better results.

Think of it another way: if you just describe a solution without the problem, even if the solution is a brilliant one, a programmer will often struggle to implement it well.  A thousand little decisions need to be made when implementing something, and without a good understanding of the problem being solved, these decisions aren’t going to be made well, adding up to a sub-par result.

On the other hand, if you explain the problem first, there’s nothing stopping you from giving solution ideas too.  We’ll understand them all the better now they’re in the context of a problem.

#2 Explain why

Sometimes, you simply have to deliver a solution rather than a problem.  It’s not practical to go to a programmer with every single problem you come across.  If you’re designing game mechanics, your entire job is to come up with a solution (the game mechanics) to a problem (a high-level project vision).  Nobody wants to take your job away from you.

When this happens, remember that programmers crave understanding.  We need to know the answer to “why?”  We can be like a five-year-old-child asking it over and over, just as maddeningly persistent and without the cuteness.  Logic rules our world, and to see someone do something illogical is hard for us.  So, take the time to explain why you took some of the big decisions.  Explain the other roads you considered and why you didn’t take them.  It might seem like a lot of effort, it might seem unfair to have to justify your work (after all, we don’t necessarily do a good job of explaining the technical decisions we’re making), but I’m not suggesting you do this simply to keep programmers happy.  You should do this because you’ll better results out of them.  By understanding why, we can implement your intentions, capture the spirit of your vision – not just the letter of a specification document.  We might disagree with your decisions sometimes, but we’ll respect the careful thought you put into them.

microscope_slides

I’ve sometimes seen designers write out their designs in minute detail in an attempt to improve how closely the implementation matches their vision.  This is generally a big mistake.  If you use design documents as part of your process, the best thing you can do to improve them is to explain “why”.  Going one step further though, a design document isn’t a great format for explaining the process of reaching the design – so the best thing you can really do is to collaborate closely, involve other disciplines throughout, and iterate.

Universals?

I set out to write a piece about programmers, but I actually think these two tips apply generally to any human being.  You don’t go into the doctor’s and tell them what medicine you need; you tell them your symptoms.  You don’t do this just to humour the doctor, you do it because they’re experts in the problem domain.

Don’t tell experts what to do; describe the problem you want solving.  Separate your goals from your solution ideas.

And you already know that explaining why is important when asking people to do something for you.  Think of the trickier requests you make of other people in everyday life … asking a favour, demanding a refund … you know instinctively that explaining “why” improves your chances.  A famous experiment by Ellen Langer at Harvard showed that significantly more people let someone skip ahead of them in a photocopier queue when they gave a reason (90% when the word ‘because’ was used in the request, 60% without); the part I don’t like about this is that it can be used as a bit of a trick: the technique worked equally well when the reason given was completely meaningless (“because I need to make some copies”).  That’s certainly not what I was suggesting!  Apart from the fact that a good programmer would probably see through a bogus reason, always go for methods of influence that are genuine, that leave both parties better off, and that build a good relationship.

Explain ‘why’ to give people motivation and a context to what they’re doing.

I just came across this post by Nicholas Lovell in response to Dave’s talk at the GameHorizon conference.  I don’t want to get into debating all of it (obviously I disagree on the “epic fail” bit!!), but item #2 really got me thinking:

My view: different business models require different gameplay. A subscription business needs a very different set of hooks to a microtransaction business or an advertising-funded business. Trying to shoe-horn a business model onto a game not designed for it is very very hard and is likely to cause huge difficulties in the future.

I think this is very much missing the point.  I wouldn’t dispute that “shoe-horning” a business model onto a game not designed for it is hard, but it doesn’t logically follow that you should build your game around a business model.  I’ve no idea what Dave actually said, still less what meaning he intended to convey, but I think there are two key reasons why Dave was right (and this is all just my personal opinion rather than RTW’s).

#1 Basic economics of consumer software

Here are the variables in how much profit you make from software:

  • Development cost
  • Number of users
  • Pricing and payment model – how much each user pays you
  • Operational costs, if your software is a service (online game, website, etc)

It’s a tricky equation because they’re all intertwined (pricing affects the number of users; number of users could affect the price you think you should charge; spending extra development cost could allow you to reduce running costs, increase price or increase users; etc etc).  But, a few simple observations help to clarify:

  • Number of users is incredibly important.  The beauty of consumer software compared to other products is its ability to scale.  It costs you nothing to produce copies of the software, and the number of users has enormous scope for increase: this is the variable with the leverage in the equation, and if you want to be really successful, this is the key.
  • Development cost is less important than you think, because it’s a one-off cost.  I’m not saying it doesn’t matter, but it doesn’t scale.  There’s a limit to what you can do by trying to play with that variable, particularly as any significant reduction is going to hurt quality so much that you fail entirely.
  • Operational costs are far more important, because they do affect your ability to scale.  It would obviously be a mistake to price your software below its running costs unless you had a plan to change that relationship over time.
  • Pricing isn’t something you can tweak to cover your costs.  Correct pricing strategy has nothing to do with your costs; it’s all about value to the consumer. So this factor is critically important, but it’s not a variable in the sense of a number that you can just play with in a spreadsheet to get the profit you want (incidentally, this fact about pricing has nothing to do with software specifically … as you’d expect, Joel Spolsky has a good article and book recommendation here).  In a creative endeavour like an online game, you typically don’t know in advance exactly what the customer’s experience will be like; you don’t know what aspects of your game will be most fun.  So you can’t possibly know what the perceived value will be, and it’s therefore very hard to think up-front about how you charge the consumer.

I would suggest that these four factors imply that your main “business model” concerns early in development should be, in order:

  1. Making a great product that lots of people want to play.  Without this, you stand no chance, whereas with this, you’ll find a way to make money from it.
  2. Operational running costs, as these are the main thing that could prevent a great product from being profitable.

Although I have no idea how Dave and the board approached APB’s business model, I do know for sure that the development team has paid close attention to both of these.

#2 Development (creative and engineering) considerations

Naturally, you may need to build software features to support the business model, but as an engineer, I’d say these features are always likely to be more straightforward (read: lower risk) than the creative/gameplay/fun features.  I would much rather focus on the creative stuff early on the project, when the time is right for heavy iteration and exploration of design space, and then build the business features near the end when you know what you’ve got.  I suspect this is part of what Dave was getting at: he’s a creative person and wants to nail that part of the project above all.  He understands that you can’t just plan a game design up front and implement it from a spec.  You have to iterate, you have to experience what you’re building to guide further decisions.

Finally, whichever route you take, this isn’t a binary decision.  There’s a continuum between the extreme positions of building a game entirely around a business model, and the opposite extreme of building a game with absolutely no thought to the business model.  I would expect either extreme to be a bad idea, and I’d personally interpret Dave’s position as leaning toward one end of the scale, rather than not caring at all about the business model.  The fact that we’ve managed to raise $80m of VC, and the fact that the engineering team have paid close attention to things like bandwidth, suggest that we haven’t been completely blase about the business model during development!

I had a curious little problem recently: in our new bug database, we couldn’t play video attachments within the browser (e.g. with the VLC plugin for Firefox, or Windows Media Player embedded in IE).  You could download the video as a file and play it – fine.  You could launch a media player as an external application and play it – fine.  But embedded in the browser window, it just didn’t work.  It’s also worth pointing out that our old bug database worked fine in all cases, so it didn’t seem to be a problem in the video playback/codec department.

I’ll put the solution at the end – the question you want to ask yourself is, how does a browser plugin behave differently from the browser saving the file to disk?

star

I started out by using Firebug to have a look at what was going on.  If you haven’t tried Firebug, I can’t recommend it highly enough – it allows you to do all kinds of powerful debugging of what’s going on in Firefox: way, way more than I’ll describe here.  For my problem, I just used the ‘Net’ tab – it shows you all the network requests made by the browser, allowing you to inspect both the request and the response, including all the headers.  This is pretty cool but I wasn’t able to spot the problem from this: the data appeared to be transferred correctly … weird!

Oh WordPress, you make so many requests for one page!

Oh WordPress, you make so many requests for one page!

I wondered whether the plugin was doing anything directly over the network and bypassing Firebug, and it was suggested to me that I try Wireshark.  It’s the first time I’ve used it, and it was impressive.  Within a minute or two I was inspecting all the traffic between my PC and the bug database.  It goes down to lower level protocols (you can see the raw network packets) but also provides a nice high-level (HTTP-level) summary.

Not sure what it all means, but it looks cool!

Not sure what it all means, but it looks cool!

Within seconds of looking at the data, I had the solution.  Turns out, the browser plugins don’t send the cookies that the browser does.  Our bug database was configured to require everyone to log in to view data, and without the cookies, it figured you weren’t logged in.  Rather than getting video data back, the plugin was getting an HTML page saying “Access Denied”!  Interestingly, when playing the video with the plugin, the browser firstly requests the data itself (until it gets some data back, it doesn’t even know it’s going to find a video at that URL), and of course gets it successfully.  The plugin then makes a fresh request to get the data itself.  Since Firebug was showing the first request, I foolishly thought I was looking at the data being transferred correctly (and wasted a lot of time looking into slight differences in the response headers, thinking they might cause the plugin to interpret the data incorrectly).

That’s it.  Not a big deal (we quickly reconfigured the bug database so as to allow anyone to view data without logging in), but I rather enjoyed the solution.  Most of all, I thought Firebug and Wireshark were both totally awesome – highly recommended :)

I’ve been working on APB for a little while now.  It didn’t seem worth posting about at first as there wasn’t much to see of the game publicly, but after E3, there’s plenty:

I still feel like a total newbie.  Much like Timothy at the end of the (excellent) first podcast, just “making a bloody awesome boot”, I have plenty to learn about the project.  It’s been a big change, from a project starting out to a project finishing off, from a team I helped build to a team I hardly know, from being involved hands-on with the code to having not opened the IDE once: a world of new challenges.

I don’t suppose it’s often you get to work on something this cool, and I’m very excited to be a part of it.  Here’s to shipping a bloody awesome game :)

We’ve all heard the games industry overtime horror stories.  ea_spouse is still the most famous example but if you work in games, you’ll know more.  I’ve heard of people that were forced to stay late every night because of a “nobody leaves until everybody’s finished” policy – with people challenging the policy losing their bonuses.  I’ve heard of another employer criticising a programmer for not looking tired enough (they started playing WoW at work).  And I’ve worked some fairly brutal weeks myself!

But how much overtime is reasonable to work on a software project?  I suspect it’s a question every software developer must confront at some point in their career (if you haven’t, count yourself lucky!).  Let’s start by caricaturing two opposing points of view I’ve heard:

  1. Overtime should be avoided at all costs.  Nobody looks back at the end of their life and wishes they’d spent more time at work.  You’re much more likely to regret missed time with family and friends, or experiences you missed out on in life.  Overtime is a direct result of poor management – why should the developers pay the price for this?  You get paid your salary for a standard working day – if the project requires more effort than this, the employer should extend the schedule, hire more people or cut features.  Making great software requires top quality work, not just a great quantity – and excessive overtime leads to tiredness, reducing quality – thereby harming the product over the long run.
  2. Greatness is not achieved by working 9-5 and viewing your work as just a way to pay the bills.  Making great art requires sacrifice.  If you don’t work overtime, you’re missing the chance to ship sooner and add polish.  You’re saying “that’ll do” rather than “I gave it my all”.  Adding more people doesn’t work because it’s tough to maintain quality when you hire lots of people, and increasing team size introduces all kinds of new problems (possibly even making you later).

Personally, I can see both points of view.  Part of me is highly driven, loves my work with a passion and measures myself by my success at work.  I have some definite workaholic tendencies as a result.  At the same time, I have a wonderful family that I love spending time with.  Our 3-year-old daughter changes at a frightening rate, and each stage of her development needs to be treasured before it’s gone.

Learning functional programming

Learning functional programming

Nowadays, I definitely lean more towards the “overtime is bad” end of the spectrum.  I got turned off overtime back at VIS.  It took me a while to realise, but we were working some crazy hours with no end to the project in sight.  We suffered from a basic inability of management to balance the scope of the project with the budget.  We were trapped in a classic bad publisher/developer relationship, constantly hanging by a thread, driven to a spineless position where feature creep and random changes in direction were normal.  When we finally realised NARC had slipped way behind schedule, we “renegotiated” a longer timescale that came with a ton of new work.  It wasn’t more time to put us back on schedule, it was more time so we could catch up ourselves by working harder (and an on-site producer to check up on us).  The real killer, of course, was that NARC was clearly destined to be crap (and I do say that with love).  Once I realised how futile the overtime was, I started looking for something else.

Moving to RTW

Moving to Realtime Worlds was a hugely pleasant surprise after that background, as they’re definitely an enlightened employer in this respect.  I no longer work overtime as a result of blatantly unrealistic scope or underfunding.  I do still work overtime, generally a steady-but-low amount – and always out of choice, to turn something good into something great, to satisfy my own pride.

We manage our projects pretty carefully, we have sufficient funding to achieve our goals and no publisher applying arbitrary external pressure.  We value the wellbeing of our employees and respect their own preferences on overtime.  In other words, we’ve eliminated all the wrong reasons for doing overtime.  That leaves us with an interesting question: what should you do then?  What place does overtime have in our projects?  What does it look like when it’s done right?  If it’s useful, what are the right reasons for doing it?

Here’s how we do it right now:

  • Overtime is voluntary.  This allows people to make their own choice, depending on what they value more (achievement at work versus life outside work).  It’s easier said than done, especially when you take into account things like peer pressure that can quickly slip out of your control.
  • Overtime is paid for.  I wasn’t fully sure about this at first; I thought we might be in danger of displacing intrinsic motivation.  That may still turn out to be the case; it’s too early to tell.  But I like the positive intent: it says “we value our employees’ time”, loud and clear.  I’ve noticed that it looks attractive when we’re recruiting.
  • Overtime is valuable in moderate, sustainable amounts.  For one thing, it’s incredibly hard to hire top quality developers – so a bit of overtime from your existing team can make a lot of sense.  Secondly, the resourcing demands of a project aren’t constant across time, and standard hiring mechanisms are too slow to react to short-term adjustments (plus, reducing headcount is best avoided whenever possible, for obvious reasons).  Finally, strategic use of overtime can unlock rewards out of proportion to the effort invested – for example when one programmer stays late to make a critical pipeline improvement for the art team to use the next day.
  • Extended, excessive periods of overtime are harmful, and we can make world-class products without them.

We’re definitely still finding our way; this is very much an emerging view that changes over time (hopefully always for the better!)

Morning mist

The finishing line

The main question I’ve been struggling with recently is whether overtime really needs to be weighted more towards the end of the project.  This weighting seems to be pretty standard, but does it have to be that way?  My first thought is, why should it be more valuable at the end?  Couldn’t it be equally valuable early on, where discovery of new techniques, invention of new gameplay concepts, and large boosts to artist productivity (by tool and pipeline improvements) are most likely?  And isn’t it best (as an employee) to spread your overtime relatively evenly across a project, ensuring a sustainable level and avoiding all-out crunch?  Sure, it’s psychologically easier to accept overtime when the finishing line is in sight – particularly when your product has a chance to be great – but it still doesn’t seem good for your personal life, particularly when “the end of the project” can last quite a while!  And whether it’s good or bad, the curiosity inside me wants to understand why things are the way they are.

The main reason I can see is that the end of a project is your only real deadline.  Every other deadline is non-final and has wiggle room.  You can easily choose to polish later, cut features, hire extra developers or delay the ship date when you have trouble meeting an intermediate deadline.  But the end is different:

  1. Once you’ve arranged a ship date with distribution and marketing, it becomes outrageously expensive to move it.
  2. Once you’ve placed expectations of feature set with the press and the public, it can really harm your product’s public image (and hence sales) to cut things.
  3. Simply being “final” means there are no opportunities to make adjustments and change the course of the project.

These reasons seem compelling to me.  Here are a couple of other reasons I’ve heard:

  • Once you prepare to release your game for real, your quality standard goes up; very few people are able to retain that kind of quality bar early on.
  • The amount of work required to truly reach “done” is always more than you think.

I don’t like these two.  These two, in principle, come under the “bad management” category for me.  Granted, they aren’t the most outrageous examples of bad management; they’re more like the difference between decent and outstanding management.  But ultimately, ensuring that there isn’t a huge pile of unexpected work at the end of the project is a management responsibility and not an excuse for overtime.

It all depends

Views on overtime are highly context-sensitive.  It depends on the project (I’d work overtime far more readily on a project that has a chance to be great), the employer (particularly whether they’re a new start-up or an established place with a revenue stream) and most of all the employee.  My own views are constantly developing.

How do you view overtime?  If you’ve worked somewhere that’s “done it right”, do tell :)

Older Posts »