Extensible tools

Part 6 (final part) of “Building World-class Art Tools” from Develop 08

The more you’ve invested in a tool, the more projects you want to use it on, to recover that investment.  This means in turn that it’s harder to anticipate everything the tool must do: each project will be different.  You want to try and meet these different needs, but you can’t afford to recompile different versions of the tool for different games, and you certainly don’t want it to become an ugly amalgam of potentially conflicting ideas.  This is where extensibility mechanisms come in.

If you’re going to build them, they’re certainly easier to build early on, rather than retrofit.  It’s a big decision, though: flexible, extensible code is far more work to write (some say 3 times more), you have to give your extensibility API a lot of thought (bearing in mind that once people use it, they’re dependent on it and changing it becomes harder: just look at the 3DS Max API – they can’t rewrite it because of their rich plugin ecosystem), and you have to worry about new ways in which your users can break your application, even accidentally.

Prince Edward Viaduct

Building an extensible bridge: the Prince Edward Viaduct

There are two main models of extensibility, which I’ll call the Emacs and Eclipse models.  Let’s have a look at each in turn.

Emacs

The “Emacs” model is characterised by having a lightweight, dynamic scripting language – although Emacs uses Lisp, you’d probably choose something that content creators stand a better chance of using, such as Python, Ruby or Lua.  This model has some great advantages:

  • You can have a scripting console and poke around your tool at runtime – in other words, you can actually perform development within your tool.
  • If you expose enough functionality to the script, there are really no limits to what you can do with this approach.  It has maximum flexibility and can be used to create extensions which were never envisaged by the tool’s original developers.
  • It’s typically quite an accessible way to develop: technically-minded artists are capable of doing some clever bits of automation with this approach.  On the downside, these users might require more help and support than programmers.  If your application becomes fragile when used through its scripting API, you could be in for some support headaches!
  • If you have a recording facility for changes made to your model (as I mentioned last time), you can actually have it record in the format of your scripting language.  This allows them to be easily replayed, and if you let users watch the recorded script as it happens, it can teach users how to script the tool.

Eclipse

The “Eclipse” model is a more heavyweight plugin-based approach, generally using a compiled language.  Here are some of the pros and cons:

  • You typically get a more rigorous, structured approach to distributing plugins than scripts.  Versioning and careful release procedures are possible, so you can manage the code the artists are using.
  • Plugins can be limited to extending the application in very specific ways anticipated by the tool’s developer, typically by requiring them to implement a specific interface.  Importers and exporters for new data formats are good examples of how this can be useful.  This doesn’t provide such flexible long-term extensibility, though.
  • Plugin development is somewhat slower and more painful than writing scripts.  You have to recompile code, and you have to unload the plugin itself whenever you change it (unloading can be a tricky area of course, and may be impossible if there are objects from the plugin still alive in memory).
  • Plugins are generally for programmers to write, so they’re not useful for the users of the tool to extend it.
  • By using a compiled language, you get more efficiency, which can be essential if the extension has to do intensive processing on large data sets.

The best thing we ever did with a plugin was to separate the renderer from our editors.  Any team wanting to use our editor has to implement a plugin to do the rendering.  We put this in place for APB, our first title based on UnrealEngine; the APB team provide their own plugin using Unreal rendering so that the artists can apply their Unreal materials and see a preview that’s close to in-game, without making the editor itself dependent on Unreal in any way.  When APB take new code drops from Epic, they can update their plugin without needing to recompile the editor itself.

If you’re really interested in providing a plugin model, you should take a look at Eclipse.  Eclipse is entirely built out of plugins; the application itself is essentially a tiny stub which just looks for a “startup” plugin and runs that.  Plugins can load other plugins in turn, so the entire application is a set of cooperating plugins that provide services to one another.  By copying different plugins into the application’s folder, you can completely change what the application does.  It’s an extreme idea, but the ideas involved, including their “extension point” mechanism, are powerful, and can be used even if you don’t intend to build your entire application this way.  This article gives a good overview.

Solar eclipse

Language support

A couple of posts back, I talked about choosing a better language than C++.  When it comes to implementing extensibility, this is especially important.  Here’s how .Net helps us, specifically:

  • Automatic memory management is a huge win for plugin development.  Of course you can do plugins with native C++, but memory becomes a headache.  If a plugin allocates memory, is it responsible for freeing it later?  Does the application take ownership?  What happens if a plugin developer gets it wrong?  It’s pretty hard for the application to defend against plugins de-stabilising it.
  • Reflection is reasonably useful for plugins too, making it easy for the application to discover things about what the plugin can do.
  • Mono.Addins offers something very similar to the Eclipse plugin mechanisms, in .Net.  There’s now a newer option in the form of System.Addin.
  • IronPython makes integrating Python scripting into your application a doddle, and Ruby is on its way.  You can literally get this working in a few hours in its most basic form, and expose specific bits of your editor’s C# code to the scripting language. The hard part is designing the API to expose.

You don’t have to choose

Choosing “Emacs” and “Eclipse” as representative applications is slightly misleading, because it paints a picture of these as competing models.  It’s almost good versus evil for some people (try Googling “Emacs versus Eclipse”!).  Commercial 3D modelling software proves, of course, that having both is both possible, and useful.  Max and Maya both have plugin models, using native C++ dlls, and their own scripting languages.  If you’ve spent enough time extending these applications, you should have a good feel for the strengths of each approach, and what they could bring to your tool.

This entry was posted in Develop 08, Software development, Tools. Bookmark the permalink.