After the rant, it’s time to be constructive. I was pleasantly surprised last time round by the comments: while I may have vented more strongly than some would have liked, there’s a clear dissatisfaction with capital-A Agile in many quarters, and a definite perception from many that it’s had its day. Over time, management fads do just die away, and people get on with using their valuable parts, without zealotry (there’s a good piece about management fads and their lifecycle here).
Some people gave Agile a good go but are now drifting away from the church of capital-A. For them, its marketing message is losing its effectiveness as the battle against overly-heavyweight processes has mostly been won, and younger developers come through taking that for granted. These people are wondering where to go now.
A couple of people have started using the term “post-Agilism” to describe something along these lines. Jonathan Kohl defines it as
- a growing movement of former Agilists who have moved beyond Agile methods, using a wide variety of software development tools and methodologies in their work.
- an emerging era. Now that the Agile movement has moved to the mainstream, what’s next?
Jason Gorman explains it here. For him, “Post-Agilism is simply doing what works for you”. Personally, I find the term “Post-Agilism” to be a little pretentious-sounding, and I don’t really see why we need more movements or terminology, particularly to define something as mundane as “what works for you” (which presumably, by definition, existed before Agile ever came along!). I’m not convinced that “what’s next?” is a great question, either. Whether or not it was intended that way, it sounds a bit too much like looking for a new trend to follow. Other than that, I can sympathise with these views, and it does at least illustrate that even people who’ve tried Agile aren’t necessarily happy with it any longer.
Some are trying to evolve Agile from within; Brian Marick has written some excellent stuff on how the Agile manifesto has now done its job of changing the relationship between business and developers, and how it’s time to focus on executing software projects better. I can’t argue with that sentiment, and it’s nice to see honesty about Agile being a pure marketing term from one of the people who invented it:
This is spot on. Something I’ve noticed is that your ability to be “agile”, or nimble, often seems to be more a property of your code than your development process. I think a number of developers have found the move to web programming to be refreshingly “agile”. I’ve been doing a few hobby projects recently in Ruby on Rails, and the “write code/save file/hit F5 in browser” cycle is incredibly productive, and conducive to evolving a working application. I love how easily HTML allows you to mock up large parts of the application and gradually add the functionality behind it. Friends who’ve moved to working on smaller games have found a similar refreshing quality that’s in massive contrast to working on hundreds of thousands of lines of C++ code. Try all the processes you like; there is absolutely no way you’re ever going to be “agile” in that world😦
No need for names
Time to talk a little about my own development philosophy. I don’t believe in any “one true way”. I think that projects vary, that it’s essential to understand your circumstances and context. I do have some broad principles, though:
- Hiring smart, pragmatic people (and trusting them to get on with their job) trumps everything else. If you hire sufficiently good people, it literally doesn’t matter what your processes are like (provided you don’t enforce bad processes rigidly), because your staff will adapt them (or overthrow them if they’re bad enough). Conversely, if you don’t hire good people, it doesn’t matter what your processes are like either: you’re looking at various shades of doom.
- Putting strong engineering practices in place is next. Version control, bug tracking, automated builds, unit testing, code reviews – you know what I’m talking about. If you hired smart people, they should do this part for you, although some of them require more discipline, and may require some encouragement. Project management processes aren’t going to save you from poor engineering.
- Be eclectic. If your projects vary, it pays to be familiar with a variety of different development approaches and their pros and cons. Don’t be afraid to try new things. Always look for ways to improve, but don’t change just for the sake of following fashions. Use your common sense and think hard about the way you work.
One commenter offering a similar point of view was Tim Beck from the pliant alliance. I don’t really like the word ‘pliant’ that much – it sounds a bit odd for some reason. Personally, I don’t feel the need to give my approach a name. I’m not interested in marketing it; it’s the approach of common-sense people who are just trying to get better at what they do, and doesn’t need any selling beyond that. Smart, pragmatic people get it without being told. And when I call it “my” development philosophy, I don’t mean it in the sense of taking any credit: it’s been around for as long as software development itself.
I originally became interested in managing software projects back at VIS when I read Steve McConnell’s “Rapid Development”. I asked our producer to order “Code Complete”, and he picked up Rapid Development on one of those Amazon partner book deals. He then left it totally unread on the shelf; being the bookworm that I am, I simply couldn’t bear to leave it there, completely unloved!
I was engrossed. I can still remember reading his list of classic mistakes and nodding along at all the things we were doing wrong. Even looking back at it today, “Rapid Development” still captures my approach to development more or less perfectly. There is no universal “process” here, and there is nothing to be marketed. It reads more like a reference book for smart teams that are just trying to get better at delivering software: a bag of techniques, with pros and cons to help you choose based on your circumstances. In McConnell’s own, timeless words:
In no other area … has there been as much disinformation … Nearly useless development practices have been relentlessly hyped … Other practices are genuinely useful, but they have been hyped so far beyond their real capabilities that they too have contributed to developers’ cynicism … Unlike other books on rapid development, I’m not asking you to put all of your eggs into a single, one-size-fits-all basket. I recognize that different projects have different needs, and that one magic method is usually not enough to solve even one project’s schedule problem.
It’s almost spooky that this was written before the Agile manifesto came along, until you realise that software development never really changes that much. Fred Brooks more or less summed up much of the Agile philosophy (certainly, the iterating-on-working-software part) in “No Silver Bullet“, back in 1986, and that paragraph makes strong reference in turn to a book by Harlan Mills from 1971! The same goes for the rest of McConnell’s book: many of Agile’s key practices are fully described in here, including a much more sophisticated discussion of iterative lifecyles than any Agile text I’ve ever seen.
McConnell is spot on with people: “peopleware issues have more impact on software productivity and software quality than any other factor”. He also emphasises heavily the importance of solid, effective development before trying to introduce choosing processes to trim your schedule. He talks about doing that in two steps: classic mistake avoidance, and applying fundamentals of development. His list of classic mistakes is well worth re-reading every few years. I guarantee you’ll end up nodding and smiling, whether at stuff you used to get wrong and have fixed, or at things you’re still getting wrong. A true classic🙂
Take the case of the half of my team working on in-house tools. We’ve put quite a substantial investment into these and have fairly smooth processes for delivering them. In no particular order:
- We work really, really hard on recruiting the best people we can, developing their skills, and retaining them. The result is a group of people that are just really good at what they do.
- All our work goes into Bugzilla: anything from major features to the smallest of bugs. Unifying all your work into one list (and not treating bugs as a special case) makes sense to me because you always want to do the work that makes most difference to your users: who cares whether it’s classified as a feature or a bug? Making that classification isn’t always possible, either: as luck would have it, Jeff Atwood just explained much of this for me anyway. Bugzilla isn’t the prettiest of things, but it has plenty of powerful features for collaboration and organisation of the list.
- For our internal tools, we make “major releases”, where the main version number changes, and “minor releases”, where the point number changes. Major releases vary a little in frequency, often driven by the biggest piece of work within them, but they’re usually about 1 to 2 months apart. They represent a significant upgrade, with substantial changes (increasing the risk of things being broken, and hence a thorough testing pass). Minor releases are done on demand: they typically contain lower-risk work, or contain fixes for genuine blocker problems. Typically they end up being no more than once a week. We use a release branch to allow minor releases to be made safely alongside development for the next major release. Every release has notes automatically generated by a script from Bugzilla, using the ‘target milestone’ field.
- We have our own dedicated QA. They do lots of testing as well as providing a customer support service: being the experts on the tools, and particularly on workarounds for currently known issues. They also work on user-facing documentation on our wiki.
- Release planning is done with the users of the tools. For major releases, we sit down with them, look through Bugzilla, and pick a set of things to work on, based on their priorities and our own knowledge of technical priorities and dependencies. Minor releases are usually done quite rapidly in response to a direct request for a fix. Major releases are mapped out on a timeline diagram that our users can consult whenever they want.
- We’re not able to sit right next to all the users of our tools, so we work hard to understand their needs: meeting once a week, performing post-mortems after major milestones, demonstrating new releases, and going to sit and watch them work. We get them involved in the early phases of designing new features, and we check up with them after features have been delivered, to see how happy they are with the results.
- We write requirements documents, and we write specifications. Before you can say “heavyweight”, let me point out that they’re not, the way we use them. We use them for specific features rather than mapping out an entire product. We only do them for big features (perhaps 4 weeks or more of work), so that the overhead of writing them isn’t punitive. We keep the documents themselves concise so they’re not too much work (I’d guess a day or two). Often, it’s the process of generating the document that’s valuable, not the document itself. Sitting down to write forces you to think through stuff that you could easily miss if you skim through it in your head. We now write requirements in a 2-column table, where the right hand column must explain why something is required. This helps cultivate a mindset of solving real problems for good reasons, and not doing things simply for their technical interest (or some other bogus reason). It’s surprising how many developers benefit from this discipline. After a while, they get it, and then it becomes a very quick way to communicate simply “this is the problem I’m trying to solve”. As for specifications, Joel Spolsky has a great explanation of why they’re important.
- Individual developers sometimes choose to use “estimate spreadsheets”, similar to what’s described here, for the development of a specific feature (but we don’t maintain an overall schedule that way). It’s entirely optional, with some people taking to it and some not. Where used, it can be an interesting way to track your progress, and to look back and review the quality of your estimates once you’re done. We’re now working on a better web-based system for tracking estimates.
- We hold a weekly team meeting where people can demonstrate what they’ve done in the last week, and discuss the week ahead. A final AOB agenda item often leads to discussing processes or technical issues.
- We generate a “routemap” diagram (in Visio) that shows our medium-term plans in a high-level, visual way. We’ve found this to be a good way to communicate plans at a high level, such as areas and themes of work, delivery of major new features, and how our plans fit alongside the milestone schedules of our internal customers. They’re also important for figuring out how many people we plan to hire in the next year (yes, we live in the real world where the finance team want to know this kind of thing!).
I’m not interested in trying to formalise any of this as a process, or to recommend it to anyone else. There are elements of supposedly-heavyweight processes in here (such as requirements documents and planning ahead), and plenty of elements that will look familiar to Agile people. There are some fairly arbitrary decisions, and some elements that have just evolved that way for historical reasons. I’m not going to make any great claims for it, except that it works reasonably well for us and our current circumstances. I’m well aware that it’s not perfect, and we’re always trying to improve it – specifically, in our newer projects, we’re:
- Trying to do more code reviews
- Introducing rigorous use of unit testing (our old use was sporadic at best, you could say!)
- Switching to C# from C++
What we have is better than what we had before (we got rid of MS Project scheduling a long time ago – but that’s a topic for a whole other post!). I don’t see the process as fixed: we’re constantly evolving and re-evaluating. What I’ve described here is a work in progress, a snapshot in time of a working software team that’s good but not perfect, and always looking to improve.
That’s the way it should be.