Game Development in a Post-Agile World
Of course, there were always the more levelheaded developers, unswayed by the silky promises of besuited "consultants" - even the ones offering certificates and everything. Today these more progressive thinkers talk of "post-agile" - or common sense as it were previously known.
But ripples of reason spread slowly. If I have to sit through another meeting with some little "agile" toe-rag defending their train wreck of a project then I may end up forcibly ramming a kanban where the scrum does not shine. It is not that I have anything particularly against agile, quite the contrary, but at the end of the day I have a product to ship and no patience for the quixotic.
Project management experiences are hit and miss in an industry that is just growing out of adolescence. To engage with these misconceptions we need to take a step back and put agile into context before taking a holistic look at what contemporary game development might mean.
As the Technical Director of a large studio, I oversee our own projects and interact regularly with publishers and other developers. There is plenty of opportunity to observe different management styles in the wild. Agile has certainly gained in popularity in recent years but I have also observed a few trends that go with it.
Firstly, people use the words "agile" and "scrum" interchangeably. The idea that agile and scrum are separate things appears to be an alien concept. Yet despite the prevalence of scrum as the methodology of choice, there is a decided lack of scrum-like language or attitude amongst many of the teams. When pressed on what people actually do on a day-by-day basis it seems that "scrum" is wide and varied. Sometimes I struggle to find any correlation between what the teams are doing and what I understand it to be.
Another common opinion is that the alternative to agile is waterfall. Many people talk in these terms. Therefore, if you are not an agile devotee then you must be stuck in the past, desperate to cling to the familiar and the conventional. Clearly, you need enlightening or worse, you just do not "get it".
We have all been on the receiving end of bad management at some point so it is easy to empathise with developers yearning for a more natural process and agile promises a seductive alternative to the traditional wisdom. However, whilst developers complain about management stupidity, the leadership is often frustrated by the naïve views from the rank and file who fail to appreciate the reality of business.
There is general misunderstanding of what agile really means for a company, what the real issues are and how it all fits into the bigger global picture. Ignorance impairs constructive debate and I often find myself spending a lot of time clearing unhelpful philosophical hurdles before we can get down to tackling the actual problems in front of us. This, frankly, is annoying and the driving force behind this little rant.
Many people think agile is a new idea and these are contemporary issues. Actually, project management has been around for centuries and as applied to software then certainly for a few decades. If you consider agile has its roots in lean manufacturing then "agile thinking" has been around at least since the 1940’s (although I suspect the pyramid builders and artisans had similar debates). Over the years, a large body of knowledge, empirical data and case studies has built up so we can make some qualified statements.
However, if we confine ourselves to software development we can see there are in fact quite a few more formal methodologies than just Scrum or Waterfall. Here is a handful:
Of the items above, some are generally considered agile whilst others not so much. All of them have their advocates and used successfully on many projects and this raises a number of interesting questions.
If they are all successful then why does it matter which we use? Are some more effective than others and if so then why? Are the agile techniques more effective than the non-agile ones? If there is more than one approach to "agile" then why is scrum so popular? Is it truly "best-of-breed" or does it just have better PR?
We would not be the first people to ask questions like these or attempt to measure the effectiveness of different project management techniques. For example, the American Department of Defence developed the Capability Maturity Model as a framework for assessing the project management facility of potential military contractors.
However, as a basic thought exercise I want to start with a much simpler proposition.
Of the methodologies listed above, which are more agile and which are less?
A simple question and to answer this we need a formal definition of "agile". Fortunately, the Agile Manifesto provides us with one. I like the manifesto as it shows a clear understanding of software development and a well thought out statement of values. It contains valuable insights that one should not dismiss.
Consider the first proclamation, "we value individuals and interactions over processes and tools". The Agile signatories are making a bold statement (and moreover it is their first statement) in valuing personal interaction over the process-orientated aspects of development.
This gives us a more clear-cut dimension for differentiability. We can consider techniques that focus on people orientated activities as "more agile" and methods that focus on process as "less agile". Of course, this is only one axis of comparison, a crude taxonomy perhaps but a useful and tangible metric to take for a spin.
You might be wondering what we mean by the slight abstract terms "people orientated" and "process orientated".
The idea of "process" is politically charged and one of the reasons people can have strong emotional responses to project management "methodologies". This idea taps into more fundamental and nebulous concepts.
Process is an instrument of authority. A process is a statement of what to do, how we should do it and in what order. Process is conservative, hierarchical and formal. Process upholds the status quo. It is management imposing order and control from the top down. If process were a political party, it would be in the right Hegelian camp (and incidentally the right hand side of our line).
The alternative to process is to delegate responsibility, to trust individuals to organise themselves more effectively because they are better able to understand the task and therefore be more productive. Self-organisation is liberal, communal, informal, a meritocracy. It is emergent, bottom-up, grassroots. If we were to abuse the political metaphor even further, it would be some kind of hemp-wearing longhaired hippy socialist.
This is I feel very much cathedral and bazaar territory. To come back to something more tangible, we could consider "process" as belonging to traditional software engineering and self-organisation as belonging to software artisanship. Something Paul Graham so effortlessly explored in his essay Hackers and Painters.
With all these complex and fuzzy memes floating just beyond the grasp of our consciousness, it is perhaps no surprise that discourse often becomes tenuous and emotional. This milieu of uncertainty is a fertile breeding ground for religions like Agile and the peddlers that profit from it. Ask yourself why there is such a thing as a manifesto in the first place. It is unabashed political positioning rather than an objective view.
I am not a partisan of either camp. I am only interested in results and do not have the time or inclination to advocate a particular point of view. Getting effective and timely results is often about being flexible (although not always I might add) so it is worth understanding what you get from each.
So what is so great about process? Well, it gives us:
- Repeatable and predictable results
- Quality Assurances (through the above)
- Cost savings through the ability to optimise work flows
- Defined work flow allows us to use cheaper labour
- The promotion of best practices and conceptual integrity
- The ability to scale to large numbers
- A means to effectively track our progress against the objectives
McDonalds is a great example of successful process. No matter where you are in the world, you know what you are going to get and you get it quickly and cheaply. This process has successfully scaled to thousands of restaurants. Whether you consider this good or bad it is hard to argue with the results.
Nevertheless, software development is much harder than frying beef burgers. Process is sometimes inappropriate or unconstructive.
- Increase cost and reduce performance through bureaucratic overhead and waste
- Hinder our ability to change or adapt to new situations
- Stifle our capacity for innovation and creativity
- Require discipline and training which takes effort
- Only effectively be applied to known quantities
Corollary, it follows that the opposite holds true for more people orientated approaches.
Consider what would happen if we went to the extreme ends of the line. Too much process and the project would grind to a halt, stifled by bureaucratic red-tape, protocol and form filling (much like government). Alternatively, no process at all and the project would be a chaotic mess of uncoordinated activity with lots of energy spent thrashing but generating very little momentum to move us forward (um - ok, much like government).
At the extremes, we always fail but of course, in practice nothing is "all process" or "all people". The trick is to define process where we need it and relax the requirement when it gets in the way. Good management is flexibility, identifying what is important and understanding when to apply a different approach. Many projects get into trouble when they try to take a technique that worked very well in one situation and apply it wholesale in a completely different scenario without understanding what made it a success in the first place.
All projects have different pressures, different criteria to meet and desiderata to balance. When I board a 747 for a transatlantic flight, it is pretty important to me that the plane will get to its destination in one piece and not smack into the middle of the ocean somewhere because some muppet forgot a semicolon. It is not so easy to restart a plane after it crashes to "see if it does it again". Safety critical systems demand assurances they will function correctly within well-defined parameters when we run them. This is more important than the time or cost of development and process gives us those guarantees.
On the other hand, updating an e-commerce website with the latest promotion or the CEOs colour de jour requires the ability to react quickly. An agile approach gives us that flexibility.
Even across very similar projects, the goals can be different. In game development for example, hitting the shelves for Christmas and keeping the costs under control may be crucial for a project. Whereas establishing a new IP then quality may be the overriding factor.
That is probably more than enough exposition, it is time to arrange our methods in order of how much process or people orientated they are.
Now you might be wondering how I decided if one particular method was more or less process orientated than another. You may well disagree with my positioning on the scale and you are probably right.
The problem is that every method contains a mixture of different (and hopefully complementary) activities. In addition, the degree of "process" can change depending on the actual implementation.
Consider scrum. Amongst other things, we can define Scrum as
- Scrum Roles
- Sprint Planning Meeting
- Sprint Retrospective
- Sprint Review Meeting
- User Stories
- Daily Scrums
- Estimated Backlog
- Burn Down Charts
- Release Planning
- Stakeholder Collaboration
Teams often also include related activities such as
- Task Boards (kanban)
- Test Driven Design (TDD)
Clearly, some of these activities are "people orientated", such as daily scrums. However, TDD or Burn Down Charts are in my opinion process.
Microsoft has published empirical data that shows that the process overhead for TDD increases the development effort by 15% - 35%. Despite the many positive benefits from TDD, we cannot possible consider anything that adds an extra 35% effort to produce artefacts the customer will never see as lean. Amazingly, people still try and associate this with "agile".
To get a more accurate picture we should consider each methodology as a dynamic range encompassing all activities, which moves according to the actual real-world practice.
The point that I am trying to make is that when you get down to the details, defining what is or is not "agile" is not straightforward or clear-cut and putting activities into a box and saying what we can or cannot use just seems unhelpful to me. Agile was a necessary and important counterpoint to traditional software engineering thinking but only a small part of the overall picture.
I have seen very successful agile teams but many others that were abject failures. Similarly, some teams that have flourished with a traditional waterfall approach and then there were those that have consistently missed deadlines. It seems that success is not entirely dependent upon any one particular management approach and that the problem is more complicated than we might first assume.
There are many things that contribute to the success of a project and not all of them are always in our control. Whilst there are no guarantees, if we at least recognise the dynamics we have a better prospect of increasing our chances of success.
OK, so saying we need to pick the right tools for the job is perhaps not that profound. Nevertheless, I think it is important to say it and there are certainly people who need to hear it.
Once we buy into the idea that agile is just one of many tools, the questions become much more interesting. How do I know which is the most appropriate tool for a given job? What tools are there in the box? What are the essential characteristics of different jobs? "People and process" is a useful construct for illustration of course but only opens the batting on this front.
To answer these questions we need to get down to the specifics, those dirty and inconvenient details. Every game, every team, every company is unique and there are many factors that can affect a project - the size of the team, corporate culture, internal politics, budget limitations, physical constraints and so on.
However, there are certain dynamics and pressures we frequently find and are unique amongst creative projects. There are plenty of books that cover the usual suspects so instead I have picked out five things that I think are either particularly important or peculiar to the game development.
In my opinion, the most important success factor in project management is the people. Talented, motivated and experienced people deliver despite the management. They will find a way around processes that get in the way or add structure where there is chaos.
"Surround yourself with the best people you can find, delegate authority, and don’t interfere", Ronald Reagan
This idea underpins agile development and is fundamental to non-linear management. As this is perhaps, the single biggest factor influencing your success you should put every effort into hiring good people and weeding out the chaff. All this fancy management nonsense is really just icing on the cake.
However to assume you can always hire the best is somewhat naïve. There are some other things to consider.
- If you need to scale up the team size significantly you will face some hiring challenges and the best people will not always be available when you want them.
- Experience is expensive and your budget probably will not support an entire team of elite developers.
- Recognising areas where we can "deskill" without the loss of quality reduces costs which means we can either improve our commercial competitiveness or have more funds to redirect to other areas (you would not staff McDonalds with PhD graduates for example, although it might be interesting to try).
- Too many chiefs can lead to trouble. Who will lead and will experienced people be happy playing second fiddle? Unclear direction and authority will lead to poor conceptual integrity.
In the meantime, you have to work with what you have so understanding a person’s capabilities is a key skill for any manager and different people respond better to different things. Whilst some people like structure, others prefer more freedom for example.
Most agile methods assume that you have a brilliant, experienced and motivated team. Many agile techniques may work well but inexplicitly fail when applied to average, junior or apathetic members of staff.
Experience also plays a factor. For example, I would be prepared to take my best programmer, unleash him on some thorny technical problem, give him everything he needs, sit back and wait for the magic to happen. However, I would be less willing to extend that courtesy to a fresh faced graduate straight out of university. Most juniors need guidance, mentoring and structure until they find their feet.
It follows that a process-orientated approach would work better for juniors and a more people orientated approach may be applicable for senior members of staff. Corollary, we can see that the team composition in terms of experience and personalities will have a bearing in what approach works best.
Of course, people are much more complicated and idiosyncratic for this to be enough. My point is merely that different people respond to different approaches and that experience and skill are factors to consider when wondering if an "agile" or a "planned" approach is better for you.
Creativity vs. Production
There are two fundamental drivers in game development, creativity and production.
Production pressure comes from the business need to deliver a product on time and within budget. To do this we need to be able to quantify the work and risks, and this means planning and scheduling. This is hard work and requires a bit of effort but essential for answering those two key questions - when will we be done and how much will it cost?
For a plan to be precise and accurate, we ideally need to work with known quantities. Production therefore naturally leans towards quantifiable process driven activities.
Creativity on the other hand is the heart and soul of games development and pushes us to create better games. It is a woolly organic process of curiosity and exploration. There are no guaranteed outcomes or convenient timeframes and this is frustrating from a production perspective. This often leads to attempts to manage the chaos that more often than not stifles the creative process and blunts that "je ne sais quoi" quality.
It is not surprising then that some of the most innovative and critically acclaimed games have come from studios whose project management could charitably call disorganized . Similarly, "corporate" studios with mature project management structures tend to produce more derivative products. It seems these forces are mutually incompatible.
However, we have to be careful about what conclusions we draw from this. It does not follow that chaos is a prerequisite for creativity or that if we want to ship a game on time and within budget we must forgo innovation. We need to find a way to balance both harmoniously with the objectives and within the boundaries of each project.
There is a lot of mysticism surrounding "creativity", particularly within the games industry. This is a big topic beyond the scope of this essay. However, people have been studying creativity for many years and there various effective models and techniques we could leverage .
Creativity is a skill, like many others that we can teach, nurture and develop in everyone. It is less about a spark of genius and more about fostering the right environment that stimulates and incubates ideas. If you are interested in further reading I highly recommend the book "Orbiting the Giant Hairball" by Gordon Mackenzie which explores his experiences of being creative within a corporate environment.
Because of the iterative and changing nature of the creative process, it lends itself naturally to a more agile orientated management style. However naively applying scrum in the hope that you will magically be more creative is somewhat misguided. Flexibility is necessary but it is not sufficient.
Finding a harmonious balance between creativity and production is not easy and not all projects are equal. Whilst I would love to work on creative and innovative titles all the time, this is not always possible. The reality is that the business sensibilities (and by extension, production) are more important for the majority of companies. To be sustainable all businesses need to be profitable. If we want to keep staff employed and continue to make games then we need a profitable business.
For example, missing a milestone payment could be fatal for a small "work for hire" start-up living hand-to-mouth, so production (survival) is more important in this case. Alternatively, a large publisher-backed operation may want to establish a new IP so they can reap the long-term rewards this offers. For this project, they will have more appetite (money) to grease the creative wheels.
Product portfolio is another aspect to consider, a large studio may have one project to "pay the rent" (production focused) and a second to increase the prestige of the studio (creatively focused). Understanding the business context is important in assessing the capacity and ambition for each game and ergo, the best way to run it.
Some people make the argument that commercial success will simply follow from making a good game. It certainly does not hurt and in a perfect meritocratic world, this would be the true. However, we do not live in that world or at least it is not that simplistic. It is true that some studios have carved a niche out as being creative and innovative; this in turn has led to commercial success and investment interest. This is not viable for all studios and the difficulty they will face will be sustaining that momentum, continuing to be creative and innovative beyond one or two projects and surviving in the long-term. I wish them luck. Ultimately, the market will be the final arbiter.
Phases of Production
Another facet that is peculiar to the games industry (boxed products anyway) is the timeline - pre- production, production and post-production . The exact definitions vary between projects so I am going to define them broadly speaking as:
- Pre-production being the period in which we work out what the game is and how we are going to make it
- Production as the period in which we make the game
- Post-prod as the period in which we take the finished game and actually turn it into a good and shippable product. This in turn has two phases - alpha (game play polish, i.e. making it fun and balanced) and beta ("pure" bug fixing).
Interestingly even the most ardent agile proponent I have met still refers to these phases of development despite the obvious similarities with the design, build and test stages of a waterfall model. It appears they are intrinsic to game development at a fundamental level.
Agile methods prefer iteration to following a plan. This works in environments where the requirements are constantly changing and the overhead of rebuilding a plan each time would be too great. In game development, requirements change constantly so naturally an iterative approach would seem the natural fit.
We could for example apply this idea wholesale to the entire project lifespan and dispense with the pre, prod and post phases, which would look something like this.
However, the problem here is the cost of change increases with the more content we produce. Say we decide to change the distance the player can jump, and innocent enough change. This in turn has a knock on effect on the level designs and potentially a number of other systems and bits of content. The more levels we have produced the more expensive that simple change will be.
Above is a traditional cost of change graph you can find in many books on software engineering. The earlier we make a change the less it costs. Applying an iterative approach to the entire development lifecycle exposes us to potentially expensive changes later on.
Ideally, we want to have all the fundamental questions answered early so we do not have to change them later with the associated knock on cost implications. The pre-production period should have precisely that goal.
Pre-production is the most creative period of any game. Early pre-prod is very organic and exploratory; the latter part needs to focus on providing specific answers and technology so that when we enter production we have a good handle on the scope and pipeline. Therefore, if ever there were a time to be agile and iterative then it would be during pre-production.
The production period is where we create the content and in turn where we spend most of the money. We need to keep costs under control and the more process driven and deterministic this is the better. We still need to respond to feedback and change but we need to keep this manageable.
In alpha, we revert back to an iterative cycle as we refine, polish and tweak. Beyond alpha everyone is working from the bug database, which in my opinion makes it process driven albeit of the self-sustaining, organic kind.
We can change our approach throughout a project lifecycle; we do not always have to stick to the same thing throughout.
Another peculiar aspect of the games industry compared to other development environments are the skill sets involved, the three most common disciplines being code, art and design .
The different nature of the work and the complex blend of interactions between disciplines is something we need to consider. Analysing the metrics gleaned from our in-house project management software we can see some common trends within my studio.
Art tends to be a deterministic discipline. Once we have generated a list of assets required and the workflow involved we can predict with quite a high degree of accuracy how long it will take and the end quality we will get. In addition, this work is divisible so adding more people means we get there quicker.
Code tasks on the other hand have very different characteristics. In comparison, the data shows that programmers spend much more time doing unscheduled tasks (anywhere between 20% - 40%) and are more inaccurate with their estimates. This is because they respond to change requests more often, fix bugs or deal with hidden complexity and imprecise designs. Additionally, complex systems are not divisible in the same way so adding more people is not always helpful.
It follows that what works well for artists may not necessarily suit programmers and communication difficulties may surface between the gaps. You could for example run the code team in an agile way but apply a more planned production approach to art, particularly in production.
One interesting aspect of agile is the desire for stakeholder involvement. For simplicity, let us assume that our stakeholders are the publisher stumping up the case for the product. In games we have always had stakeholder participation in the form of an external producer who acts as a spoke person for other interested parties on the publisher side (marketing, legal, QA, etc).
This would seem to tick the boxes but whilst customer collaboration is a noble aim, not all clients are created equal. There are many factors to consider - how much trust and freedom are they willing to extend to you? How much meddling and interference can you expect? How much political clout does your EP have within the rest of the business? Just a few things that contribute to the environment in which the project must survive and understanding the external forces helps you plan your strategy and management style.
On a basic level, if you have a flaky client who is constantly moving the goal posts then an agile approach is your friend. If you have a client who does not care about the outcome too much so long as it is cheap and on time, then good planning will serve you well.
To talk about "agile vs. waterfall" is simplistic. My gripe with agile is not that it is bad or does not have value but that it is all too often used as an excuse by too many people for too many bad project management decisions. Developing software is hard and developing games is particularly tough. We only have to look at the software development landscape to see the rotting corpses of failed projects as evidence of this fact. But then if it were easy, it would not be anywhere near as enjoyable.
Whilst we continue to be blind to the essential complexities, there is fertile ground for charlatans peddling their snake oil with the fancy nomenclature and a "silver bullet" promise. Take a step back to consider your own position and revisit your assumptions. To be successful you need to work continually at project management, not be afraid to try new things, to experiment, to measure, to question but above all remember to have fun as you do. After all, is that not why we wanted to make games in the first place?
No two projects are the same, well - unless you make racing games.