Friday, 5 February 2010

Game Development in a Post-Agile World

The hype and dogma of Agile evangelists has left in its wake a trail of broken projects, ruined businesses and misguided neophytes bleating the tired doctrines of their long departed prophets. The games industry was no exception, with many swept up in the phantasmagoria from which we are only now beginning to witness the debris.

Of course, there were always the more levelheaded developers, unswayed by the silky promises of besuited "consultants" - even the ones offering certificates and everything. Today these more progressive thinkers talk of "post-agile" - or common sense as it were previously known.

But ripples of reason spread slowly. If I have to sit through another meeting with some little "agile" toe-rag defending their train wreck of a project then I may end up forcibly ramming a kanban where the scrum does not shine. It is not that I have anything particularly against agile, quite the contrary, but at the end of the day I have a product to ship and no patience for the quixotic.

Project management experiences are hit and miss in an industry that is just growing out of adolescence. To engage with these misconceptions we need to take a step back and put agile into context before taking a holistic look at what contemporary game development might mean.

As the Technical Director of a large studio, I oversee our own projects and interact regularly with publishers and other developers. There is plenty of opportunity to observe different management styles in the wild. Agile has certainly gained in popularity in recent years but I have also observed a few trends that go with it.

Firstly, people use the words "agile" and "scrum" interchangeably. The idea that agile and scrum are separate things appears to be an alien concept. Yet despite the prevalence of scrum as the methodology of choice, there is a decided lack of scrum-like language or attitude amongst many of the teams. When pressed on what people actually do on a day-by-day basis it seems that "scrum" is wide and varied. Sometimes I struggle to find any correlation between what the teams are doing and what I understand it to be.

Another common opinion is that the alternative to agile is waterfall. Many people talk in these terms. Therefore, if you are not an agile devotee then you must be stuck in the past, desperate to cling to the familiar and the conventional. Clearly, you need enlightening or worse, you just do not "get it".

We have all been on the receiving end of bad management at some point so it is easy to empathise with developers yearning for a more natural process and agile promises a seductive alternative to the traditional wisdom. However, whilst developers complain about management stupidity, the leadership is often frustrated by the naïve views from the rank and file who fail to appreciate the reality of business.

There is general misunderstanding of what agile really means for a company, what the real issues are and how it all fits into the bigger global picture. Ignorance impairs constructive debate and I often find myself spending a lot of time clearing unhelpful philosophical hurdles before we can get down to tackling the actual problems in front of us. This, frankly, is annoying and the driving force behind this little rant.

Many people think agile is a new idea and these are contemporary issues. Actually, project management has been around for centuries and as applied to software then certainly for a few decades. If you consider agile has its roots in lean manufacturing then "agile thinking" has been around at least since the 1940’s (although I suspect the pyramid builders and artisans had similar debates). Over the years, a large body of knowledge, empirical data and case studies has built up so we can make some qualified statements.

However, if we confine ourselves to software development we can see there are in fact quite a few more formal methodologies than just Scrum or Waterfall. Here is a handful:

Of the items above, some are generally considered agile whilst others not so much. All of them have their advocates and used successfully on many projects and this raises a number of interesting questions.

If they are all successful then why does it matter which we use? Are some more effective than others and if so then why? Are the agile techniques more effective than the non-agile ones? If there is more than one approach to "agile" then why is scrum so popular? Is it truly "best-of-breed" or does it just have better PR?

We would not be the first people to ask questions like these or attempt to measure the effectiveness of different project management techniques. For example, the American Department of Defence developed the Capability Maturity Model as a framework for assessing the project management facility of potential military contractors.

However, as a basic thought exercise I want to start with a much simpler proposition.

Of the methodologies listed above, which are more agile and which are less?

A simple question and to answer this we need a formal definition of "agile". Fortunately, the Agile Manifesto provides us with one. I like the manifesto as it shows a clear understanding of software development and a well thought out statement of values. It contains valuable insights that one should not dismiss.

Consider the first proclamation, "we value individuals and interactions over processes and tools". The Agile signatories are making a bold statement (and moreover it is their first statement) in valuing personal interaction over the process-orientated aspects of development.

This gives us a more clear-cut dimension for differentiability. We can consider techniques that focus on people orientated activities as "more agile" and methods that focus on process as "less agile". Of course, this is only one axis of comparison, a crude taxonomy perhaps but a useful and tangible metric to take for a spin.

You might be wondering what we mean by the slight abstract terms "people orientated" and "process orientated".

The idea of "process" is politically charged and one of the reasons people can have strong emotional responses to project management "methodologies". This idea taps into more fundamental and nebulous concepts.

Process is an instrument of authority. A process is a statement of what to do, how we should do it and in what order. Process is conservative, hierarchical and formal. Process upholds the status quo. It is management imposing order and control from the top down. If process were a political party, it would be in the right Hegelian camp (and incidentally the right hand side of our line).

The alternative to process is to delegate responsibility, to trust individuals to organise themselves more effectively because they are better able to understand the task and therefore be more productive. Self-organisation is liberal, communal, informal, a meritocracy. It is emergent, bottom-up, grassroots. If we were to abuse the political metaphor even further, it would be some kind of hemp-wearing longhaired hippy socialist.

This is I feel very much cathedral and bazaar territory. To come back to something more tangible, we could consider "process" as belonging to traditional software engineering and self-organisation as belonging to software artisanship. Something Paul Graham so effortlessly explored in his essay Hackers and Painters.

With all these complex and fuzzy memes floating just beyond the grasp of our consciousness, it is perhaps no surprise that discourse often becomes tenuous and emotional. This milieu of uncertainty is a fertile breeding ground for religions like Agile and the peddlers that profit from it. Ask yourself why there is such a thing as a manifesto in the first place. It is unabashed political positioning rather than an objective view.

I am not a partisan of either camp. I am only interested in results and do not have the time or inclination to advocate a particular point of view. Getting effective and timely results is often about being flexible (although not always I might add) so it is worth understanding what you get from each.

So what is so great about process? Well, it gives us:

  • Repeatable and predictable results
  • Quality Assurances (through the above)
  • Cost savings through the ability to optimise work flows
  • Defined work flow allows us to use cheaper labour
  • The promotion of best practices and conceptual integrity
  • The ability to scale to large numbers
  • A means to effectively track our progress against the objectives

McDonalds is a great example of successful process. No matter where you are in the world, you know what you are going to get and you get it quickly and cheaply. This process has successfully scaled to thousands of restaurants. Whether you consider this good or bad it is hard to argue with the results.

Nevertheless, software development is much harder than frying beef burgers. Process is sometimes inappropriate or unconstructive.

Process can:

  • Increase cost and reduce performance through bureaucratic overhead and waste
  • Hinder our ability to change or adapt to new situations
  • Stifle our capacity for innovation and creativity
  • Require discipline and training which takes effort
  • Only effectively be applied to known quantities

Corollary, it follows that the opposite holds true for more people orientated approaches.

Consider what would happen if we went to the extreme ends of the line. Too much process and the project would grind to a halt, stifled by bureaucratic red-tape, protocol and form filling (much like government). Alternatively, no process at all and the project would be a chaotic mess of uncoordinated activity with lots of energy spent thrashing but generating very little momentum to move us forward (um - ok, much like government).

At the extremes, we always fail but of course, in practice nothing is "all process" or "all people". The trick is to define process where we need it and relax the requirement when it gets in the way. Good management is flexibility, identifying what is important and understanding when to apply a different approach. Many projects get into trouble when they try to take a technique that worked very well in one situation and apply it wholesale in a completely different scenario without understanding what made it a success in the first place.

All projects have different pressures, different criteria to meet and desiderata to balance. When I board a 747 for a transatlantic flight, it is pretty important to me that the plane will get to its destination in one piece and not smack into the middle of the ocean somewhere because some muppet forgot a semicolon. It is not so easy to restart a plane after it crashes to "see if it does it again". Safety critical systems demand assurances they will function correctly within well-defined parameters when we run them. This is more important than the time or cost of development and process gives us those guarantees.

On the other hand, updating an e-commerce website with the latest promotion or the CEOs colour de jour requires the ability to react quickly. An agile approach gives us that flexibility.

Even across very similar projects, the goals can be different. In game development for example, hitting the shelves for Christmas and keeping the costs under control may be crucial for a project. Whereas establishing a new IP then quality may be the overriding factor.

That is probably more than enough exposition, it is time to arrange our methods in order of how much process or people orientated they are.

Now you might be wondering how I decided if one particular method was more or less process orientated than another. You may well disagree with my positioning on the scale and you are probably right.

The problem is that every method contains a mixture of different (and hopefully complementary) activities. In addition, the degree of "process" can change depending on the actual implementation.

Consider scrum. Amongst other things, we can define Scrum as

  • Sprints
  • Scrum Roles
  • Sprint Planning Meeting
  • Sprint Retrospective
  • Sprint Review Meeting
  • User Stories
  • Daily Scrums
  • Estimated Backlog
  • Burn Down Charts
  • Release Planning
  • Stakeholder Collaboration

Teams often also include related activities such as

  • Task Boards (kanban)
  • Test Driven Design (TDD)

Clearly, some of these activities are "people orientated", such as daily scrums. However, TDD or Burn Down Charts are in my opinion process.

Microsoft has published empirical data that shows that the process overhead for TDD increases the development effort by 15% - 35%. Despite the many positive benefits from TDD, we cannot possible consider anything that adds an extra 35% effort to produce artefacts the customer will never see as lean. Amazingly, people still try and associate this with "agile".

To get a more accurate picture we should consider each methodology as a dynamic range encompassing all activities, which moves according to the actual real-world practice.

The point that I am trying to make is that when you get down to the details, defining what is or is not "agile" is not straightforward or clear-cut and putting activities into a box and saying what we can or cannot use just seems unhelpful to me. Agile was a necessary and important counterpoint to traditional software engineering thinking but only a small part of the overall picture.

I have seen very successful agile teams but many others that were abject failures. Similarly, some teams that have flourished with a traditional waterfall approach and then there were those that have consistently missed deadlines. It seems that success is not entirely dependent upon any one particular management approach and that the problem is more complicated than we might first assume.

There are many things that contribute to the success of a project and not all of them are always in our control. Whilst there are no guarantees, if we at least recognise the dynamics we have a better prospect of increasing our chances of success.

OK, so saying we need to pick the right tools for the job is perhaps not that profound. Nevertheless, I think it is important to say it and there are certainly people who need to hear it.

Once we buy into the idea that agile is just one of many tools, the questions become much more interesting. How do I know which is the most appropriate tool for a given job? What tools are there in the box? What are the essential characteristics of different jobs? "People and process" is a useful construct for illustration of course but only opens the batting on this front.

To answer these questions we need to get down to the specifics, those dirty and inconvenient details. Every game, every team, every company is unique and there are many factors that can affect a project - the size of the team, corporate culture, internal politics, budget limitations, physical constraints and so on.

However, there are certain dynamics and pressures we frequently find and are unique amongst creative projects. There are plenty of books that cover the usual suspects so instead I have picked out five things that I think are either particularly important or peculiar to the game development.

In my opinion, the most important success factor in project management is the people. Talented, motivated and experienced people deliver despite the management. They will find a way around processes that get in the way or add structure where there is chaos.

"Surround yourself with the best people you can find, delegate authority, and don’t interfere", Ronald Reagan

This idea underpins agile development and is fundamental to non-linear management. As this is perhaps, the single biggest factor influencing your success you should put every effort into hiring good people and weeding out the chaff. All this fancy management nonsense is really just icing on the cake.

However to assume you can always hire the best is somewhat naïve. There are some other things to consider.

  • If you need to scale up the team size significantly you will face some hiring challenges and the best people will not always be available when you want them.
  • Experience is expensive and your budget probably will not support an entire team of elite developers.
  • Recognising areas where we can "deskill" without the loss of quality reduces costs which means we can either improve our commercial competitiveness or have more funds to redirect to other areas (you would not staff McDonalds with PhD graduates for example, although it might be interesting to try).
  • Too many chiefs can lead to trouble. Who will lead and will experienced people be happy playing second fiddle? Unclear direction and authority will lead to poor conceptual integrity.

In the meantime, you have to work with what you have so understanding a person’s capabilities is a key skill for any manager and different people respond better to different things. Whilst some people like structure, others prefer more freedom for example.

Most agile methods assume that you have a brilliant, experienced and motivated team. Many agile techniques may work well but inexplicitly fail when applied to average, junior or apathetic members of staff.

Experience also plays a factor. For example, I would be prepared to take my best programmer, unleash him on some thorny technical problem, give him everything he needs, sit back and wait for the magic to happen. However, I would be less willing to extend that courtesy to a fresh faced graduate straight out of university. Most juniors need guidance, mentoring and structure until they find their feet.

It follows that a process-orientated approach would work better for juniors and a more people orientated approach may be applicable for senior members of staff. Corollary, we can see that the team composition in terms of experience and personalities will have a bearing in what approach works best.

Of course, people are much more complicated and idiosyncratic for this to be enough. My point is merely that different people respond to different approaches and that experience and skill are factors to consider when wondering if an "agile" or a "planned" approach is better for you.

Creativity vs. Production
There are two fundamental drivers in game development, creativity and production.

Production pressure comes from the business need to deliver a product on time and within budget. To do this we need to be able to quantify the work and risks, and this means planning and scheduling. This is hard work and requires a bit of effort but essential for answering those two key questions - when will we be done and how much will it cost?

For a plan to be precise and accurate, we ideally need to work with known quantities. Production therefore naturally leans towards quantifiable process driven activities.

Creativity on the other hand is the heart and soul of games development and pushes us to create better games. It is a woolly organic process of curiosity and exploration. There are no guaranteed outcomes or convenient timeframes and this is frustrating from a production perspective. This often leads to attempts to manage the chaos that more often than not stifles the creative process and blunts that "je ne sais quoi" quality.

It is not surprising then that some of the most innovative and critically acclaimed games have come from studios whose project management could charitably call disorganized . Similarly, "corporate" studios with mature project management structures tend to produce more derivative products. It seems these forces are mutually incompatible.

However, we have to be careful about what conclusions we draw from this. It does not follow that chaos is a prerequisite for creativity or that if we want to ship a game on time and within budget we must forgo innovation. We need to find a way to balance both harmoniously with the objectives and within the boundaries of each project.

There is a lot of mysticism surrounding "creativity", particularly within the games industry. This is a big topic beyond the scope of this essay. However, people have been studying creativity for many years and there various effective models and techniques we could leverage .

Creativity is a skill, like many others that we can teach, nurture and develop in everyone. It is less about a spark of genius and more about fostering the right environment that stimulates and incubates ideas. If you are interested in further reading I highly recommend the book "Orbiting the Giant Hairball" by Gordon Mackenzie which explores his experiences of being creative within a corporate environment.

Because of the iterative and changing nature of the creative process, it lends itself naturally to a more agile orientated management style. However naively applying scrum in the hope that you will magically be more creative is somewhat misguided. Flexibility is necessary but it is not sufficient.

Finding a harmonious balance between creativity and production is not easy and not all projects are equal. Whilst I would love to work on creative and innovative titles all the time, this is not always possible. The reality is that the business sensibilities (and by extension, production) are more important for the majority of companies. To be sustainable all businesses need to be profitable. If we want to keep staff employed and continue to make games then we need a profitable business.

For example, missing a milestone payment could be fatal for a small "work for hire" start-up living hand-to-mouth, so production (survival) is more important in this case. Alternatively, a large publisher-backed operation may want to establish a new IP so they can reap the long-term rewards this offers. For this project, they will have more appetite (money) to grease the creative wheels.

Product portfolio is another aspect to consider, a large studio may have one project to "pay the rent" (production focused) and a second to increase the prestige of the studio (creatively focused). Understanding the business context is important in assessing the capacity and ambition for each game and ergo, the best way to run it.

Some people make the argument that commercial success will simply follow from making a good game. It certainly does not hurt and in a perfect meritocratic world, this would be the true. However, we do not live in that world or at least it is not that simplistic. It is true that some studios have carved a niche out as being creative and innovative; this in turn has led to commercial success and investment interest. This is not viable for all studios and the difficulty they will face will be sustaining that momentum, continuing to be creative and innovative beyond one or two projects and surviving in the long-term. I wish them luck. Ultimately, the market will be the final arbiter.

Phases of Production
Another facet that is peculiar to the games industry (boxed products anyway) is the timeline - pre- production, production and post-production . The exact definitions vary between projects so I am going to define them broadly speaking as:

  • Pre-production being the period in which we work out what the game is and how we are going to make it
  • Production as the period in which we make the game
  • Post-prod as the period in which we take the finished game and actually turn it into a good and shippable product. This in turn has two phases - alpha (game play polish, i.e. making it fun and balanced) and beta ("pure" bug fixing).

Interestingly even the most ardent agile proponent I have met still refers to these phases of development despite the obvious similarities with the design, build and test stages of a waterfall model. It appears they are intrinsic to game development at a fundamental level.

Agile methods prefer iteration to following a plan. This works in environments where the requirements are constantly changing and the overhead of rebuilding a plan each time would be too great. In game development, requirements change constantly so naturally an iterative approach would seem the natural fit.

We could for example apply this idea wholesale to the entire project lifespan and dispense with the pre, prod and post phases, which would look something like this.

However, the problem here is the cost of change increases with the more content we produce. Say we decide to change the distance the player can jump, and innocent enough change. This in turn has a knock on effect on the level designs and potentially a number of other systems and bits of content. The more levels we have produced the more expensive that simple change will be.

Above is a traditional cost of change graph you can find in many books on software engineering. The earlier we make a change the less it costs. Applying an iterative approach to the entire development lifecycle exposes us to potentially expensive changes later on.

Ideally, we want to have all the fundamental questions answered early so we do not have to change them later with the associated knock on cost implications. The pre-production period should have precisely that goal.

Pre-production is the most creative period of any game. Early pre-prod is very organic and exploratory; the latter part needs to focus on providing specific answers and technology so that when we enter production we have a good handle on the scope and pipeline. Therefore, if ever there were a time to be agile and iterative then it would be during pre-production.

The production period is where we create the content and in turn where we spend most of the money. We need to keep costs under control and the more process driven and deterministic this is the better. We still need to respond to feedback and change but we need to keep this manageable.

In alpha, we revert back to an iterative cycle as we refine, polish and tweak. Beyond alpha everyone is working from the bug database, which in my opinion makes it process driven albeit of the self-sustaining, organic kind.

We can change our approach throughout a project lifecycle; we do not always have to stick to the same thing throughout.

Skill Sets
Another peculiar aspect of the games industry compared to other development environments are the skill sets involved, the three most common disciplines being code, art and design .

The different nature of the work and the complex blend of interactions between disciplines is something we need to consider. Analysing the metrics gleaned from our in-house project management software we can see some common trends within my studio.

Art tends to be a deterministic discipline. Once we have generated a list of assets required and the workflow involved we can predict with quite a high degree of accuracy how long it will take and the end quality we will get. In addition, this work is divisible so adding more people means we get there quicker.

Code tasks on the other hand have very different characteristics. In comparison, the data shows that programmers spend much more time doing unscheduled tasks (anywhere between 20% - 40%) and are more inaccurate with their estimates. This is because they respond to change requests more often, fix bugs or deal with hidden complexity and imprecise designs. Additionally, complex systems are not divisible in the same way so adding more people is not always helpful.

It follows that what works well for artists may not necessarily suit programmers and communication difficulties may surface between the gaps. You could for example run the code team in an agile way but apply a more planned production approach to art, particularly in production.

One interesting aspect of agile is the desire for stakeholder involvement. For simplicity, let us assume that our stakeholders are the publisher stumping up the case for the product. In games we have always had stakeholder participation in the form of an external producer who acts as a spoke person for other interested parties on the publisher side (marketing, legal, QA, etc).

This would seem to tick the boxes but whilst customer collaboration is a noble aim, not all clients are created equal. There are many factors to consider - how much trust and freedom are they willing to extend to you? How much meddling and interference can you expect? How much political clout does your EP have within the rest of the business? Just a few things that contribute to the environment in which the project must survive and understanding the external forces helps you plan your strategy and management style.

On a basic level, if you have a flaky client who is constantly moving the goal posts then an agile approach is your friend. If you have a client who does not care about the outcome too much so long as it is cheap and on time, then good planning will serve you well.

To talk about "agile vs. waterfall" is simplistic. My gripe with agile is not that it is bad or does not have value but that it is all too often used as an excuse by too many people for too many bad project management decisions. Developing software is hard and developing games is particularly tough. We only have to look at the software development landscape to see the rotting corpses of failed projects as evidence of this fact. But then if it were easy, it would not be anywhere near as enjoyable.

Whilst we continue to be blind to the essential complexities, there is fertile ground for charlatans peddling their snake oil with the fancy nomenclature and a "silver bullet" promise. Take a step back to consider your own position and revisit your assumptions. To be successful you need to work continually at project management, not be afraid to try new things, to experiment, to measure, to question but above all remember to have fun as you do. After all, is that not why we wanted to make games in the first place?

No two projects are the same, well - unless you make racing games.


Anonymous said...

Great post all around. Thanks for writing it.

Entirely agree -- They're tools, not religions.

Every team and project may have its own distinct answer, and it's up to us to figure out which tool fits the job and the situation best, constantly improving our own ability to manage in the process.

- Scott

Scott H said...

^--- and slightly less anonymous.

- S.

Anonymous said...

Good developers don't need gurus to tell them to work in pairs or be test driven. The most successful software products/projects I've been involved with were managed using no particular methodology, but instead relied on developer motivation, collaboration, communication, and clear requirements from the customer or consuming organization. I guess you could call this a self-managed teams methodology, but what it really comes down to is people being effective at their jobs. In the last ten years or so, I've seen the developer role change to something almost secretarial. Actually, most developers aren't even considered professionals anymore and are dehumanized to the point of being represented as "resources" or simply a cost center. I think this is why some of the best software these days comes from independent developers and the OSS community. Apple also has an interesting approach where design is the most significant phase in development. Skilled developers shouldn't really need complex processes and formal steps to realize a solid design, they just need the time and motivation to complete the work. I've also watched the typical development cycle go from six months to three months, to six weeks, to "we need it yesterday." In my opinion, this unrealistic management of resources is the paramount reason for failure, not the proper usage of Agile methods.

Dances_with_the_beast said...

I don't see how calling your coworkers pigs, your customers chickens, bringing stopwatches to meetings and favoring the vomiting of nonsense over clear communication is considered the most people oriented approach.

rengolin said...

Unlike the Waterfall, Agile gives you the impression that everything is fine until you hit the deadline and have little or nothing to show, even though what you have is "working" and "tested".

People are still attached to their toys and a "good" burn-down chart is motive to give some free pizza to the team, even when the team reduces its own velocity every month to get a good chart.

About an year and a half ago I've written a post about pretty much the same thing and considered writing again, but yours is just perfect, I'll leave yours to it.

Doofusdan said...

I enjoyed the rant, thanks!

One comment though - I followed the link to the MS research paper on TDD, and I think you left out half of the conclusion to balance the 35% increase in development effort. Specifically:

"The results of the case studies
indicate that the pre-release defect density of the four products decreased between 40% and
90% relative to similar projects that did not use the TDD practice."

Speaking as a customer, I can assure you it DOES benefit me to have 40-90% fewer bugs in the software I use; whether or not I 'see' the work that went into it is rather too abstract a question for me.

Speaking as a solution developer, it's worth both my time and my customers' money to get that kind of quality increase. (Unless of course its a project where quality really isn't very important.)

Speaking as someone who deploys systems to production users, I'd also say that it can be (small-l) lean to have TDD, because it allows you to have big savings in configuration management, production release testing, and support.

Anyway, thanks again for an enjoyable and informative rant. :-)

TODD said...

The Agile Manifesto states a preference for being more people-oriented than process-oriented. For all of its "we value X over Y" statements, they quite clearly state you need BOTH X and Y. You should NOT overzealously discard Y whenever you possibly can in favor of X. So, already, your comprehension of the document is poor. There is a point where if you gain too much X at the expense of Y, then the value of the additional X greatly diminishes and Y surpasses it.

Because of the above, People vs. Process is anything but a "tangible metric". It's not even a metric. Period. Therefore, making the assumption that the more People-leaning you are, the more Agile you are is a red herring.

Joe said...

Spot on.

Nice observations that fit exactly with my experiences in various technology industries over the last 25 years.

I've also found the best PMs have spent time in and around the trenches, in a variety of roles, including technical and traditional people management roles.

I'm willing to bet that a great conductor is one who can read, write, *and* play music.

Anonymous said...


Dead-on assessment. Though an apparently well-formed article, there is a clear lack of grasp of the fundamental concepts behind an agile approach.

I think this is the critical problem that most people that believe they have worked in an agile environment face. I'd wager that most failed agile implementations had strong resistance, were either overly focused on the process or hamstrung by no process, and had management that did not understand and was not comfortable with the approach.

The last point is something I have seen over and over again. An agile approach impacts the entire workings of an organization so must be "sold" across the board.

Trasselkalle said...
This comment has been removed by the author.
unwesen said...

Great article, overall!

I have a few comments, though. Starting with the MS research, that raises two important questions not addressed in their paper:

First, how experienced were the developers at TDD? I've seen devs new to TDD develop software that had unit tests for each public function that tested the expected cases only. The whole point of unit tests is to test unexpected cases; the expected ones you go through automatically as you integrate your software. The point? Writing tests such that there are tangible benefits needs experience.

And that ties in neatly to what Doofusdan said: if there are tangible benefits, then some overhead - even 35% overhead - can seem cheap.

The second thing about the MS study is that it doesn't exactly cover the part of software development where TDD provides benefits. The IBM case is especially bad, in that they were essentially rewriting existing code for a different platform, where most of the difficult part - the figuring out how to do stuff - had been done already.

Writing fresh new code like that is the exception rather than the rule in software development; in most cases you're revisiting existing code to improve on it or add new features. And this is where TDD truly shines - make a change, and you'll see what existing tests break. Then consider how to repair that damage without violating any prior or new requirements.

Unit tests are a fantastic tool for measuring and containing the impact of changes to existing code. But in order to get there, you need a lot of them, and those are more easily produced early on in the development process - hence the usefulness of TDD.

To put it into your own terms, if your pre-production phase is test-driven, then production and post-production phases are likely to be shorter.

Now, on to other bits.

This is I feel very much cathedral and bazaar territory. To come back to something more tangible, we could consider "process" as belonging to traditional software engineering and self-organisation as belonging to software artisanship.
You'll find that the truly successful "bazaar"-type projects out there have very strict processes by which they operate. Those processes may not be geared towards producing results on time so much as processes of damage prevention and control, and they might not be put down in writing anywhere, but they're processes nonetheless.

Interestingly even the most ardent agile proponent I have met still refers to these phases of development (...) It appears they are intrinsic to game development at a fundamental level.
Coming from a non-game software development background, I find that those phases exist in all software development. The better users of agile methodologies such as scrum schedule bug fixing sprints, for example.

What I liked about your post best is that while I've experienced scrum used reasonably well, I've seen it used badly all too often. Used well, I could predict the effort for individual tasks down to an hour, which makes for beautifully easy planning of releases. Most people adopting scrum tend to drop bits and pieces here and there, or add new ones to suit them.

Doing that is a bit like bad game design: you can't make changes to a beautifully balanced game without careful rebalancing afterwards. But that latter part gets dropped from modified scrum, and then people are left disillusioned by the whole thing.

To pick one particular example of misapplied scrum techniques, the scrum master would interrupt developers with questions during daily scrums, turning the whole thing into a lengthy discussion of anything that might be related to the project. Yet at the same time he would insist on the stand-up part, as that was prescribed by the process. Yes, he was a bad manager, but to be brutally honest, good managers don't need to be told how to manage a project.

JonR said...

total horseshit.

Scarabb said...

Prefer Y over X, but don't be too overzealous. Well. That didn't help me much. Much like those hordes of consultants selling common wisdom like as it is some kind of philosopher's stone. So what agile really means is, if your project or initiative or service has apparently failed from applying agile methodologies - you probably got it wrong. Maybe the mix wasn't right or you had the wrong magic potion - IT HAS NOTHING TO DO WITH THE METHODOLOGY! IT IS YOUR FAULT!

Anonymous said...

To simply rant against every technique that one can think of based on your "reading" is so similar to what I see with people ranting about political parties while claiming to be constitutional scholars. This blog post contains shallow misapplications of the principles followed by a bluster of hot I Told You So's.

Your effort to simplify everything down to one dichotomy of people vs. process shows your conflict with managing projects. Maybe decentralized vs. centralized or chaos vs. order. However, people vs. process doesn't work because you can have a very people oriented process.

Contrary to your central premise, agile is very process oriented. It is also oriented towards having concrete results. Anyone working on a project that doesn't have concrete results until the end is not using agile methodologies. Anyone who doesn't have automated tests to control for quality is either: a. just starting the project so developers can keep everything in their heads, or b. working on a more mature project that has become a quagmire and wasting time manually testing the same thing again and again.

Most important in Agile is the concept of identifying risks and using prototyping and user feedback to mitigate those risks. Don't expect that everything can be figured out up front. Think ahead, but realize when things are getting fuzzy. Have confidence, not arrogance.

It seems more often than not you are rebelling against the confines "someone else's ideas". It really is hard to gain from reading if you can't absorb the concepts to make them your own.

Marcus Dijkstra said...

Quoting from,+first-order+components+in+software+development : "I finally concluded that there is something there, in front of us all the time, which we are not seeing: people. People’s characteristics are a first-order success driver, not a second-order one. In fact, I have reversed the order, and now consider process factors to be second-order issues." Still a nice and instructive read.
Secondly, bearing in mind that from different perspectives "bad" might be defined equally different, when things turn bad in a project, I think it is safe to say that nine out of ten times the root is "something there, in front of us all the time", without causing an appropriate readjustment of whatever needed to be readjusted in the process.
Cowardice will do you in, as well as .

Marcus Dijkstra said...

sorry, the last line ended with: as well as "reader-defined-list-of-appropriate-sins".

Martin Aspeli said...

I nearly stopped reading after the first paragraph, but I'm glad I changed my mind. Excellent article.

I've mostly stopped using the "a-word" with my clients. Instead, I talk to them about the challenges of software development, such as the relatively low degree of task repetition making it difficult to produce accurate estimates, and the fact that a shared language rarely exists between those asking for features and those delivering it. The best client conversations tend to centre around how we are going to ensure that we collectively understand and deliver upon the most important requirements, and how we use the learning that we gain throughout the project to improve our performance and estimations.

We normally advocate that every project starts with a review that is aimed at designing an appropriate process. We use a "spider" diagram that has various dimensions such as risk, time criticality, stability of requirements, integration complexity and so on (tailored to the project), mapping each into a scale from low to high. The further out from the centre you go, the more project controls and the more of a "heavyweight" methodology you need, from lightweight Scrum (or even less) to heavy-weight Waterfall. Importantly, once you've picked a process that fits (and by all means, they're all processes), you review and adapt it, not just at the beginning of the project, but as you are going along. So there's always some element of iterative work. But that's just common sense, as you said.

There are two things that annoy me greatly about agile:

1. A lot of lazy programmers who think they can code up anything in a weekend if just managers (and customers?) would get out of their way have used it as an excuse to do no process. Agile is *all about* process, it's just about picking an appropriate process.

2. Whenever you read about agile (in particular older works), it tends to be defined as a response to waterall. This type of defensive argument becomes a self-fulfilling prophecy, and immediately starts off the conversation in the negative ("this is bad, so here's how to do it differently").

Agile is really just common sense with a label that makes it easier to talk about and lends some legitimacy. But it's the common sense of people with lots of experience, who really understand that software development is not like laying bricks or developing public policy or planning a party. Purely linear, task-reductionist, time-estimated planning works poorly in almost all software development projects, because we don't know what we're building, we haven't done it exactly like this before so we can't estimate well, and anyway things change on almost a daily basis. And that's OK, we just need to be honest about it and not pretend like we can come up with perfect plans that never change.

Unfortunately, there are no shortcuts. And most of all, as you said, software development (not just agile) requires good people, good managers, and good supporting organisational structures. Give me a crap development environment and a flakey process with a great team any day.


Anonymous said...

Nice post, but I think those multiple pages where trying to say:

a. Design, Build, Test -- can't avoid it if you want to make money in the software. F/OSS, (and not OSS) is a different 'business' and scenario.

b. Exploit your domain knowledge: if you hiring a bunch of genius developers, have them sit down with a real [s/w] engineer and design for change. Use the domain knowledge and "design for change" in strategic places of your s/w. Process is a guide to exploit that knowledge.

c. Every manager wants "brilliant developers": that's known as a Red Herring. It takes management to build a good, motivated team that if works well together will accomplish your results. Management does have a job vs. setting a bunch of PhD loose to create something. Remember, the moonshot was a HEAVY process-oriented, average engineer-skilled effort, with some PhD experts, and look at the results (landed on the moon).

d. S/W is hard.

pellepim said...

One of the most sobering posts on development methodologies I have ever read. A real fun page turner too.

Anonymous said...

Funny article. I don't agree with 1/2 of the points, but you've added value to this agile vs. other stuff talk. And that's all the s/w dev community should be looking for... more value.

I'm sure you'll be getting the flood of 'agilistas' posting in the comments shortly to defend "Agile".

Anonymous said...

OMG - Are you saying managers should use COMMON SENSE to manage a project??? HERESY!! ;)

brilliant rant BTW

Anonymous said...

One thing you didn't mention is the collective overhead of a "shared resource" in a scrum setting. In a company that has multiple sprint teams, there are frequently people shared among them -- UI designers, QA testers, and doc writers. When you're only on one team, the meeting overhead (15 minutes for daily standups, 1 hour per sprint for planning, 1 hour for estimation, 1 hour for retrospective, maybe some time for a demo) isn't too bad. When you're on multiple teams, you're suddenly spending half your week on scrum process meetings in which you only have a small part in contributing.

Scrum may be more people-oriented than most other methodologies, but only if you're on a single sprint team!

Anonymous said...

Insightful, but where's the print link. Why do we always have to look at these dreadful mohawk pages. Why not just have one word per line; or one character: don't make it too easy on us.

sappworks said...

I feel like the PM technique should match the life-cycle of the product. If you're releasing the software in increments, which is often the case in business apps, agile seems like a good fit. When you're releasing complete products like games or avionics, agile doesn't seem to fit at all.

To me agile seems to accomplish one thing - it is a way of mitigating the risk of making poor choices during needs assessment.

Marc Thibault said...

An excellent presentation of a difficult topic.

One quibble:

I don't manage software projects anymore; I decided to leave that responsibility to people who were good at it. Since then I've concentrated on requirements gathering and planning.

When you say that Agile works well when the requirements are constantly changing, that says to me that Agile works well on poorly planned and managed projects.

If the requirements constantly change, someone has done a really poor job of requirements gathering. If the requirements change in mid-course it's reasonable to ask why the correct requirement wasn't caught in the first place, when the cost of making a change was negligible.

Agile should not be an excuse to make scope creep a methodology.

What I've seen on other people's projects is that the business impact of some technical decision comes to bite them. The requirements were both incomplete and wrong, because whoever did the use cases (or, god help us, the functional spec) failed to consider the business process that justified the technology.

Whoever does the requirements has to have a clear understanding of the business process and priorities, and translate those into optimal automation to support the business processes. The target is the business process, not the software.

This is dull stuff and probably should not involve programmers who live for excitement.

indomitablehef said...

Your statement that "Microsoft has published empirical data that shows that the process overhead for TDD increases the development effort by 15% - 35%." - Is patently false.

The study itself says:

"Another interesting observation from the outcome measures in Table 3 is the increase in
time to develop the features attributed to the usage of the TDD practice, as subjectively
estimated by management.
The increase in development time ranges from 15% to 35%.
From an efficacy perspective this increase in development time is offset by the reduced
maintenance costs due to the improvement in quality (Erdogmusand Williams 2003),an
observation that was backed up by the product teams at Microsoft and IBM."

When the study authors included the phrase "as subjectively estimated by management" they are basically screaming to the reader: "HEY, this part is NOT EVEN CLOSE to empirical!".

Ashtaroth said...

Great article!

I had only read part way through when I wrote and published my own post only to find that you touched on many of the same topics that I had something to say about.

Thorton said...

The political analogy seems confusing to me.

I thought the left was about big government and the right was about less government. Less government seems like less process, not more.

Also, socialism is about more government control. Again, the political analogy seems reversed.

Gwaredd said...

Well I am glad to see some polarized opinions. I would much rather that than no opinion at all ;)

For the record, in my younger days I used to be one of the a "Big A" Agilest, which is perhaps why I react more strongly to the hype nowadays.

I would love to respond to all the insightful comments here if I had more time.

However, one thing I would like to clear up is the MS TDD research, which I don't think I represented very well. My point merely was that by my definition, TDD is not an "agile" practice, which was more of a throwaway comment.

IIRC (I read it years ago) the 40%-90% improvement was related to post-release defect density. In game development we rarely fix bugs post release, so how representative it is is dubious.

I am very much in favour of TDD-like practices and recognise the many positive benefits. However, trying to convince a producer to a 35% increase in the schedule is a tough sell ;) I would need to show a tangible RoI and then get them on board to spend the money.

Actually in games development I have had much more mileage from functional tests rather than unit tests - but that would be a whole new article.

Anonymous said...

Why would anyone trust someone in the games industry for advice on how to run an engineering project? The games business has produced the most spectacular and embarrassing project failures in the history of consumer software.

Games are user-antagonistic, usually late, often filled with bugs, require tremendous hardware and bandwidth resources to run and grind their developers to dust in the effort to produce this kind of miserable quality.

In some businesses, we have to get our software right the first time, and it runs our business. We don't make software for children to waste away some time with in front of the TV.

This article was infuriating to read, the pure ego shines in every word. Go back to blowing smoke up each others' asses and keep you "process" opinions to yourself.

Gulli said...

This article was really promising until the TDD part. This has already been answered by some of the comments, but I'd like to add a couple of reasons why TDD falls under the agile umbrella.

1. Error proofing (making sure errors cannot occur) is a fundamental Lean technique. Read up on the automatic looms that formed the basis for the success of the Toyodas for explanation on this. Preventing problems as close to the root as possible is also a fundamental Lean
approach. TDD falls under this umbrella, and is more effective than defensive programming techniques, which is also a case of error proofing.

2. Dismissing 40-90% reduction in error density as "some benefits" is, well, unbelievable. I guess you work with some really, really good coders where errors and slowdown due to late bug fixing isn't a problem (are you sure they're not doing TDD??? ;-). Industry averages suggest that is far from the norm, so I have to question the relevance of of what you write.

3. TDD is an individual/pair practice, NOT a process involving multiple roles in different steps as you article suggests. TDD overhead thus isn't a process overhead, its a "go more carefully" overhead. It is furthermore a practice that is very unwise or even impossible to impose efficiently on individual programmers.

I've only read the summary of the MS/IBM study, so I don't know the experience levels of these guys or whether it is factored in in the study. As a guy who has been practicing and teaching TDD for years, those overhead numbers actually don't ring true, my experience is that I'm faster developing code using TDD, because 50% of the time it works perfectly the first time I integrate, and the other 50% of the time it takes a very short time to find the integration omissions. I very rarely have logic errors which I need to hunt down using debuggers. So for highly experienced TDD practitioners working with a technology stack that lends itself to TDD (no JSP for instance), I suspect this overhead to be even smaller. In addition, the design of TDD'd code is normally far superior of non-TDD code, and is much cheaper to maintain in the mid and long run, both due to better decoupled design, and the test suite you get as a side effect from TDDing. So not doing TDD is only potentially cheaper in the very short term.

Anonymous said...

Excellent article. You didn't mention one philosophical motivator however; leadership versus management. In Leadership companies, agile is great because it rewards iterative development. In management driven companies, agile doesn't work because it discounts artifacts.

My recent trouble with systems integrators pushing agile is that they refuse to tie down to fixed budget projects. Agile is great if time and materials is the way, but rails against the control of the fixed bid.

My personal opinion is that agile really isn't much different from trial and error software development, even though you have TDD to try develop the APIs, you often refactor and the late game architectural changes that "just can't be figured out (bs)" can cause massive refactors. There is no value in this, as you showed, design changes late in the game have huge cost. My arguments inevitably become religions because I battle with developers entrenched in agile think, that cannot mention any other methodolgy to compare...

Anyway, thanks for your rant...


cobolhacker said...

Good post. Understanding people and process applies to all kinds of stuff, not just software creation.

I haven't worked as a programmer in years, but one of the things I saw happening over and over in shops was a lack of discipline, sometimes on the part of the coders, but also with management.

There needs to be a solid plan and it needs to be stuck to. What you intend to make needs to decided on before any code gets written. It may sound obvious, but a lot of projects start out with a rather nebulous plan and poor managers often leave it to the developers to tighten it up. The devs often don't know how, or have no way to co-ordinate with other groups to do it. If the plan is no good, it doesn't matter what methodology you're using.

Sure, if someone comes up with a good idea or a problem, you adapt, but you should always have a good idea of what you intend to make.

Troy Dunn said...

Wonderful post! I work in Web Marketing but we face many similar challenges. Creative Vs Production is an ever present debate and as part of the management team, I struggle with the balance of People Vs Process. Thanks for taking the time to write this up. I look forward to reading more on your site.

Anonymous said...

Great article. Love it.

Anonymous said...

You mention that in game dev, you don't usually fix bugs post-prod. That is correct, however you DO pay for them anyways in lost sales.

So it's pick your poison - pay devs to fix it and release a nonbuggy game or don't and get a reputation for garbage games that don't run.

As to the notion of throwing agile away during production phase, this is a recipe for disaster and no small part of why the game industry has absurdly high turnover of people and even entire companies.

Anonymous said...

Initially I was put off by the title, yet after reading your post I agree 100% with the actual topic: Agile is just another mask for a horribly disfigured system.

Agile won't fixed your problems, nor will any other tool.

It also always the same everywhere when often the only working part of the machine (the developers) is forced to adapt to TheNextBestProcessModel.

Someone once said 'a fool with a tool is still a fool'.
But in this case, as you correctly state, it is management that tries to use this tool to hide it's own inability.

Gulli said...

Quote on anonymous: "Agile won't fixed your problems, nor will any other tool."

And I thought I was cynical :-)

I think you are trying to say that agile won't fix your problems without work and understanding of what you are doing. I'm going to add, neither will anything else, and to state it a bit stronger, everything else will require more work and deeper understanding, because everything else is grounded in the wrong paradigms, namely scientific command-and-control management and software development as a manufacturing process, which are both fundamentally a bad fit for software development.

Tom Gilb said...

fairly well put both paper and comments.

Here is my take: revised agile principles

which will be in the new (free)

Tom Gilb

TS192717 said...

1. If you read the entire microsoft report, you'll find that the conclusion of the research team is that TDD "...can significantly reduce the defect density of developed software without significant productivity reduction of the development team."
You seem to disagree with their findings about productivity, why?

2. TDD and agile are often connected for reasons of refactoring support and the ability it gives a developer to focus on high value functionality rather than low value requirements and design activities.

3. It's not for you. As you've pointed out, agile is suited for environments where the understanding of requirements by both the development team and the customer is changing. It's core feature is, well, agility. If I was running a project in an insular, controlled environment, like nasa, or a game development co., I wouldn't even consider an agile methodology - it isn't even close to fit for purpose.

As many posters have pointed out, the real problem seems to be misunderstanding of agile, development management and process, and software engineering in general.

Jim Anning said...

Interesting post - and I think you are spot on with the emphasis of picking the right tools for the job.
When I get involved with projects my starting point is always to understand two things...
1. How much experience do we have of doing this kind of thing?
2. How certain are we about what the outcome of the project will be?

When the answers are 1. Lots & 2. v.certain (think Bovis Homes building the 100th cookie-cutter house on an estate), process and gannt charts and 'traditional' project management are your best buddies.

Where there is experience, but little certainty then we are probably towards the more iterative end of the spectrum.

Where there is certainty, but little experience then we are either playing a procurement game or a 'do something - learn - do something - learn' type loop.

And for the final choice - no experience & no certainty (and it happens all the time - usually its a 'we are up the creek & have to do something, but dont know what' type situation), we are in the business of gaining consensus around little baby steps and inching our way towards a common understanding of the right way to go.

Each of these scenarios needs different tools, different leadership styles and different types of team. Call it "Situational Project Management" if you like.

Ben Sizer said...

It's interesting how many people employ the "No True Scotsman" defence here. "Agile didn't work for you? Then you mustn't have been using REAL Agile methods!" It might be nice to assure yourself that your success was because you followed the method properly when others didn't, but perhaps it's possible that the method is not as widely applicable as you think.

@Marc Thibault: unfortunately, in game development, requirements gathering really isn't that easy. Usually in games you're necessarily pitching to 2 customers simultaneously, one of whom controls the pursestrings but isn't the actual end user, and one who is the end user but who isn't paying you. On top of that, 3rd parties can release software in the interim that changes market conditions to such a degree that your requirements change drastically.

And to the guy who said, "The games business has produced the most spectacular and embarrassing project failures in the history of consumer software" is narrowing the goalposts somewhat. When you take out the companies who've had the luxury to produce iterations of the same software for decades (eg. Photoshop, Word, AutoCAD) then there's actually not much of a consumer software market left.

Anonymous said...

As one poster linked, and one poster mentioned:

Open source software has produced some cool things, BUT the stuff most people use in that world (The Linux Kernel, FFMpeg, GIMP, OpenOffice, Emacs/vi, X windows, etc... were all based on older, existing, Unix *designs*, which had heavy design development cycles. All the new stuff (kde, pulseAudio, etc..) are as buggy as anything from Apple or Microsoft...They maybe 'popular', but provide more hassle than value.

Kevin Gadd said...

Microsoft has published empirical data that shows that the process overhead for TDD increases the development effort by 15% - 35%. Despite the many positive benefits from TDD, we cannot possible consider anything that adds an extra 35% effort to produce artefacts the customer will never see as lean. Amazingly, people still try and associate this with "agile".

Are you suggesting that 35% more effort isn't worthwhile if you can decrease defects by 40% or more? Customers care a lot about a quality product, especially if the alternative is constant, visible regressions (for an example, look at basically every patch Blizzard has ever released for World of Warcraft.)

Anonymous said...

I find it ironic that you comment that people should use the right tool for the job, and proceed to write a multi-page rant about how things didn't work for you when you chose the wrong tool for the job.

Agile's greatest strength lies in contract development where "Just get the requirements right!" is pure fiction. Software Engineering, as a discipline, has had thirty years to figure out how to "Just get the requirements right!" and still hasn't done it. Requirements WILL change, your early understanding WILL be imperfect, and resorting to "But this is what the requirement said!" as a legal defense WILL NOT result in a successful project and a happy customer.

Game design seems to be much more suited to the waterfall model - customer input/requirements volatility is minimal, and as you said you rarely bother to fix bugs anyway, so who cares about improving quality to reduce maintenance costs?

So you bash people for expecting one tool to solve all problems, then proceed to bash the overall usefulness of the tool because it doesn't solve YOUR problem.

Very nice.

Justin said...

I agree 100% - I had this discussion with my lead the other day who is a self proclaimed "agile" enthusiast.

I made the proposition that "agile" (whatever that means) teams that perform well are probably just made of better than average individuals, and those individuals would perform well under *any* methodology, system, or religion.

Example: Give Michael Schumacher a Honda Jazz to race around Monaco. Now, I'm 100% sure he'd beat me, even if I was driving a S2000. Innate skill is king.

Anyway my lead reluctantly agreed with me. Thanks again for the article - hope things are well, I still haven't made it back to the UK for a warm beer yet.

Tony said...

Thanks for this article.

I appreciate seeing your posts that help cut down on the hype and pinpoint what makes "Agile" effective.

And it isn't MAGIC!

Now I need one for "Cloud."

Anonymous said...


You must have a pretty weak lead then. It's true that the team talent is the number one determinant of the outcome, but to say that *any* process has an equal chance of success is pretty dumb.

Give Michael Schumacher a Schwinn and race him in your S2000.

Good leaders don't blame process or tools for not working miracles. They aren't "frustrated by the naïve views from the rank and file who fail to appreciate the reality of business". They educate, mentor and guide them down the right path.

Justin said...


To clarify:

I didn't say the team would perform *equally* well under any methodology.

I was trying to say that innate performance is a stronger indicator to team success than methodology.

Essentially agreeing with Gwaredd.

Gulli said...

Justin said:

"I made the proposition that "agile" (whatever that means) teams that perform well are probably just made of better than average individuals, and those individuals would perform well under *any* methodology, system, or religion."

There is research that suggests the contrary, that is that methodology has a lot to to do whether you can harness the wisdom/knowledge/skills of the group. Given the right methods, groups regularly outperform the "best" individuals, also within the group itself. Command-and-control based methodologies and management practices are for instance guaranteed *not* to harness the group to its full potential. I recommend reading James Surowiecki's Wisdom of The Crowds for more on this.

Justicle said...

... If you give Schumy a Ferrari he'll do even better. Of course methodology counts, or we wouldn't be talking about it.

Anonymous said...

"August 25th 2009 marks the day we officially entered the post agile era when manifesto co-signatory, Alistair Cockburn was piped on stage and announced “I come not to praise agile, but to bury it” at the Agile 2009 keynote presentation."

Anonymous said...

"Post-agile" is yet another BS label.

BTW, if Cockburn is so "post-Agile", why is he still teaching "Certified ScrumMaster" classes?

Marc Thibault said...

Scrum is independent of Agile and predates it by a few decades. Agile uses Scrum but Scrum doesn't need Agile.

If you had watched the presentation, you'd know what he meant, which is not what you naively assume.

no. said...

Amiga is better than Atari ST and Mac rules over PC... I hate religions.

Agile is just a framework. When I was taught Scrum, it was all about "Inspect and adapt" and "Common sense".

Tools (Processes, methodologies, whatever...) are toys and without suitable implementation, they will eventually be broken by the context or worse, by your teams.

Keep the big picture in mind, keep interest in technical implementation details, be inspired, be fair and honest, remain sensible to risk and opportunity, balance power and pressure, share your vision, listen to people, gather feedbacks, adjust your act, communicate, decide, report: Manage... The rest is playground : Use the toys your teams are comfortable with.

There is no magic recipe for management of software development projects. Especially in computer games industry where creativity and innovation cannot be ignored anymore.

Great post. Thank you.

unwesen said...

"Actually in games development I have had much more mileage from functional tests rather than unit tests - but that would be a whole new article."

Well, functional tests without unit tests is like trying out whether a plane flies without pre-flight checks. It can work out for you 90% of the time, but the remainder of it you're dead.

Functional and unit tests complement each other, they're not alternatives. They test different things. Unit tests help ensure that when you run functional tests, you can somewhat rely on the results.

unwesen said...

"When you take out the companies who've had the luxury to produce iterations of the same software for decades (eg. Photoshop, Word, AutoCAD) then there's actually not much of a consumer software market left."
Except for the whole internet.

Ben Sizer said...

@unwesen: how much of that is both 'consumer' and 'market'? ie. How much commercial software is the typical consumer actually buying to use the internet? Very little. And of what is left and isn't open source, how many distinct products are actually in widespread use? I expect you could count it on your fingers. This is why there are virtually no high street shops here that sell normal software and hundreds that sell games - that's pretty much the only software that consumers actually go out and buy themselves.

Anonymous said...

What's unique about large "consumer software" projects that hasn't impacted other large projects? Don't large military projects that do most of their planning up front have huge problems? Projects like the baggage handling system for Denver just prove that you can't plan your way out of everything with complex projects. You have to iterate.

Anonymous said...

You completely misunderstand "people over process". Process doesn't give you guaranteed control over "muppets", "toe rags" (or whatever else you call your people) and what they produce.

It's about the line between what you should rely on for process to do and what you should rely on for people to do. Breaking apart Scrum practices and placing them on an axis between the two doesn't make sense. All the practices are process. But what the entire framework is trying to do is allow some emergence of self-organization to handle appropriate things at the team level.

Too often people in management, like CTOs, don't get this. They are all about process and don't respect/support/mentor people.

James Peckham said...

well you touched on it in several places... your tools and process matters but your people and how they interact matter more.

odiseo said...

Great article.
But I think no methodology is sold by their authors or "devotes" as either "silver bullet" nor "effective in any circumstance".
Anyways, your ideas expressed are insightful.

Anonymous said...

problem is, these methodlogies are created from teams of the like of kent beck, martin fowler etc.. which if were honest, they guys could create a cross-stitch methodology were everyone knitted their own table cloth and the code would still be clean and the project work, not because of mystical knitting prowess but they are competent and capable of working in a team, oh and they firmly believe if there is a crap team member who cant be sorted sharpish, fire him.

nearly any methodology will work with competent, dedicated and aware teams who understand the business they are working in. if not, theres not a methodology to save you or your business.

Tadhg said...

An interesting article.

However I'd put it to you that the failure or success of any project management methodology is very heavily reliant on the quality of leadership. Not management, not the day to day stuff. Rather, the leadership.

Time and again the complaints that arise from agile are nothing at all to do with agile. What they actually come from is a senior leadership in a company having hired in or instituted a culture that deeply ingrains certain kinds of values and processes, both of which create inertia.

Inertia is what makes organisations very resistant to change (such as the "but when deadline day comes and you have nothing to show" comment above is an example: The problem isn't that you have a smaller amount of much better functioning code, the problem is your organisation's leadership has created a culture of poor deadline practises). It tends to lead to a lot of talk about "common sense" and "in the real world" and so on and so forth.

All of it is perfectly understandable, and all of it is a reflection of the organisations from which people come, found or join.

I personally find agile planning in particular very useful to try and visualise and understand a project and help hit on the right parts early enough. I am fortunate enough to have a company culture around me (my own) that is interested in that kind of thinking, and also some clients who genuinely want to try it too. I have also worked in many places where agile and scrum were just words, and would never be able to actually work in an agile fashion because of the leadership.

So in real terms, it doesn't matter how you work. Whatever method works for you works, there's really no mystery to it. What's the more vital question to ask is whether your employer is providing the kind of leadership that makes your work encouraging. Don't blame the process or the methodologies. Blame the leadership. 98% of the time when anything is wrong in how a company does things, it's because of either poor or intentionally negative leadership.

Marc Thibault said...

tadhq: Interesting perspective. Just to show the world isn't always simple, the last few cock-ups I know about:

(1)The project was sabotaged by the client. She was using the project as a political weapon.

(2)The project was sabotaged by the programmers. The requirement was for a complex multi-modal system; they ignored the requirement and built a simple system that they could understand.

(3)Sabotage would infer more intelligence than this bunch had. They said they were doing Agile, but they just knew the words. The client ran out of money before any useful function was delivered.

(4)This highly political project was sabotaged by the BAs who wrote requirements serving their political agenda instead of the actual requirements.

I can't think of a single project whose failure can be blamed on the "leadership" or lack of it.

It's a big world.

Anonymous said...

my ex project manager got me, a then recent graduate in his first developer job to be part of an agile team and use xp fully on a new project. I got everybody together, really worked hard, met all the demands, always had something workable and clearly an improvement to show management and BAs etc.. and then, all of a sudden I got hauled up for mistakes that were not mine, project might be getting cancelled, meetings arranged without me, I ended up having a severe anxiety attack and collapsed, with the added bonus of they fired me while I was off sick.

I found out that 1 week later a project I never heard of which came from nowhere but apprently was another agile project run by my project manager had succeeded in all its glory where mine had apprently failed, it even used my code and they brought up the same concerns I had with the existing structure

my project manager, senior dev and fellow dev all got promoted

I talked to one of them a little while ago and he just said, they all new what was going on, If I had just let the project fail I probably would have been kept on.

I now have anxiety and stress problems frequently (I never had a single problem with stress in my life before) and have issues, thinking my work is never good enough and always putting myself down.

Term Papers said...

I have been visiting various blogs for my term papers writing research. I have found your blog to be quite useful. Keep updating your blog with valuable information... Regards

bowling score said...

I think that to succeed in a project management are the most important people, if they are encouraged either monetarily or royalties, encourage people to achieve better performance.

Policy Administration Solution said...

I think this is the critical problem that most people that believe they have worked in an agile environment face thank you.

viagra online said...

People are still attached to their toys and a "good" burn-down chart is motive to give some free pizza to the team, even when the team reduces its own velocity every month to get a good chart.

Ivan Tihonov said...

Can't agree about last "Client" part.

In your case Publisher is a kind of management above and slightly to the side of you. They do not buy your product, you sell it via them.

Your real clients are gamers with which you interact in face of testers.

John said...

I would like to appreciate the great work done You

Prince2 Training Courses said...

raucous and humourous too!

Charles said...

Thanks for sharing, I will bookmark and be back again
Scrum Process

me109 said...

Thumbs up! I really enjoyed the insight.

I'm heading down this path with my own business and need all the info I can get my hands on

seona cruz said...

Thanks . That was good information. I had a question - given all the buzz around Scrum (my company has started adopting it in a big way with almost all projects having stand ups, whiteboards coming up everywhere etc.), do you think it is a good idea to get a certification in Scrum?