Game Development in a Post-Agile World

The hype and dogma of Agile evangelists has left in its wake a trail of broken projects, ruined businesses and misguided neophytes bleating the tired doctrines of their long departed prophets. The games industry was no exception, with many swept up in the phantasmagoria from which we are only now beginning to witness the debris.

Of course, there were always the more levelheaded developers, unswayed by the silky promises of besuited "consultants" - even the ones offering certificates and everything. Today these more progressive thinkers talk of "post-agile" - or common sense as it were previously known.

But ripples of reason spread slowly. If I have to sit through another meeting with some little "agile" toe-rag defending their train wreck of a project then I may end up forcibly ramming a kanban where the scrum does not shine. It is not that I have anything particularly against agile, quite the contrary, but at the end of the day I have a product to ship and no patience for the quixotic.

Project management experiences are hit and miss in an industry that is just growing out of adolescence. To engage with these misconceptions we need to take a step back and put agile into context before taking a holistic look at what contemporary game development might mean.

As the Technical Director of a large studio, I oversee our own projects and interact regularly with publishers and other developers. There is plenty of opportunity to observe different management styles in the wild. Agile has certainly gained in popularity in recent years but I have also observed a few trends that go with it.

Firstly, people use the words "agile" and "scrum" interchangeably. The idea that agile and scrum are separate things appears to be an alien concept. Yet despite the prevalence of scrum as the methodology of choice, there is a decided lack of scrum-like language or attitude amongst many of the teams. When pressed on what people actually do on a day-by-day basis it seems that "scrum" is wide and varied. Sometimes I struggle to find any correlation between what the teams are doing and what I understand it to be.

Another common opinion is that the alternative to agile is waterfall. Many people talk in these terms. Therefore, if you are not an agile devotee then you must be stuck in the past, desperate to cling to the familiar and the conventional. Clearly, you need enlightening or worse, you just do not "get it".

We have all been on the receiving end of bad management at some point so it is easy to empathise with developers yearning for a more natural process and agile promises a seductive alternative to the traditional wisdom. However, whilst developers complain about management stupidity, the leadership is often frustrated by the naïve views from the rank and file who fail to appreciate the reality of business.

There is general misunderstanding of what agile really means for a company, what the real issues are and how it all fits into the bigger global picture. Ignorance impairs constructive debate and I often find myself spending a lot of time clearing unhelpful philosophical hurdles before we can get down to tackling the actual problems in front of us. This, frankly, is annoying and the driving force behind this little rant.

Many people think agile is a new idea and these are contemporary issues. Actually, project management has been around for centuries and as applied to software then certainly for a few decades. If you consider agile has its roots in lean manufacturing then "agile thinking" has been around at least since the 1940’s (although I suspect the pyramid builders and artisans had similar debates). Over the years, a large body of knowledge, empirical data and case studies has built up so we can make some qualified statements.

However, if we confine ourselves to software development we can see there are in fact quite a few more formal methodologies than just Scrum or Waterfall. Here is a handful:

• Scrum (& XP)
• PSP & TSP
• Prince2
• RUP
• AUP
• DSDM
• FDD
• PMP
• EPF & OpenUp
• HERMES

Of the items above, some are generally considered agile whilst others not so much. All of them have their advocates and used successfully on many projects and this raises a number of interesting questions.

If they are all successful then why does it matter which we use? Are some more effective than others and if so then why? Are the agile techniques more effective than the non-agile ones? If there is more than one approach to "agile" then why is scrum so popular? Is it truly "best-of-breed" or does it just have better PR?

We would not be the first people to ask questions like these or attempt to measure the effectiveness of different project management techniques. For example, the American Department of Defence developed the Capability Maturity Model as a framework for assessing the project management facility of potential military contractors.

However, as a basic thought exercise I want to start with a much simpler proposition.

Of the methodologies listed above, which are more agile and which are less?

A simple question and to answer this we need a formal definition of "agile". Fortunately, the Agile Manifesto provides us with one. I like the manifesto as it shows a clear understanding of software development and a well thought out statement of values. It contains valuable insights that one should not dismiss.

Consider the first proclamation, "we value individuals and interactions over processes and tools". The Agile signatories are making a bold statement (and moreover it is their first statement) in valuing personal interaction over the process-orientated aspects of development.

This gives us a more clear-cut dimension for differentiability. We can consider techniques that focus on people orientated activities as "more agile" and methods that focus on process as "less agile". Of course, this is only one axis of comparison, a crude taxonomy perhaps but a useful and tangible metric to take for a spin.

You might be wondering what we mean by the slight abstract terms "people orientated" and "process orientated".

The idea of "process" is politically charged and one of the reasons people can have strong emotional responses to project management "methodologies". This idea taps into more fundamental and nebulous concepts.

Process is an instrument of authority. A process is a statement of what to do, how we should do it and in what order. Process is conservative, hierarchical and formal. Process upholds the status quo. It is management imposing order and control from the top down. If process were a political party, it would be in the right Hegelian camp (and incidentally the right hand side of our line).

The alternative to process is to delegate responsibility, to trust individuals to organise themselves more effectively because they are better able to understand the task and therefore be more productive. Self-organisation is liberal, communal, informal, a meritocracy. It is emergent, bottom-up, grassroots. If we were to abuse the political metaphor even further, it would be some kind of hemp-wearing longhaired hippy socialist.

This is I feel very much cathedral and bazaar territory. To come back to something more tangible, we could consider "process" as belonging to traditional software engineering and self-organisation as belonging to software artisanship. Something Paul Graham so effortlessly explored in his essay Hackers and Painters.

With all these complex and fuzzy memes floating just beyond the grasp of our consciousness, it is perhaps no surprise that discourse often becomes tenuous and emotional. This milieu of uncertainty is a fertile breeding ground for religions like Agile and the peddlers that profit from it. Ask yourself why there is such a thing as a manifesto in the first place. It is unabashed political positioning rather than an objective view.

I am not a partisan of either camp. I am only interested in results and do not have the time or inclination to advocate a particular point of view. Getting effective and timely results is often about being flexible (although not always I might add) so it is worth understanding what you get from each.

So what is so great about process? Well, it gives us:

• Repeatable and predictable results
• Quality Assurances (through the above)
• Cost savings through the ability to optimise work flows
• Defined work flow allows us to use cheaper labour
• The promotion of best practices and conceptual integrity
• The ability to scale to large numbers
• A means to effectively track our progress against the objectives

McDonalds is a great example of successful process. No matter where you are in the world, you know what you are going to get and you get it quickly and cheaply. This process has successfully scaled to thousands of restaurants. Whether you consider this good or bad it is hard to argue with the results.

Nevertheless, software development is much harder than frying beef burgers. Process is sometimes inappropriate or unconstructive.

Process can:

• Increase cost and reduce performance through bureaucratic overhead and waste
• Hinder our ability to change or adapt to new situations
• Stifle our capacity for innovation and creativity
• Require discipline and training which takes effort
• Only effectively be applied to known quantities

Corollary, it follows that the opposite holds true for more people orientated approaches.

Consider what would happen if we went to the extreme ends of the line. Too much process and the project would grind to a halt, stifled by bureaucratic red-tape, protocol and form filling (much like government). Alternatively, no process at all and the project would be a chaotic mess of uncoordinated activity with lots of energy spent thrashing but generating very little momentum to move us forward (um - ok, much like government).

At the extremes, we always fail but of course, in practice nothing is "all process" or "all people". The trick is to define process where we need it and relax the requirement when it gets in the way. Good management is flexibility, identifying what is important and understanding when to apply a different approach. Many projects get into trouble when they try to take a technique that worked very well in one situation and apply it wholesale in a completely different scenario without understanding what made it a success in the first place.

All projects have different pressures, different criteria to meet and desiderata to balance. When I board a 747 for a transatlantic flight, it is pretty important to me that the plane will get to its destination in one piece and not smack into the middle of the ocean somewhere because some muppet forgot a semicolon. It is not so easy to restart a plane after it crashes to "see if it does it again". Safety critical systems demand assurances they will function correctly within well-defined parameters when we run them. This is more important than the time or cost of development and process gives us those guarantees.

On the other hand, updating an e-commerce website with the latest promotion or the CEOs colour de jour requires the ability to react quickly. An agile approach gives us that flexibility.

Even across very similar projects, the goals can be different. In game development for example, hitting the shelves for Christmas and keeping the costs under control may be crucial for a project. Whereas establishing a new IP then quality may be the overriding factor.

That is probably more than enough exposition, it is time to arrange our methods in order of how much process or people orientated they are.

Now you might be wondering how I decided if one particular method was more or less process orientated than another. You may well disagree with my positioning on the scale and you are probably right.

The problem is that every method contains a mixture of different (and hopefully complementary) activities. In addition, the degree of "process" can change depending on the actual implementation.

Consider scrum. Amongst other things, we can define Scrum as

• Sprints
• Scrum Roles
• Sprint Planning Meeting
• Sprint Retrospective
• Sprint Review Meeting
• User Stories
• Daily Scrums
• Estimated Backlog
• Burn Down Charts
• Release Planning
• Stakeholder Collaboration

Teams often also include related activities such as

• Task Boards (kanban)
• Test Driven Design (TDD)

Clearly, some of these activities are "people orientated", such as daily scrums. However, TDD or Burn Down Charts are in my opinion process.

Microsoft has published empirical data that shows that the process overhead for TDD increases the development effort by 15% - 35%. Despite the many positive benefits from TDD, we cannot possible consider anything that adds an extra 35% effort to produce artefacts the customer will never see as lean. Amazingly, people still try and associate this with "agile".

To get a more accurate picture we should consider each methodology as a dynamic range encompassing all activities, which moves according to the actual real-world practice.

The point that I am trying to make is that when you get down to the details, defining what is or is not "agile" is not straightforward or clear-cut and putting activities into a box and saying what we can or cannot use just seems unhelpful to me. Agile was a necessary and important counterpoint to traditional software engineering thinking but only a small part of the overall picture.

I have seen very successful agile teams but many others that were abject failures. Similarly, some teams that have flourished with a traditional waterfall approach and then there were those that have consistently missed deadlines. It seems that success is not entirely dependent upon any one particular management approach and that the problem is more complicated than we might first assume.

There are many things that contribute to the success of a project and not all of them are always in our control. Whilst there are no guarantees, if we at least recognise the dynamics we have a better prospect of increasing our chances of success.

OK, so saying we need to pick the right tools for the job is perhaps not that profound. Nevertheless, I think it is important to say it and there are certainly people who need to hear it.

Once we buy into the idea that agile is just one of many tools, the questions become much more interesting. How do I know which is the most appropriate tool for a given job? What tools are there in the box? What are the essential characteristics of different jobs? "People and process" is a useful construct for illustration of course but only opens the batting on this front.

To answer these questions we need to get down to the specifics, those dirty and inconvenient details. Every game, every team, every company is unique and there are many factors that can affect a project - the size of the team, corporate culture, internal politics, budget limitations, physical constraints and so on.

However, there are certain dynamics and pressures we frequently find and are unique amongst creative projects. There are plenty of books that cover the usual suspects so instead I have picked out five things that I think are either particularly important or peculiar to the game development.

People
In my opinion, the most important success factor in project management is the people. Talented, motivated and experienced people deliver despite the management. They will find a way around processes that get in the way or add structure where there is chaos.

"Surround yourself with the best people you can find, delegate authority, and don’t interfere", Ronald Reagan

This idea underpins agile development and is fundamental to non-linear management. As this is perhaps, the single biggest factor influencing your success you should put every effort into hiring good people and weeding out the chaff. All this fancy management nonsense is really just icing on the cake.

However to assume you can always hire the best is somewhat naïve. There are some other things to consider.

• If you need to scale up the team size significantly you will face some hiring challenges and the best people will not always be available when you want them.
• Experience is expensive and your budget probably will not support an entire team of elite developers.
• Recognising areas where we can "deskill" without the loss of quality reduces costs which means we can either improve our commercial competitiveness or have more funds to redirect to other areas (you would not staff McDonalds with PhD graduates for example, although it might be interesting to try).
• Too many chiefs can lead to trouble. Who will lead and will experienced people be happy playing second fiddle? Unclear direction and authority will lead to poor conceptual integrity.

In the meantime, you have to work with what you have so understanding a person’s capabilities is a key skill for any manager and different people respond better to different things. Whilst some people like structure, others prefer more freedom for example.

Most agile methods assume that you have a brilliant, experienced and motivated team. Many agile techniques may work well but inexplicitly fail when applied to average, junior or apathetic members of staff.

Experience also plays a factor. For example, I would be prepared to take my best programmer, unleash him on some thorny technical problem, give him everything he needs, sit back and wait for the magic to happen. However, I would be less willing to extend that courtesy to a fresh faced graduate straight out of university. Most juniors need guidance, mentoring and structure until they find their feet.

It follows that a process-orientated approach would work better for juniors and a more people orientated approach may be applicable for senior members of staff. Corollary, we can see that the team composition in terms of experience and personalities will have a bearing in what approach works best.

Of course, people are much more complicated and idiosyncratic for this to be enough. My point is merely that different people respond to different approaches and that experience and skill are factors to consider when wondering if an "agile" or a "planned" approach is better for you.

Creativity vs. Production
There are two fundamental drivers in game development, creativity and production.

Production pressure comes from the business need to deliver a product on time and within budget. To do this we need to be able to quantify the work and risks, and this means planning and scheduling. This is hard work and requires a bit of effort but essential for answering those two key questions - when will we be done and how much will it cost?

For a plan to be precise and accurate, we ideally need to work with known quantities. Production therefore naturally leans towards quantifiable process driven activities.

Creativity on the other hand is the heart and soul of games development and pushes us to create better games. It is a woolly organic process of curiosity and exploration. There are no guaranteed outcomes or convenient timeframes and this is frustrating from a production perspective. This often leads to attempts to manage the chaos that more often than not stifles the creative process and blunts that "je ne sais quoi" quality.

It is not surprising then that some of the most innovative and critically acclaimed games have come from studios whose project management could charitably call disorganized . Similarly, "corporate" studios with mature project management structures tend to produce more derivative products. It seems these forces are mutually incompatible.

However, we have to be careful about what conclusions we draw from this. It does not follow that chaos is a prerequisite for creativity or that if we want to ship a game on time and within budget we must forgo innovation. We need to find a way to balance both harmoniously with the objectives and within the boundaries of each project.

There is a lot of mysticism surrounding "creativity", particularly within the games industry. This is a big topic beyond the scope of this essay. However, people have been studying creativity for many years and there various effective models and techniques we could leverage .

Creativity is a skill, like many others that we can teach, nurture and develop in everyone. It is less about a spark of genius and more about fostering the right environment that stimulates and incubates ideas. If you are interested in further reading I highly recommend the book "Orbiting the Giant Hairball" by Gordon Mackenzie which explores his experiences of being creative within a corporate environment.

Because of the iterative and changing nature of the creative process, it lends itself naturally to a more agile orientated management style. However naively applying scrum in the hope that you will magically be more creative is somewhat misguided. Flexibility is necessary but it is not sufficient.

Finding a harmonious balance between creativity and production is not easy and not all projects are equal. Whilst I would love to work on creative and innovative titles all the time, this is not always possible. The reality is that the business sensibilities (and by extension, production) are more important for the majority of companies. To be sustainable all businesses need to be profitable. If we want to keep staff employed and continue to make games then we need a profitable business.

For example, missing a milestone payment could be fatal for a small "work for hire" start-up living hand-to-mouth, so production (survival) is more important in this case. Alternatively, a large publisher-backed operation may want to establish a new IP so they can reap the long-term rewards this offers. For this project, they will have more appetite (money) to grease the creative wheels.

Product portfolio is another aspect to consider, a large studio may have one project to "pay the rent" (production focused) and a second to increase the prestige of the studio (creatively focused). Understanding the business context is important in assessing the capacity and ambition for each game and ergo, the best way to run it.

Some people make the argument that commercial success will simply follow from making a good game. It certainly does not hurt and in a perfect meritocratic world, this would be the true. However, we do not live in that world or at least it is not that simplistic. It is true that some studios have carved a niche out as being creative and innovative; this in turn has led to commercial success and investment interest. This is not viable for all studios and the difficulty they will face will be sustaining that momentum, continuing to be creative and innovative beyond one or two projects and surviving in the long-term. I wish them luck. Ultimately, the market will be the final arbiter.

Phases of Production
Another facet that is peculiar to the games industry (boxed products anyway) is the timeline - pre- production, production and post-production . The exact definitions vary between projects so I am going to define them broadly speaking as:

• Pre-production being the period in which we work out what the game is and how we are going to make it
• Production as the period in which we make the game
• Post-prod as the period in which we take the finished game and actually turn it into a good and shippable product. This in turn has two phases - alpha (game play polish, i.e. making it fun and balanced) and beta ("pure" bug fixing).

Interestingly even the most ardent agile proponent I have met still refers to these phases of development despite the obvious similarities with the design, build and test stages of a waterfall model. It appears they are intrinsic to game development at a fundamental level.

Agile methods prefer iteration to following a plan. This works in environments where the requirements are constantly changing and the overhead of rebuilding a plan each time would be too great. In game development, requirements change constantly so naturally an iterative approach would seem the natural fit.

We could for example apply this idea wholesale to the entire project lifespan and dispense with the pre, prod and post phases, which would look something like this.

However, the problem here is the cost of change increases with the more content we produce. Say we decide to change the distance the player can jump, and innocent enough change. This in turn has a knock on effect on the level designs and potentially a number of other systems and bits of content. The more levels we have produced the more expensive that simple change will be.

Above is a traditional cost of change graph you can find in many books on software engineering. The earlier we make a change the less it costs. Applying an iterative approach to the entire development lifecycle exposes us to potentially expensive changes later on.

Ideally, we want to have all the fundamental questions answered early so we do not have to change them later with the associated knock on cost implications. The pre-production period should have precisely that goal.

Pre-production is the most creative period of any game. Early pre-prod is very organic and exploratory; the latter part needs to focus on providing specific answers and technology so that when we enter production we have a good handle on the scope and pipeline. Therefore, if ever there were a time to be agile and iterative then it would be during pre-production.

The production period is where we create the content and in turn where we spend most of the money. We need to keep costs under control and the more process driven and deterministic this is the better. We still need to respond to feedback and change but we need to keep this manageable.

In alpha, we revert back to an iterative cycle as we refine, polish and tweak. Beyond alpha everyone is working from the bug database, which in my opinion makes it process driven albeit of the self-sustaining, organic kind.

We can change our approach throughout a project lifecycle; we do not always have to stick to the same thing throughout.

Skill Sets
Another peculiar aspect of the games industry compared to other development environments are the skill sets involved, the three most common disciplines being code, art and design .

The different nature of the work and the complex blend of interactions between disciplines is something we need to consider. Analysing the metrics gleaned from our in-house project management software we can see some common trends within my studio.

Art tends to be a deterministic discipline. Once we have generated a list of assets required and the workflow involved we can predict with quite a high degree of accuracy how long it will take and the end quality we will get. In addition, this work is divisible so adding more people means we get there quicker.

Code tasks on the other hand have very different characteristics. In comparison, the data shows that programmers spend much more time doing unscheduled tasks (anywhere between 20% - 40%) and are more inaccurate with their estimates. This is because they respond to change requests more often, fix bugs or deal with hidden complexity and imprecise designs. Additionally, complex systems are not divisible in the same way so adding more people is not always helpful.

It follows that what works well for artists may not necessarily suit programmers and communication difficulties may surface between the gaps. You could for example run the code team in an agile way but apply a more planned production approach to art, particularly in production.

Client
One interesting aspect of agile is the desire for stakeholder involvement. For simplicity, let us assume that our stakeholders are the publisher stumping up the case for the product. In games we have always had stakeholder participation in the form of an external producer who acts as a spoke person for other interested parties on the publisher side (marketing, legal, QA, etc).

This would seem to tick the boxes but whilst customer collaboration is a noble aim, not all clients are created equal. There are many factors to consider - how much trust and freedom are they willing to extend to you? How much meddling and interference can you expect? How much political clout does your EP have within the rest of the business? Just a few things that contribute to the environment in which the project must survive and understanding the external forces helps you plan your strategy and management style.

On a basic level, if you have a flaky client who is constantly moving the goal posts then an agile approach is your friend. If you have a client who does not care about the outcome too much so long as it is cheap and on time, then good planning will serve you well.

To talk about "agile vs. waterfall" is simplistic. My gripe with agile is not that it is bad or does not have value but that it is all too often used as an excuse by too many people for too many bad project management decisions. Developing software is hard and developing games is particularly tough. We only have to look at the software development landscape to see the rotting corpses of failed projects as evidence of this fact. But then if it were easy, it would not be anywhere near as enjoyable.

Whilst we continue to be blind to the essential complexities, there is fertile ground for charlatans peddling their snake oil with the fancy nomenclature and a "silver bullet" promise. Take a step back to consider your own position and revisit your assumptions. To be successful you need to work continually at project management, not be afraid to try new things, to experiment, to measure, to question but above all remember to have fun as you do. After all, is that not why we wanted to make games in the first place?

No two projects are the same, well - unless you make racing games.
read more...

Trends of 2008

It is the end of yet another year of taking on far more than anyone feasibly has time for and time for the annual brain dumping, so I can clear enough room in my head for next years worth of garbage. So here is a run down of all the interesting happenings in 2008 ...


This year has seen a number of interesting trends in computer games, the biggest I think being the continual change in gamer demographics which seen the rise in new markets, alternative revenue streams and different types of games (even 'non-games' such as Wii-Fit). It will be interesting to see how the changing economic climate affects the industry going into 2009, several big studios have already being making redundancies or have gone under. Whilst generally the industry seems to be bucking the trend, there is bound to be some kind of knock on effect leading to smaller budgets, fewer and less risky projects and more closures of the 'fatter' studios. Generally I am all for tidying up the closet (so long as that does not include me of course ;)

But what do I know about such things, I am but a humble programmer. On the technical front in 2008 there was quite a bit of interesting activity; given the success and power of GPUs over the years it was perhaps inevitable that parallel processing would find its way into general purpose computing. Whilst there has been a lot of interest in multi-threaded computing for games for a while, there has been a fair bit of movement this year with the release of CUDA 2.0, "compute shaders" for DirectX and Intel releasing more details about Larrabee and the smoke demo to get us all excited about the possibilities for multi-threading and of course helping them sell lots of new funky processors in the meantime.

Another interesting piece of technology to watch was the Enlighten engine from Geomerics, which makes real-time radiosity a realistic options for games (which is very cool). This interesting thing for me is the technology originated from a branch of mathematics called geometric algebra (which is were we get funky things like quaternions). If geometric algebras offer much more optimised maths then it might be worth exploring them further (oops, there goes my weekends). Given the subtle effect radiosity has I think Enlighten will only make sense for particular kinds of games until the cost is less prohibitive, although there is a lot of potential for real-time light map tools - allowing artists to create light maps in minutes rather than days!

This year sees the release of XNA 3.0 with Zune support. I have been very impressed with how easy and accessible XNA makes game development - particularly cross-platform game development. I had a quick terrain rendering demo (with atmospheric light scattering and reflective water) up and running in a few hours (OK - courtesy of Google and the nVidia shader library, I'm not stupid ;). The uptake for XNA has been impressive and with generous terms I think we will see some interesting things to come in 2009 with the blurring of the hobbyist space and professional space.

With the release of Expression Blend 2.0 early in the year WPF and Silverlight are looking much more interesting. I have been experimenting with WPF and the concept is great, however it will not be every ones cup of tea. The separation between logic and presentation is a good one and allows for a much more sophisticated user experience, you can clearly see the influence from web development at work here. However this means you now need artist and programmer skill sets to develop even the simplest of programs. Also the integration between WinForms and WPF components is not as fluid as it could be which can be problematic if you want to support old code bases. One of the interesting possibilities though is to use XAML as a framework to develop a GUI system for games (and in fact we are already doing this for one of our titles). Whilst there is a quite a bit of work to fully support all of XAML, we found the easiest approach was to derive our own custom components and use Expression Blend as an editor for placement and animation. This seemed like the easiest and most pragmatic approach for our needs right now and has certainly improved the artist workflow. Maybe 2009 will give us an opportunity to develop this further.

Finally, 2008 saw the growth of procedural content in games. Of courses games have always had a fair amount of procedural content but we are getting much more sophisticated in what we are trying to do. The animation system in Spore was particularly interesting in the way it can map and combine artist created animations onto user generated rigs. We also got a taste of Natural Motions 'dynamic motion synthesis' system in games like Star Wars: The Force Unleashed and GTA IV. The system is essentially a 'better rag doll' that emulates muscle response on a skeleton through custom AI controllers to give more realistic motion - although creating the controllers in the first place seems to be a bit of a dark art. Also games like Far Cry 2 and The Outsider hold the promise of 'procedural narrative' but I have much more to say on that subject later.

It looks like there will be 'interesting times' ahead for 2009, so hope you all have a Happy New Year, and good luck ;)
read more...

Personal Software Process

Largely, I have an affinity towards agile methodology and thinking with respect to programming. Especially for the kind of development I do (games) where requirements are rarely fixed for long. So I was very interested when I came across "Personal Software Process" from a talk[1] at GDC[2]. Agile processes are fine but they are only one side of the coin. For a more complete and rounded view of the whole development vista I needed the view from the other side of the fence. I have experimented with PSP over the past couple of projects and tried to collate my notes and experiences into some kind of coherent form below.



Finding a description of PSP online is actually quite tricky. Minor googling suggests most PSP related sites are trying to sell you some kind of course or book[3]. So I will try and summarise my (albeit dodgy) understanding of what PSP is, for the benefit of those of you who can't afford all those expensive courses ;)

PSP is about improving your own personal development processes. It starts with the premise that as a conscientious professional programmer you should naturally take it upon yourself to identify and rectify areas for improvement. In much the same way an athelete or sportsman would monitor their performance and tweak their training routines accordingly, a practitioner of PSP monitors their coding activities and makes adjustments in the way they work. This is the basic underlying tenet.

The question is what should we monitor and how? To save a long arduous discussion I will jump straight to the conclusion - time and defects. Time (and interruptions) are tracked against tasks in a straightforward way. Tracking defects (bugs) is more involved as you need to record when the defect was injected, when it was detected and when it was resolved - as well as what it was. PSP comes with many 'scripts' to follow for each part of the process, I guess to ensure that you do not miss a step and that the process is repeatable.

In contrast to agile processes, PSP is a high disciplined technique producing a lot of empirical data for analysis. This is a much more scientific approach; observe, collect statistics, adjust the model and repeat. Monitoring what you do, rather than what you think you did or what you were scheduled to do gives you a true reflection of the situation. This objective approach is much more practical whereas other methods potentially can degenerate into arguments about the theoretical and hypothetical. To use a programming metaphor, the approach is akin to optimising code - first you profile, then you make adjustments and profile again to see if it had any noticeable impact.

I experimented on and off with tracking time and tracking defects. In the end I decided to concentrate on just tracking time for the purposes of this experiment. My methods may be slightly different to the 'official' PSP scripts but the results are the same in essence. As most of my time is spent in front of the computer or in meetings then it made sense to use an excel spreadsheet to record the times. The advantage being that a lot of the leg work of consolidating and summarising the data can be automated. There is one file for each week which contains one sheet per day plus a weekly summary (which is automatically created from the daily logs). A final spreadsheets collates monthly summaries from the weekly logs. Process overhead is reduced to the task of logging start and end times (plus interruptions) for each task and is therefore minimal.



Whilst I reduced the process overhead to a minimal, it still took a few weeks to get used to the discipline of tracking my time. In the absence of a real stopwatch I used a virtual javascript one to track interruptions. However as the interruption timer was often under a lot of other windows on my desktop I would occasionally forget to start it and provide estimations instead. I improved the spreadsheets and process over time but there are still areas where they can be refined.

I found the usefulness of the whole process depends solely on what and how you track tasks. Deciding what constitutes a task is not necessarily as obvious as you might think. If you track everything in minute detail the process and discipline overhead increases and you just end up throwing it all away when consolidating the results. Whereas if you generalise too much then you may not have the necessary information required. The place to start I found, was to ask yourself what kind of questions you want answer about how you spend your time.

Initially I was interested in how my time was split between general administrative tasks (emails, meetings, etc), lead duties and other managerial tasks (e.g. speaking to the team) and doing 'actual work'. I was also interested in how long particular tasks took for future reference (such as how long I spent creating a milestone build, how long did I spend writing the TDD, etc). Periodically I adjusted how I logged tasks as the project progressed and the questions changed. For example, I became interested in the additional overhead of people being offsite - so I split what I classed as a generic 'admin' task, into 'admin' and 'admin external' which covered anything I felt was stuff I was doing to support people who were offsite that would not be necessary if they were onsite. If in doubt I would log a task under any description and generalise it when you collate the results into a summary at the end of the week. This saves time deliberating about how to log a task during the day and defers the decisions to a more appropriate time.



Choosing what tasks to log your time against is a personal decision. Every project is different and every programmers role within it is different. There is no generalised way to determined what should be tracked and what should not ... but then again, this is meant to be a "personal software process" after all ;)

The other thing that is logged is 'interrupts'. This is not a task as such but a distraction from the current task. Interruptions should not be overlooked as they can mount up to one or two hours a day, which means one whole day of interruptions over the course of a week. This is often forgotten when scheduling - so having some hard figures is useful. Different people will have different interruptions - for example, as a lead I expect to devote a significant portion of my time to interruptions. Similarly I expect this to grow as the team size grows. Also different environments will produce different interruptions (open plan vs individual rooms for example). Tracking interruptions is important and can highlight areas for improvement in environment and be taken into consideration when scheduling. Initially I was logging everything (going for coffee, toilet, screen breaks, people asking questions). In the end that did not show any useful information. Actually all my real interruptions are 'lead' related. So interruption tracking shows me additional time I spend doing lead stuff (I track 'scheduled' lead stuff separately). Occasionally other kinds of interruptions do creep in, but not enough to be significant. If I were to track more than one type of interruption I would probably need two stopwatches ;)

It is easy to look at the log and see very accurate information. Actually there is (quite naturally) a certain amount of error in the statistics. There are a few places where error can creep in. Sometimes I forget to clock interruptions, sometimes I start and new task and then go back and put in the time a few minutes later, also sometimes I am working on two or three tasks simultaneously and so it is difficult to proportion my time effectively. In these cases I end up providing estimations. I would like to think these estimations are reasonably accurate but it is difficult to know for sure. However, given that errors are likely to be in the magnitude of minutes, whereas the figures we are interested in is in the order of hours or days then I do not think they are significant at all. On the contrary, I think the figures are more than accurate enough for how they are used.

I do not think this process would work in all circumstances. It is easy to track time if you work on single tasks for a few hours at the time and you are at your computer. If you are away from your computer a lot, or spend a lot of time in "interrupts", or are constantly working on multiple tasks at the same time then tracking your time in this way would prove difficult. This is a technique that suits certain people doing certain types of work and should not be applied universally to all projects or roles without consideration.

But why do any of this at all? What is the point of using this high discipline process, other than to provide interesting statistics? Like any other tool or technique it has its uses and is not applicable in all circumstances.

What it does is show you how you actually spend your time, rather than what was scheduled or what you think you spent your time on. It highlights problem areas that you may not have been aware of (for example, how much time is spent in meetings or being interrupted), and gives you hard evidence so you can do something about it. It also gives you figures that can be fed back into scheduling. It allows you to accurate give answers to scheduling questions, rather than vague guesses (for example, as a lead how much buffer time should I allocate? - normally this is just a vague 50%, but actually I can show that it is closer to 60%-80% for the current project, I can also see if it changes as I expect it to as the project picks up speed - in which case I can take on more 'real work'). It would also let me compute an accurate task velocity (time taken / estimated time) as well as other trends as the project progresses.

I found once I started analysing how I spent my time, and trying to answer questions about how I will spend my time in the future, that it became apparent what I needed to log and what I did not. The first few weeks I logged lots of different tasks, when I came to collate them I would lump individual tasks into one category or another to get meaningful results. For example, lots of 'bitty' coding tasks were lumped under a "General Coding" task. The end result is a general break down of how I spend my time. The choice of 'categories' was a very personal one.

The results were interesting and undoubtedly useful. There is definitely something to be said from analysing your performance and feeding that back. Whilst initially the discipline took some getting used to, once you are past that the overhead of the process can be tiny. The real benefits are realised after a few months when you have a decent amount of statistical data to feedback into the process. The biggest benefit from collecting the data from this project will come when it comes time to schedule the next one.

The data produced by this process is certainly useful when scheduling projects and from an individual point of view can improve task estimation. However, the success or failure lays very much in the hands of the practitioner and the nature of the work that they do really needs to suit this type of process. It would be interesting to try it for a whole team providing you can get the appropriate "buy in" to make it work.

Despite the success I have had with it I am still dubious that the process will have a big impact on any one individual project for the type of projects I work on (i.e. computer games), other than to make task estimate (and hence scheduling) more accurate and to improve the visibility of the project. There are other approaches[4] which have similar improvements which are maybe more suited to games development.

---

[1] Quality of Life through applying Software Quality Management
http://www.igda.org/qol/IGDA_2005_QoLSummit_Case-Saulnier.ppt

[2] Game Developer Conference
http://www.gdconf.com/

[3] "Introduction to the Personal Software Process", by Watts S. Humphrey
http://www.amazon.com/Introduction-Personal-Software-Process-sm/dp/0201548097/

[4] Agile Development for example

read more...

Been a While

It's been a while since I've had time to write anything down ... But hey, maybe I'm a real blogger now :P Actually I've been too busy to articulate any useful, which is a good sign I think. In fact even this little missive has been interrupted several times. Which is the weird paradox of blogging (wrt programming at any rate), the people whose opinions I trust and have insightful things to say are the people who are in the trenches experiencing the verisimilitude. Conversely, intellectual musing, whilst a useful exercise have no legitimacy unless grounded in a healthy dose of reality.

So it would appear you either have something worth saying but are too busy to say it or, have plenty of time to say stuff but nothing worth saying ;) Which brings me to my point :-

"Never trust anyone who talks a lot"

Um, which kind of disproves myself or something ... I'll shut up now ;)

read more...

Comments

Comments are one of those things I tend to forget about until someone brings it up. I have heard many exaggerated claims about comments dramatically improving a project or how they are essential for success on large projects. I am not sure how big a project needs to be to be considered 'large'. 500k LOC? 1M LOC? That is a conservative estimate for the size of some projects I have worked on and they seemed pretty big. So based on my experiences, are comments really that useful? And what makes a good comment anyway? Should I comment every function? I thought it was about time I had a little rant. It has been a while ;)


Firstly, comments are documentation. Writing any kind of documentation you need to know who your audience is. Writing a user manual is different from writing a business pitch or writing a programmers reference. Fortunately in the case of comments your audience is easy, it is other programmers ... or more likely than not it will be you looking back over some piece of code that you wrote several months ago and can not for the life of you remember what the heck you were doing back then. So I take the view that when I write comments I am writing them for myself. And as I am probably the best judge of what I would like to see then in theory comments should be easy. The problem is, no one likes writing documentation.

There are obviously bad comments, for example ..

    i++; // increments i

    GetSomeValue(); // gets some value

    sprintf( str, "%i", x ); // format the string 'str' with a number

I know what the code does and if it is a library call that I am not sure about I can always look it up. Professional code is not the place for a coding tutorial. Equally, overly verbose comments are bad. Generally I like my documentation to be short and to the point and that is equally true in comments. Several pages worth of comment about a function, how it is implemented and how it is meant to be used quite frankly is not going to be read.

Self Documenting Code

A professional programmer can read code. It is kind of essential really for the job. If I want to know what a function does, or how it does something then I can take look. Comments may not be updated when the code changes anyway so the best place to find out what is going on is with the code itself. With a clear style, sensible variable and function names the code is self documenting, there is no need for comments to explain all the ins and outs of a particular function.

Why, not what or how

The code tells me what is going on, and how something is implemented. What it can not tell me is why something is happening. Comments should add value to the code, tell me something the code can not. Therefore a short description of a class or set of independent classes from a design perspective can be useful. Also, any caveats which come with the code, like a work around for a bug should be commented to stop anyone from accidentally removing them. Optimisations in the code can often look a little odd, and therefore a comment is useful to let you know why something has been done a particular way.

However, there are times when I do like to put comments in telling me "what" or "how". Some functions are large and have several steps to them. Often I find a quick comment at the top of each step useful as it makes skipping through the code easier and quicker. It separates the blocks of code into meaningful chunks. For example ...

    // initialise stuff

    ...


    // do stuff

    ...


    // clean up

    ...

Also it occurs to me that some languages might require more comments than others. Particularly assembly like programming languages (such as VU code for the PS2 or pixel shader code for graphic cards). Assembly is not the easiest thing to read so some comments explaining what is going on is much more useful here.

Commenting before writing code?

There is a school of thought that you should write the comments first before you write the code. This is a good technique if you are dealing with a particularly complex algorithm or are just not sure how you want to implement something. It is like sketching a diagram out on a white board or paper. It has a place in your programming arsenal of tricks but the majority of code is not as complex and I find I rarely need to sketch out individual functions in this way.

Finally I want to cover how comments are sometimes used in other ways.

File headers

People like to add a comment header block to the tops of files. Often containing name of the file, author, license etc. Some source control programs (such as SourceSafe) support this and will insert source control information (checkin comments, time of change, who made the change etc). Open source software often has the GNU license at the top. Personally I find this kind of thing annoying as it is just stuff to scroll past to get to the code (in 'popular' files with 100+ changes this can be several pages).

In the case of distributed code (such as open source) then having the license is probably important and it is useful to have the email address and name of the author if you need to contact them. In large companies then again maybe it is useful to know who is changing what. However the information is also accessible from your source control program so in my opinion unnecessary. Of course you could be looking at a project several years later when the source control is not available. However asking someone about the code then is probably not that useful as they might not be able to remember what they did. Maybe it would be better to have some contact information with the project generally rather than in each file.

AutoDoc Comments

Or possibly "meta comments". Specially formatted comments which are picked up by a documentation generation program such as doxygen, autodoc, phpdocumentor and the like. Often look something like ...

    /**
     * A description of some function
     * @param p some parameter
     * @return some value or other
     */

If creating documentation for your project is important then this is a good way to do it as it allows programmers to change or add documentation as they change the code. For example, a function library which is going to be used by another team would be a good candidate. However for most applications external documentation is not as useful and the extra hassle of special comments out weighs any benefit they have.

However I was particular impressed with C#'s XML documentation, which looks like ...

    /// <summary>
    ///
    /// </summary>
    /// <param name=""></param>

The advantage of these meta comments is they integrate directly with Visual Studio. Creating them is easy as Visual Studio does all the hard work for you by automatically working out what comments are needed. Once you have written them they integrate with the intellisense/autocomplete feature which is very useful. Whilst I do not normally write in C# I do find myself using this feature when I do.

Other Comment Uses

I like separators between functions or sections of the file, by which I mean things like ...

    ////////////////////////////////////////////////////////////////////////////////

If only because they let me skip through large files quickly. I can see immediately when one function ends and another begins. I am not a fan of code outlining (where functions and blocks are collapsed into expandable sections). I am old school, so I like to see all the code in one go ;) Separators make that easier. Maybe if I get the hang of outlining then they will not be as useful.

TODO's are useful comments as well. Sometimes you leave bits of code that need writing or assumptions you need to address later.

    // TODO: save the config data to a file

So long as you always use the same word (so TODO say, and not mixed it with FIX or BUG) and keep the comment on the same line then a quick grep of the code can reveal anything you may have missed out.

Some people like to comment changes they have made. Particularly if it is in a file they are not responsible for.

    // GW 2005-12-08: fixed a bug in the file handler - null pointer not being checked for

Whether these add any value is debatable. After all, you can always diff a files history from source control to see what changes someone made. I would rather people did not bother with these as it just gets in the way of reading code.

In summary. Having a good programming style with sensible names means you do not need to rely on comments that often. Write comments for yourself that you would find useful, because it will probably be you who needs to maintain the code several months later. Comment any design decisions, or anything unusual which might catch you out later. If you find using comments useful to separate blocks of code then do it. And do not listen to anyone who tells you comments are some kind of idealised magic bullet for saving the world. If my comments are sparse it is not because I am lazy (well possibly), I have my reasons.
read more...

Good Code

We probably all like to think we write good code; so therefore, it follows that we like to think we know what good code is. But what is good code? Are there rules for determining how good some code is, and if there are rules for good and bad code then why doesn't everyone write good code? Why aren't these rules taught in colleges and universities? The problem is it is a difficult question to find an answer for which is both succinct and complete. For a start the question is vague[1], you may as well ask what is art or love.


A better place to start might be if ask "What is bad code?". Now this is a much easier question to answer ;) If I don't know what good code is certainly know what bad code is. We have all worked with 'bad code', and probably been guilty of writing some :o I have worked on more than my fair share of projects with bad code[2] and equally I'm sure I've written more than my fair share myself :/

So using my vast knowledge and experience of bad code as a reference, I have compiled a list of common faux pas :-

• Style - bad code is hard to read
  
  Bad style is one of the most immediate coding horrors. Whilst what people consider 'good style' is largely subjective, there are definitely bad styles and bad habits as aptly demonstrated by the many obfuscated code contests.
  
  Conversely, good style is easy to read and understand. The intent of the code should be obvious and immediate and apply the Principle of Least Surprises. Maybe worse than bad style though is inconsistent style as it can potentially cause problems for 'grepping'[3].
  
  
  
• Design - bad code is badly designed
  
  Bad design complicates changes, bug fixes or enhancements. Bad design may not be intentional; as projects grow there is a tendency for them to turn into a big ball of mud[4]. Whether intentional or not, some common 'design' related difficulties are ...
  
  
  
  
  • logic duplication or 'de-coupling' - meaning you need to change many files in many different places to make simple changes
    
    
  • intricate dependencies amongst files and objects which results in classes and code being difficult to re-use or refactor.
    
    
  • over complication - interfaces and objects that have an unnecessarily high overhead in implementing.
    
  
  
  Good code should be well designed. It should implement things in the most straight forward and accessible way. Interfaces and 'ownership' should be intuitive. Code and 'logic' should be modular and coupling avoided where possible.
  

  
  "A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.",
  Antoine de Saint-Exupery, Aviator.
  

  
  
  
• Execution - bad code does not work
  
  No matter how good the design or the style, the code needs to work. Confidence is undermined if the code is riddled with bugs. Buggy code is harder to fix, as whilst you might think you are fixing one bug you could just be fixing the symptom of another. Additionally programs that behave erratically due to bugs (such as memory corruption caused by a buffer overflow or dangling pointer for example) are more difficult to debug than a simple logical error.
  
  Code should work. All known bugs should be fixed.
  
  
  
• Specification - bad code is not suitable for its intended use
  
  'Good' of course, is a relative word. What makes something good in one context does not necessarily follow in another. Different projects and platforms have different requirements. It is no good writing a master crafted opus of software engineering if it won't fit into memory, executes too slowly or fails on stability, safety or any other criteria.
  
  
  
  
• Stability - bad code is fragile
  
  Code should be robust. Slight changes to data or the code base should not cause the program to crash. Errors should be handled gracefully.
  
  
  
• Debugging - bad code is difficult to debug
  
  Here's an example ...
  

  
          mpLayer = mpScreen->AddLayer(
              FECK2::CLayer::Create()
                  .Name( "NonmodalControlLayer" )
                  .Depth( 10000 )
                  .EventDeclaration( new CLEFunctor( this, &FECK_LAYER_EVENT_FUNCTION_NAME( CFENonmodal, EE_DECLARE_HANDLERS ) ) ) );
  

  
  
  Not only is this an unusual and quirky way of initialising an object but makes it difficult to step into any one of the individual functions. At first glance it is not obvious where the execution will go. Generally, function calls in chains ( id est x()->y()->z(); ) should be avoided.
  
  Another debugging nightmare is objects or data that are obscured by some abstraction. For example, void pointers, CRC's instead of strings and opaque interfaces (COM?) can make identifying objects in the debugger difficult or impossible.
  
  Debugging is an integral part of programming and good code should not make it difficult.
  
  

As a list of 'bad things' you can do with code I think that covers all the important bases. Certainly many of the particular horrors that I have encountered. So maybe the answer to 'what is good code' is just 'not bad code'.

Corollary, from our list we could define good code as :-

• well presented
  
• having a good design which is as only as complex as it needs to be
  
• easy to enhance or repurpose
  
• fulfilling its intent within the given constraints
  
• stable, robust and with no known bugs
  
• friendly to all the usual tools we use to work with code (debuggers, grepping, etc)
  

This is probably an acceptable (and maybe even workable answer[5]). Following these guidelines we can be pretty sure of not writing any bad code. But as an answer to the question of 'what is good code' it is not very satisfying. It feels there should be more to the art of programming than just following a set of do's and dont's. Not writing bad code should be the norm - and therefore 'average' code. There must be something else that separates good code and lets it shine above the rest.

In mathematics we have the notion of 'elegance', which in my view can be equally applied to code. In fact, if we search the wikipedia for elegance we find the following quote ...

"The proof of a mathematical theorem is considered elegant if it is surprisingly simple yet effective and constructive; similarly, a computer program or algorithm is elegant if it uses a small amount of intuitive code to great effect."

http://en.wikipedia.org/wiki/Elegance

And this to me captures the essence of what separates good code from the reset. Good code is elegant, beautiful and a joy to work with. It solves difficult and intricate problems in simple, clever and intuitive ways.

But as a practical answer it is probably not very useful. It is not always obvious why a piece of code is good or elegant without an appreciation of the problem and the many bad solutions that have been applied. Elegance isn't something that can easily be defined or something you can achieve by just blindly following a set of rules. But then again, it's probably fitting that the answer is just as vague and nebulous as the question ;)

---

[1] Naturally I can not possibly be at fault, blame for my inability to come up with a suitable response must lie squarely with the question ;)

[2] In fact some I would go as far as saying had 'horrendous code'. I remember one project which was using goto's to jump in between cases in different switch statements *shudder* ... I still have nightmares ;)

[3] The humble 'grep' (or "Find in Files" for VC users), despite the many advances in IDE's is imo still one of the most useful tools available to programmers. So therefore good code should always be 'grep' friendly imo.

[4] Big Ball of Mud, Brian Foote and Joseph Yoder, Department of Computer Science University of Illinois, http://www.laputan.org/mud/mud.html

[5] Workable in the sense of gleaning some kind of metric or as a basis for objective code critique.
read more...