Personal Software Process

Largely, I have an affinity towards agile methodology and thinking with respect to programming. Especially for the kind of development I do (games) where requirements are rarely fixed for long. So I was very interested when I came across "Personal Software Process" from a talk[1] at GDC[2]. Agile processes are fine but they are only one side of the coin. For a more complete and rounded view of the whole development vista I needed the view from the other side of the fence. I have experimented with PSP over the past couple of projects and tried to collate my notes and experiences into some kind of coherent form below.

Finding a description of PSP online is actually quite tricky. Minor googling suggests most PSP related sites are trying to sell you some kind of course or book[3]. So I will try and summarise my (albeit dodgy) understanding of what PSP is, for the benefit of those of you who can't afford all those expensive courses ;)

PSP is about improving your own personal development processes. It starts with the premise that as a conscientious professional programmer you should naturally take it upon yourself to identify and rectify areas for improvement. In much the same way an athelete or sportsman would monitor their performance and tweak their training routines accordingly, a practitioner of PSP monitors their coding activities and makes adjustments in the way they work. This is the basic underlying tenet.

The question is what should we monitor and how? To save a long arduous discussion I will jump straight to the conclusion - time and defects. Time (and interruptions) are tracked against tasks in a straightforward way. Tracking defects (bugs) is more involved as you need to record when the defect was injected, when it was detected and when it was resolved - as well as what it was. PSP comes with many 'scripts' to follow for each part of the process, I guess to ensure that you do not miss a step and that the process is repeatable.

In contrast to agile processes, PSP is a high disciplined technique producing a lot of empirical data for analysis. This is a much more scientific approach; observe, collect statistics, adjust the model and repeat. Monitoring what you do, rather than what you think you did or what you were scheduled to do gives you a true reflection of the situation. This objective approach is much more practical whereas other methods potentially can degenerate into arguments about the theoretical and hypothetical. To use a programming metaphor, the approach is akin to optimising code - first you profile, then you make adjustments and profile again to see if it had any noticeable impact.

I experimented on and off with tracking time and tracking defects. In the end I decided to concentrate on just tracking time for the purposes of this experiment. My methods may be slightly different to the 'official' PSP scripts but the results are the same in essence. As most of my time is spent in front of the computer or in meetings then it made sense to use an excel spreadsheet to record the times. The advantage being that a lot of the leg work of consolidating and summarising the data can be automated. There is one file for each week which contains one sheet per day plus a weekly summary (which is automatically created from the daily logs). A final spreadsheets collates monthly summaries from the weekly logs. Process overhead is reduced to the task of logging start and end times (plus interruptions) for each task and is therefore minimal.

Whilst I reduced the process overhead to a minimal, it still took a few weeks to get used to the discipline of tracking my time. In the absence of a real stopwatch I used a virtual javascript one to track interruptions. However as the interruption timer was often under a lot of other windows on my desktop I would occasionally forget to start it and provide estimations instead. I improved the spreadsheets and process over time but there are still areas where they can be refined.

I found the usefulness of the whole process depends solely on what and how you track tasks. Deciding what constitutes a task is not necessarily as obvious as you might think. If you track everything in minute detail the process and discipline overhead increases and you just end up throwing it all away when consolidating the results. Whereas if you generalise too much then you may not have the necessary information required. The place to start I found, was to ask yourself what kind of questions you want answer about how you spend your time.

Initially I was interested in how my time was split between general administrative tasks (emails, meetings, etc), lead duties and other managerial tasks (e.g. speaking to the team) and doing 'actual work'. I was also interested in how long particular tasks took for future reference (such as how long I spent creating a milestone build, how long did I spend writing the TDD, etc). Periodically I adjusted how I logged tasks as the project progressed and the questions changed. For example, I became interested in the additional overhead of people being offsite - so I split what I classed as a generic 'admin' task, into 'admin' and 'admin external' which covered anything I felt was stuff I was doing to support people who were offsite that would not be necessary if they were onsite. If in doubt I would log a task under any description and generalise it when you collate the results into a summary at the end of the week. This saves time deliberating about how to log a task during the day and defers the decisions to a more appropriate time.

Choosing what tasks to log your time against is a personal decision. Every project is different and every programmers role within it is different. There is no generalised way to determined what should be tracked and what should not ... but then again, this is meant to be a "personal software process" after all ;)

The other thing that is logged is 'interrupts'. This is not a task as such but a distraction from the current task. Interruptions should not be overlooked as they can mount up to one or two hours a day, which means one whole day of interruptions over the course of a week. This is often forgotten when scheduling - so having some hard figures is useful. Different people will have different interruptions - for example, as a lead I expect to devote a significant portion of my time to interruptions. Similarly I expect this to grow as the team size grows. Also different environments will produce different interruptions (open plan vs individual rooms for example). Tracking interruptions is important and can highlight areas for improvement in environment and be taken into consideration when scheduling. Initially I was logging everything (going for coffee, toilet, screen breaks, people asking questions). In the end that did not show any useful information. Actually all my real interruptions are 'lead' related. So interruption tracking shows me additional time I spend doing lead stuff (I track 'scheduled' lead stuff separately). Occasionally other kinds of interruptions do creep in, but not enough to be significant. If I were to track more than one type of interruption I would probably need two stopwatches ;)

It is easy to look at the log and see very accurate information. Actually there is (quite naturally) a certain amount of error in the statistics. There are a few places where error can creep in. Sometimes I forget to clock interruptions, sometimes I start and new task and then go back and put in the time a few minutes later, also sometimes I am working on two or three tasks simultaneously and so it is difficult to proportion my time effectively. In these cases I end up providing estimations. I would like to think these estimations are reasonably accurate but it is difficult to know for sure. However, given that errors are likely to be in the magnitude of minutes, whereas the figures we are interested in is in the order of hours or days then I do not think they are significant at all. On the contrary, I think the figures are more than accurate enough for how they are used.

I do not think this process would work in all circumstances. It is easy to track time if you work on single tasks for a few hours at the time and you are at your computer. If you are away from your computer a lot, or spend a lot of time in "interrupts", or are constantly working on multiple tasks at the same time then tracking your time in this way would prove difficult. This is a technique that suits certain people doing certain types of work and should not be applied universally to all projects or roles without consideration.

But why do any of this at all? What is the point of using this high discipline process, other than to provide interesting statistics? Like any other tool or technique it has its uses and is not applicable in all circumstances.

What it does is show you how you actually spend your time, rather than what was scheduled or what you think you spent your time on. It highlights problem areas that you may not have been aware of (for example, how much time is spent in meetings or being interrupted), and gives you hard evidence so you can do something about it. It also gives you figures that can be fed back into scheduling. It allows you to accurate give answers to scheduling questions, rather than vague guesses (for example, as a lead how much buffer time should I allocate? - normally this is just a vague 50%, but actually I can show that it is closer to 60%-80% for the current project, I can also see if it changes as I expect it to as the project picks up speed - in which case I can take on more 'real work'). It would also let me compute an accurate task velocity (time taken / estimated time) as well as other trends as the project progresses.

I found once I started analysing how I spent my time, and trying to answer questions about how I will spend my time in the future, that it became apparent what I needed to log and what I did not. The first few weeks I logged lots of different tasks, when I came to collate them I would lump individual tasks into one category or another to get meaningful results. For example, lots of 'bitty' coding tasks were lumped under a "General Coding" task. The end result is a general break down of how I spend my time. The choice of 'categories' was a very personal one.

The results were interesting and undoubtedly useful. There is definitely something to be said from analysing your performance and feeding that back. Whilst initially the discipline took some getting used to, once you are past that the overhead of the process can be tiny. The real benefits are realised after a few months when you have a decent amount of statistical data to feedback into the process. The biggest benefit from collecting the data from this project will come when it comes time to schedule the next one.

The data produced by this process is certainly useful when scheduling projects and from an individual point of view can improve task estimation. However, the success or failure lays very much in the hands of the practitioner and the nature of the work that they do really needs to suit this type of process. It would be interesting to try it for a whole team providing you can get the appropriate "buy in" to make it work.

Despite the success I have had with it I am still dubious that the process will have a big impact on any one individual project for the type of projects I work on (i.e. computer games), other than to make task estimate (and hence scheduling) more accurate and to improve the visibility of the project. There are other approaches[4] which have similar improvements which are maybe more suited to games development.

[1] Quality of Life through applying Software Quality Management

[2] Game Developer Conference

[3] "Introduction to the Personal Software Process", by Watts S. Humphrey

[4] Agile Development for example


Popular posts from this blog

Game Development in a Post-Agile World

Reflection on Reflections (Part 1)