#2782

Takeaways from Agile 2006

As some of you may know I’ve been at Agile 2006 for the last week. So while I was flying back I thought I’d write it up and pass on some stuff I learnt. These are an edited version of the notes I sent out to the VSTO Team.

Some general things can be said about the state of agile software development on the basis of the ideas and results presented at this conference.

• Agile methodologies are starting to a gain significant foothold in software development. Of 4200 developers* surveyed 41% said that they were using some form of agile methodology and 65% were using some agile practices. Personally my gut feeling is that these numbers are too high however the conference attendance figures speak for themselves. Agile 2006 boasted 1200 attendees up from a few hundred attendees five years ago, next year they are planning for 1800. Over half the survey respondents also had seen positive improvements in productivity and quality as a result of their adoption.
• Agile methodologies are moving out of the “small project” domain and into large, complex projects in Fortune 500 projects; e.g. Microsoft, Fidelity, read the Primavera white paper.  If your project is too large or too complex for agile then the simple truth is maybe it’s just too large and too complex. So said keynote speaker Peter Coffee.
• A second generation of agile methodologies are now appearing that are designed to scale to these larger projects; XP 2.0, Enterprise Scrum and MSF Agile. Practitioners are now coming up for best practices in these larger environments.
• While some agile methodologies are easy to implement they are all hard to make stick. For example only 25% of organizations that implement Scrum will its full benefit, usually when the pain of implementation is less than the existing pain within the organization’s existing process. There still is no silver bullet.
• As suggested by the survey results and paper above there are benefits to be reaped by investing and making these processes and practices stick. I have more case studies here including one of the largest banks on the Pacific Rim which used FDD to rebuild its load processing system.

Overall it was a lively conference with lots of people to talk to. As I really only converse on two topics; climbing and software development. There were no climbers so I ended up talking about software a lot, often late at night after the sessions had ended.

I attended numerous talks and tried to focus on areas which would be of most value to the VSTO Team.

• Agile Methodologies – Typically we’ve been building our own processes so I tried to expand the breadth of my knowledge to increase the size of our agile toolbox rather than focus exclusively on one methodology. I did pay special attention to Scrum and Lean as these are somewhat related to Feature Crews. Feature Crews is the name for our current process, as promised I’ll blog about these at some point. 
• Metrics and measuring – VSTO has an interest in this, largely to satisfy our business need to increase visibility up the management chain.
• User Stories, Estimation and Planning – this seems to be one area where we could get significant value in terms of predictability if we could improve our estimations.
• Coaching – I know I could improve my coaching skills so I tried to get to at least one hands on workshop on this. I consider this important as the VSTO Team does not have a dedicated coach.

I didn’t attend any of the technical development tracks on TDD, pairing, refactoring and the like. Alas, my current role doesn’t allow me to do development day to day so there seemed to be minimal value in focusing on these. Not that this didn’t stop me hanging out with numerous developer types as well as general networking.

Outlined below are some of the key takeaways from specific talks or tracks. The conference web site and Agile 2006 Wiki are also useful.

On balance I thought this conference was hugely beneficial both in terms of content and networking opportunities. I would seriously consider attending next year (DC in August – Ick!). 

—

* Scott Ambler’s (IBM) survey of readers of Dr Dobb’s Journal – a publication for professional software developers. See http://www.ambysoft.com/scottAmbler.html for further results.

Deep Dive

This goes into specific sessions and/or tracks. You can skip to the bit that interests you.

Agile Methodologies

One of the conferences opening sessions was an overview of the six most talked about agile methodologies. In most cases  each talk was given by one of its original architects; XP (Kent Beck), Scrum (Ken Schwaber), Lean Software Development (Mary Poppendieck), DSDM, Feature Driven Development (David Anderson, MSFT) and Crystal Clear (Alastair Cockburn).

Each methodology embraces the four key tenets of the agile manifesto; responding to change, working products, customer collaboration and individuals and interactions although they differ in how they do this and the level at which the prescribe. For example Scrum is a series of lightweight management practices that says little or nothing about how software is developed. XP on the other hand is very developer focused, promoting specific complementary practices including pair programming and TDD. However they all empower the team to inspect and adapt their process over time provided the core values are maintained.

A more in depth comparison of some of these techniques can be found here.

Agile Methodologies – Enterprise Scrum

Ken Schwaber, architect of Scrum, gave an overview of implementing Scrum in large organizations. Many of the same themes from the Scrum course came up again.

• Don’t roll out Scrum from the top down. Encourage your people adopt Scrum and ensure that they have a supportive environment to succeed. Change must be led not commanded.
• When individuals or teams don’t want to follow one of the rules of Scrum it’s usually because there is something to hide.
• Implementing Scrum successfully requires changes at all levels of the organization; team based compensation, co-located teams,
• Don’t modify Scrum, use Scrum to point out the changes and then make them.
• Most of the effort isn’t about Scrum, it’s removing the existing dysfunctionality exposed by Scrum.
• Major change must be managed carefully.
• Be ready for (staff) turnover

And remember…

“Culture eats strategy for breakfast”
    – Ford Corporation

I also went to another talk by a Michele Sliger from Rally Development. This wasn’t as good although it did have a set of ten ideas for implementing agile in a more traditional environment which were worth consideration.

Agile Methodologies – Lean Software Development

Lean Software Development grew out of the pioneering work done at Toyota that lead to the Toyota Manufacturing Process and Toyota Development Process. Lean Software Development focuses on seven principles

1. Eliminate waste – anything that doesn’t add customer value is waste; internal specs, delays, unused features
2. Build in quality – testing should prevent defects not find them
3. Create knowledge – learn from the process and use feedback to improve it
4. Defer commitment – Keep your options open, don’t specify everything up front
5. Deliver fast – competitive advantage, drives high quality production
6. Respect people – teams thrive on trust, commitment, pride and praise
7. Optimize the whole – focus on the whole value stream and deliver a complete product

Mary Poppendieck is a fine speaker and her work meshes in very closely with Scrum. Lean is particularly interesting because of its basis in the already established work of Toyota. Hopefully one of her books should arrive on my desk next week. I think some of its thinking can be applied to feature crews.

Metrics and Measuring

I went to a couple of talks on metrics and measurement. There were a couple of short papers on Earned Value – although the general consensus was that calculating EV, while possibly helpful in communicating with the customer, is probably not applicable to an agile project which inherently does not have a known scheduled baseline from day one.

The key take away I got the discussion of the things we measure can be considered to fall into two categories; metrics and diagnostics. Metrics relate directly to the business’ bottom line, e.g. ROI, profit, customers served. Diagnostics on the other hand measure the process and how it produces – they are used to aid the inspection and adaptation of our process.

Diagnostics are usually typically temporary and are only used until the team no longer finds them useful to drive improvements. This prevents a culture of measurement for measurement’s sake and the overhead that eventually incurs and also makes gaming the metric harder. There may be some semi permanent metrics; bug counts, but even they should not be considered wholly sacred. 

One of the exercises was to create a diagnostic based on a worksheet defining your diagnostic as follows:

• Clear and meaningful name
• Reason to use it – how does it relate to creating value?
• Question answered by this metric/diagnostic
• Appropriate audience level
• Basis of measurement
• Assumptions about data or collection
• Expected trend
• Cost of collection
• When to stop using the diagnostic
• How to game it
• Warnings

User Stories, Estimation and Planning

I went to two excellent talks by Mike Cohn on using user stories to scope a project and then how to estimate the work using planning poker to produce a product backlog sized in story points, or idea hours and then further break that down into product backlog tasks that can be estimated in ideal hours. There was much discussion as to the merits of points over hours, turns out that there are many reasons for using points over hours.

I have one of his books and should order the other one. Rather than summarize his talk here I would urge you to look at the slide decks.

User Stories for Agile Requirements

Agile 2006 Conference in Minneapolis, July 26, 2006
These slides are from a 90-minute presentation at the Agile 2006 conference.

Agile Estimating and Planning

Agile 2006 Conference in Minneapolis, July 25, 2006
These slides are from a half-day presentation at the Agile 2006 conference. The tutorial was given on July 25 and repeated on July 27.

Coaching

There was a “Coaching 101” course which I attended for an afternoon. Personally I don’t think you can do too many of these although they are no substitute for coaching a real team in the company of a more accomplished coach. We spent the afternoon doing situational exercises and reviewing each other’s performance and giving/listening to feedback. 

There were actually a lot of hands on sessions like this at the conference but I didn’t get to go to many of them.