A Source of Conflict During Agile Adoption

Many organizations try to adopt agile development by plugging in a set of engineering practices without changing the organization's management style. Fortunately, depending on your starting point, some big gains can be made by adopting more effective engineering practices like test-driven development and continuous integration. Unfortunately, to make any big leaps in productivity and predictability, you also need to change the way you manage projects.

By only implementing low-level changes, and avoiding changes that have broader impact, organizations generally don't see the improvements they are hoping for (although they often see just enough improvement to become complacent with their new half-agile method of development).

This is a specific example that illustrates a broader point: there are two ways of solving any given problem:

1. Work more effectively withing your current context.
2. Change your context.
   

As in the above example, you can see some gains by working more effectively within your context, but to have major breakthroughs you usually have to change your work context.

Here's the killer: changing your work context in a large organization can be extremely, extremely challenging. When it works, it often takes a lot of time and effort to get the ball rolling. Because of the difficulty in bringing about context change, many people give up on trying to do it at all. They think that it isn't worth the effort, or almost forget it's even an option. (If you've heard the words "that won't work here" or "that's corporate America, learn to live with it", you probably know what I'm talking about.) Truth be told, often they are right. However, that doesn't stop those of us who believe strongly there is a better way of doing things. Sometimes, it is worth it.

I recently realized that this conflict of approaches to problem-solving is the underlying cause of a lot of conflict between agile evangelists and others on a team struggling towards agile development. Here's how it usually happens:

The team is faced with some particular obstacle, and gets together to figure out how to proceed. Let's say the team has to jump through some bureaucratic hoops to deploy a new version of the software to production. As a result, releases incur a lot of overhead, making frequent releases impractical. A few people make suggestions that minimize amount of time or effort it takes to jump through the hoops. The team begins discussing the benefits and drawbacks of each. An agile evangelist realizes the problem isn't an essential one, but one caused by the current work context. Frustrated with the short-sightedness of his teammates, he expresses his dissatisfaction with all the suggestions so far, and asks "Why do we have to do this paperwork every time we release anyway?", and tries steering the conversation towards strategies to change this aspect of the work context. Less idealistic members of the team realize this won't solve the problem immediately (if it's possible at all), and reiterate their own suggestions. A heated argument ensues.

The problem is neither side feels listened to. They are attacking different aspects of the problem, so there is no actual conflict between them. Both solutions could be applied. Knowing that it will probably take time to change the context, there is probably some benefit to working more effectively within the context in the meantime. Similarly, even if you can reduce the inconvenience of working within the current context, if it is possible to eliminate the inconvenience entirely by changing the context, it's probably worth trying to do that too.

But, because each side is focused so completely on their view of the problem, they don't acknowledge the other's suggestion as useful. Fortunately, I think this is one of those problems that is half-way solved once you're aware of it. Remember that you are focusing on a different way of solving the problem than someone else, and that they may both have benefit. Try to remain aware of how you share your perspective (and for that matter, focus on sharing your perspective, rather than making your point). Practice saying "Yes, and..." instead of "Yeah, but...".

Often agile evangelists and other change agents shoot themselves in the foot by letting their enthusiasm lead the way. This will leave others feeling unheard, which sets up conflict and resistance. Remember, organizational change is an emotional domain, not necessarily a rational one. Before you can understand and deal with others' emotional reactions, you have to understand how your emotions drive your behavior, and how this contributes to the reactions of others.

Work With the Customer, Not For the Customer

If you think your job is to do what the customer* tells you to do and build what the customer tells you to build, you are at risk of creating this:

We as software professionals have a responsibility to our customers that goes beyond giving them total control. We are responsible not just for writing code; we are responsible for helping to create a useful product.

A few days ago, I was listeneding to an IT Conversations interview with Joel Spolsky. He talks a bit about how XP teams work with the customer role, and makes a few good points:

• "Customers will not invent the great features. They will not come up with the 'big leap' ideas." (Your average music aficionado would never have invented the iPod, for example.)
  
• "A good programmer... will come up with features customers never would have imagined were possible." (due to knowlege of program internals and what's technically possible, combined with understanding of domain and the needs of the user.)
• You will discover more interesting insights when you say "Tell me about your job and how you use [our product]" rather than ask "What features should we do?"
• When you ask that second question, "... you get customers asking you for features that seem like obvious features to ask for but which they're never going to use or care about or need."
  

I think what Joel is getting at is that often the customer on an XP team is someone with operational experience, not product development experience. Product development expertise is needed to create a coherent, useful, and elegant product, but teams new to agile development sometimes neglect to make use of their experience to help the customer do this. Sometimes, we throw out the heavy up-front planning and design, but neglect to replace it with a deep level of collaboration with the customer throughout the development effort. This often results in a product without a coherent vision. Users may not find this product useful, but may not be able to articulate what's wrong with it.

We are all experts at living in homes, but if you designed and supervised the construction of your own home without advice from architects, engineers, and construction experts, you probably wouldn't be very happy with the result. Likewise, if you do not help shape the vision of the product, you'll likely build something your customer and users aren't very happy with (despite it being exactly what they asked for).

Unless your customer has significant product development experience, the expertise of both parties is needed when shaping the vision of the product. (I'm assuming here there is a business analyst, markitect, usability expert, or someone else - hopefully several people - with significant product development experience on the development team, and that the customer is a domain expert with a keen understanding of the problem the product is meant to solve and the people who will be using it.) As my colleague Dmitri Dolguikh recently remarked:

"[Software development] is not about the customer asking you for something and you building it for them. It's about you and the customer working together to figure out what it actually is that they need."

Responsible software development means working with your customer to create the best possible product, not working for your customer and abdicating all responsibility for the product being built.


* Please read this as "Product Owner" if you prefer Scrum to XP.