Monday, February 19, 2007

A Source of Conflict During Agile Adoption

Many organizations try to adopt agile development by plugging in a set of engineering practices without changing the organization's management style. Fortunately, depending on your starting point, some big gains can be made by adopting more effective engineering practices like test-driven development and continuous integration. Unfortunately, to make any big leaps in productivity and predictability, you also need to change the way you manage projects.

By only implementing low-level changes, and avoiding changes that have broader impact, organizations generally don't see the improvements they are hoping for (although they often see just enough improvement to become complacent with their new half-agile method of development).

This is a specific example that illustrates a broader point: there are two ways of solving any given problem:
  1. Work more effectively withing your current context.
  2. Change your context.
As in the above example, you can see some gains by working more effectively within your context, but to have major breakthroughs you usually have to change your work context.

Here's the killer: changing your work context in a large organization can be extremely, extremely challenging. When it works, it often takes a lot of time and effort to get the ball rolling. Because of the difficulty in bringing about context change, many people give up on trying to do it at all. They think that it isn't worth the effort, or almost forget it's even an option. (If you've heard the words "that won't work here" or "that's corporate America, learn to live with it", you probably know what I'm talking about.) Truth be told, often they are right. However, that doesn't stop those of us who believe strongly there is a better way of doing things. Sometimes, it is worth it.

I recently realized that this conflict of approaches to problem-solving is the underlying cause of a lot of conflict between agile evangelists and others on a team struggling towards agile development. Here's how it usually happens:

The team is faced with some particular obstacle, and gets together to figure out how to proceed. Let's say the team has to jump through some bureaucratic hoops to deploy a new version of the software to production. As a result, releases incur a lot of overhead, making frequent releases impractical. A few people make suggestions that minimize amount of time or effort it takes to jump through the hoops. The team begins discussing the benefits and drawbacks of each. An agile evangelist realizes the problem isn't an essential one, but one caused by the current work context. Frustrated with the short-sightedness of his teammates, he expresses his dissatisfaction with all the suggestions so far, and asks "Why do we have to do this paperwork every time we release anyway?", and tries steering the conversation towards strategies to change this aspect of the work context. Less idealistic members of the team realize this won't solve the problem immediately (if it's possible at all), and reiterate their own suggestions. A heated argument ensues.

The problem is neither side feels listened to. They are attacking different aspects of the problem, so there is no actual conflict between them. Both solutions could be applied. Knowing that it will probably take time to change the context, there is probably some benefit to working more effectively within the context in the meantime. Similarly, even if you can reduce the inconvenience of working within the current context, if it is possible to eliminate the inconvenience entirely by changing the context, it's probably worth trying to do that too.

But, because each side is focused so completely on their view of the problem, they don't acknowledge the other's suggestion as useful. Fortunately, I think this is one of those problems that is half-way solved once you're aware of it. Remember that you are focusing on a different way of solving the problem than someone else, and that they may both have benefit. Try to remain aware of how you share your perspective (and for that matter, focus on sharing your perspective, rather than making your point). Practice saying "Yes, and..." instead of "Yeah, but...".

Often agile evangelists and other change agents shoot themselves in the foot by letting their enthusiasm lead the way. This will leave others feeling unheard, which sets up conflict and resistance. Remember, organizational change is an emotional domain, not necessarily a rational one. Before you can understand and deal with others' emotional reactions, you have to understand how your emotions drive your behavior, and how this contributes to the reactions of others.

8 comments:

Daniel De Aguiar, P.h.D said...

Your example is indicative of a crucial conversation that was not handled as well as it could have been by either party. Interestingly, After reading the booking Crucial Converations and even blogging about it, I still find it hard to separate myself from my emotions and focus on my goal when I encounter such situations. I am finding that it takes practice and patience.

xmlblog said...

Another insight is that perhaps the wrong questions are being asked. Typically, any of us can work ourselves into a state of frustration by asking ourselves questions like, "Why do we have to fill out this stupid paperwork every time we need to do X?" The question is a bad one because of the emotional state it puts you in immediately. Right off the bat, one can't help but feel terrible about the situation especially given the embedded supposition that the paperwork is stupid. The incredible thing about the human brain is that it is the only computer that will generate an answer to *any* question it is asked. Whether or not you have enough information to come up with a logical, empirical answer is irrelevant - the brain will make up an answer to your question! Choosing the previous question as an example, it wouldn't be surprising if the little voice inside your head automatically came up with something along the lines of 'Because some idiot with (insert dubious motive here) is making us waste our time with this nonsense.' This is just the way the brain works.

Great problem solvers (and conflict resolvers) invariably approach situations like this by asking a better set of questions. The next time a situation like this comes up, try this experiment. Ask yourself (or everyone in the room!) the following questions:

* What's great about this that I hadn't noticed? (Be prepared for a pessimistic answer like 'nothing' and counter with: 'Of course, but what *could* be great about this if I wanted it to be?')

* What isn't perfect about this situation yet? - Notice the difference in the suppositions. First, I am striving for perfection while acknowledging we're not quite there. Second, the word 'yet' supposes that we will eventually get it perfect.

* What am I willing to do to make this situation exactly how I want it?

* What am I willing to do no longer to make this situation how I want it?

* What is the *best* way to make this happen *and* enjoy the process? The second part of this question is crucial. It's not enough to come up with a great solution that you won't be excited to implement. Such 'solutions' are doomed to fail because you'll never muster the energy and commitment required to see the task through to completion when the inevitable obstacles present themselves.

Lastly, for each challenge (aka obstacle or problem) that arises along the way, repeat this process.

I'm anxious to hear how this works for you!

Ryan Cooper said...

Thanks for your comments, xmlblog (BTW, your profile is not shared; I was hoping to browse your blog after reading your comments). I agree, it's very easy to become more frustrated than necessary by focusing on the negative aspects of a situation.

Our team already asks some of these questions collectively in our bi-weekly retrospectives. Generally, we discover what people feel strongly about, then try to find out what people would like to start doing or stop doing to create a more enjoyable situation. A requirement for action items is that someone must feel energized to take them up.

This seems to work pretty well, although obviously it does not always diffuse our frustration. Sometimes, as a result of asking these questions, we end up in a conversation like the one I wrote about. Different people voice different ideas about how to deal with the pain; some lean towards an intra-context fix, some want to change the context. This is a good thing because it means we are getting somewhere, but I feel we need more tools to ensure that we can come to some accord after we have reached this point. :)

I like the first question. I think it may be challenging to ask this one effectively, but I will think it over some more. One issue I see is that there are clear 'good' reasons for certain aspects of our context, but they are good reasons for someone else, not for us. For example, some paperwork essentially exists to help certain people cover their ass (which is certainly good for them). In cases like this, another useful question might be "What valuable information does this problem/pain/frustration tell us?" In this case, it might tell us that we need to build a more trusting, collaborative relationship with some external groups.

I don't think I like the second question, simply because I don't believe in reaching perfection. There will always be room for improvement, and I don't want people to get into the mind frame of "once we fix this one thing, we'll be perfect in that regard, and won't have to think about it / improve it anymore!" Continuous improvement and perfection cannot co-exist. However, we do ask a variant of this question in our retrospectives, worded differently.

Again, thanks for the insightful comments!

xmlblog said...

I agree perfection is a goal that seems perpetually out of reach. The real reason for wording it that way is to diminish the negativity of the words. Really saying something is not perfect admits an issue without generating an overly negative emotional response. I agree that while I believe the structure of the question is useful for helping oneself maintain an overall positive state, it may not work well in a group context. Perhaps a better phrasing might be, what about this situation is a great candidate for improvement? Again, this question focuses on the positive aspect - improvement rather than generating a negative emotional response such as the one you would get if you asked yourself What about this sucks? :)

I'm honored you'd like to read my blog. I've got some cookie issue preventing me from fixing the blogger profile atm, but my name is Christian Romney and if I'm not mistaken we may be working on the same project! ;) I've been an inactive blogger over the past few months, but my personal blog is at http://www.xml-blog.com and my company website (new blog coming soon) is at http://www.kipdigitalmedia.com Drop by anytime!

Anonymous said...

A more interesting question often is, how to even get to a point of adoption. Just creating enough acceptance for there to be adoption is extremely hard.

free ps3 said...

Thanks for the nice post!

Kenley William said...

A small team always participate and a good no of feedback can be collected from all the team members which may not happen in large teams. So, the project manager should be a scrum certified, who can better handle the team size. Get yourself scrum certified from http://www.scrumstudy.com which is a globally recognised certifying body for various certifcations such as scrum certification and Agile Certification

Wright Williams said...

More and more companies are trying to get nimble to enable them to respond to change with agility. Over the years, there has been a clear shift in momentum about the ways how companies manage projects. So, the project manager should be a PMP certified, who can better handle the planning, execution, and closing of any project. To get yourself prepared for PMP Certification, http://www.pmstudy.com is the best source.