Photo by Deva Darshan from Pexels

Introduction

A few years ago, the phrase "We are now working agile" would have caused a surprise. In the meantime, the phrase "we do it classically now" (meaning the waterfall method) is often met with astonishment. Hardly anyone dares to announce in large circles that this project relies on waterfall methods. Why is that so? To explain this I would like to talk about two things in this article:

Why is increasing complexity our current problem?
Why do our (old) methods often not work well in current projects?

Point one: our current problem is complexity

Looking at developments in our world over the last 500 years, it is undisputed that the speed and complexity of our environment is increasing. In my view, there are very good examples of this:

Companies now have to be globally competitive. 300 years ago it was sufficient to produce the best product within a radius of 20 km to be successful. You had no easy access to products from all over the world, because transport, communication and technology didn't make this possible. Whoever wants to run a search engine today has to compete with global corporations such as Google or Microsoft - not exactly easy.

The last economic crisis could only come about because global economic interdependencies had progressed so far that a domino effect could occur. At the same time, dependencies were so complex that we could not predict the economic crisis for all the actions that led to it. Although we can infer the sequence of events in retrospect, we were unable to predict the impact of all our actions.

Nature also offers unprecedented complexity. Who can say what happens when a certain species dies out - the dependencies in the ecosystem are so manifold that it is almost impossible to predict all the effects.

Such examples can also be found in politics: The Brexit is a good example of a change in the complex relationships of different countries - and even years after its initiation (the Brexit referendum was two years ago!) we are still struggling to understand and assess the real dimensions of this decision.

All these examples have one thing in common - complexity is at the heart of the problem. Today's complexity often makes it difficult for us to make the right decisions. But why actually?

The Cynefin Framework:

The Cynefin Framework was written by David Snowden (not to be confused with Edward Snowden, the whistleblower) - It is a situation categorization model designed to support the selection of the right approach.

The Cynefin framework categorizes situations and problems into four categories:

Wikipedia has a good summary of the quadrants of the framwork

„Simple / Obvious, in which the relationship between cause and effect is obvious to all. The approach here is Sense - Categorise - Respond, and we can apply best practice.
Complicated, where the relationship between cause and effect requires analysis, a different form of testing and/or the application of expertise. Here one approaches by means of Sense - Analyze - Respond, and one can apply good practices.
Complex in which the relationship between cause and effect can only be perceived retrospectively, but not in advance. Here is the approach Probe - Sense - Respond, and we can determine emergent practices.
Chaotic, in which there is no relationship between cause and effect at system level. Here is the Act - Sense - Respond approach, and we can discover innovative practices."“

Source: Wikipedia

Zuerst muss man das aktuelle Vorhaben richtig einordnen. Ein paar einfache Beispiele zu den Kategorien:

Simple

Shopping is an easy task. I want to buy certain goods. I have to follow certain actions. (Drive to the supermarket, select goods, pay, drive back.) The procedures are clear and we have repeated the process often enough to apply best practices. A best practice, for example, is a shopping list so that we don't forget anything.)

Complicated:

Building a house is complicated. There are a multitude of requirements and dependencies and it is difficult for us to keep track of everything at once. Nevertheless, the cause and effect in this network is very clear. (e.g. if the electrical system is not installed by date X, the plasterer cannot continue working and the move-in date is delayed). Here a waterfall procedure is recommended. Thus the complicated dependencies can be illustrated and made clear. The procedure here is: Sense - Analyze - Respond

Complex

We have already mentioned examples of complex problems and environments at the beginning - e.g. the world economy or our ecosystem. Here the cause-effect principle is not or only partially predictable. Many of our large IT projects are complex. For example, we have never implemented a comparable project before, or the dependencies on other IT systems and people are so complex that we cannot map all dependencies in one planning phase. Here the Cynefin Framework suggests the approach Probe - Sense - Respond. This approach is inherent in agile methods like Scrum. Something is developed in a fixed time window (sprint) and then evaluated to see what knowledge has been gained and whether the initial plan needs to be adjusted if necessary. (Sense & Respond)

Chaotic

Although this condition is felt to occur frequently in projects, it is in fact rarely such a condition. Should this situation actually occur, fast action is required to stabilize the situation (Act - Sense - Respond).

Point two 2: Why our current methods (often) do not work well in projects

As mentioned at the beginning, I do not want to describe in this article that the waterfall model is bad, but rather to explain where the waterfall model makes sense and under which conditions e.g. agile methods are preferable.

So back to our initial question: Why do our current methods often not work in projects? The answer is simple - we often apply the wrong model to our problem. If a problem is complex, a seemingly exact description using a waterfall model is not useful. It is even harmful, because it suggests a complete predictability of the solution, although this is not possible. This manifests itself in endless adaptations and change requests, straining everyone's patience. Here agile methods help to deal with this complexity.

But also the other way around it does not make sense to build a house with agile methods. Since the dependencies are mostly known and the project is not fundamentally new, the waterfall model will probably deliver the better project result.

Conclusion:

People have always found methods to deal with their problems. As problems were complicated we have invented waterfall models and introduced them in the last 50 years. Now, however, more and more problems are becoming complex, so we have to look for new models to deal with them. Agile methods can be the answer. It is important to consciously arrive at a correct assessment of the problem and choose the right approach.

Does everything have to be agile?