A Map to Waterfall Wasteland and the Agile Outback

Over the past 18 months, we have been iteratively developing a way to assess maturity with respect to architecture in an agile context. We did this together with a number of client organizations that were interested in improving their architecture function. I will be sharing some insights from this development in upcoming blog posts. This first insight is about balancing the responsibilities of architecture work, to avoid getting lost in either the Waterfall Wasteland or the Agile Outback.

The nature of architecture

The perception of what ‘Architecture’ should mean in the context of software engineering has gone through a number of changes since it was first used in that context.

The original 1990s perception views architecture as a set of structures that represent an abstraction of a system being delivered – needed to deal with the growing complexity of typical software systems. In that view, the main responsibilities of the architect are to create (visual) models of the system, and to use those models to validate the architecture.

In the early 2000s, a second perception emerged, with a new responsibility focus: architects needed to make important decisions in order to create the right models of their solutions (‘right’ meaning that they fulfill their stakeholders’ needs). If ‘abstraction’ and ‘structures’ describe what the architect creates, the decision making refers to how they create.

Around 2010, partly under pressure of the agile movement’s focus on business value, a third perception was proposed: the why was added to the what and the how of architecture. This view shed light on the goal of architecture: to improve organizations’ control over risk and cost – not only during design, but extending the architects’ responsibility to the fulfillment of the solution.

The list of architecture responsibilities would not be complete without the one prerequisite without which the architect would be unable to fulfill any responsibility: understanding the context of the solution, meaning the stakeholders, their needs, and the environment in which the solution is to be delivered.

So we end up with five responsibilities of architects (or of the ‘architecture function’ of organizations): understanding context, making decisions, modeling, validating and fulfillment of the solution. These five responsibilities also map very nicely to Philippe Kruchten’s “Things architects really do”, to Eoin Woods’ “Architectural focus areas” and to the RCDA practices we have been applying for years.

Dependencies

It is important to note the many dependencies between the five responsibilities identified above. Just to mention a few:

  • modelling and decision making without understanding context will lead to wrong models and decisions
  • modelling actually implies decision making (about decompositions, relationships, etc.)
  • if there are no models and no decisions, there is nothing to validate
  • fulfillment of unvalidated decisions and models leads to trouble

So fulfilling the five responsibilities in isolation is not enough: they should be fulfilled in a coherent way.

Good architecture

Since architecture is context, there’s no such thing as good architecture in an absolute sense: the best one can hope for is an architecture that fits the stakeholder needs in its context. In my experience, the best fitting architectures result from paying proper attention to all five responsibilities mentioned above. This is not easy; due to mostly cultural pressures, dogmas and misconceptions, many organizations ignore some of the responsibilities, resulting in a flawed architecture function. Two extreme examples are the Waterfall Wasteland and the Agile Outback caricatures described below.

Paying proper attention to all five responsibilities, however, does not mean always paying equal attention: depending on the context, modeling may indeed require more attention than decision making, and validation may be more critical in some situations than in others.

When talking to teams, architects and stakeholders in different organizations, we started to notice some interesting patterns in the way they took up these responsibilities. We created caricatures to identify those patterns, and called these caricatures the Waterfall Wasteland and the Agile Outback.

The Waterfall Wasteland

In the Waterfall Wasteland, the architects live in an ivory tower. They ignore the decision making and fulfillment responsibilities, which they consider to be someone else’s. They have a very clear job description: to create perfect models and validate them against stakeholder needs. If the resulting solution is unsuccessful, it’s obviously not their fault. The idea that they would be responsible for decisions or share responsibility  for successful delivery is abhorrent to them: it would mean that their success would depend on the capability of others, and on their ability to cooperate…

The Agile Outback

In the Agile Outback, teams usually don’t have architects (although they might use euphemisms  like ‘pathfinder’ or ‘master builder’). Modeling is studiously avoided, since “The best architectures…emerge from self-organizing teams”, and doing modeling might offend the agile gods and disturb the ‘magical’ emergence process. Similarly, validating designs is considered a waste of time: failing early and often is a much quicker way to learn and improve.

Essential architecture skills

The five responsibilities of architects are a good basis for deriving the essential knowledge and skills of the architect:

  • For understanding context, we need analytic skills to identify the environment and stakeholders of our solution, communication and social skills to understand stakeholder needs and knowledge management skills to cultivate and share that understanding.
  • For making decisions, we need decision making skills like making trade-offs and prioritizing. We also need extensive knowledge of the relevant architectural tactics and strategies in our (business and technology) domains.
  • For modeling, we need creative skills for visualizing our design, and knowledge of relevant modeling languages and techniques like decomposition and composition.
  • For validating, we need analytic and trade-off skills (again), and knowledge about techniques for risk management, cost estimation and making business cases.
  • To fulfill our fulfillment responsibility, we need leadership skills like communication, convincing, listening and anticipation. But we also need to know a bit about the economy of delivering software-intensive systems and the relevant software lifecycles.

Looking at this list, it is no surprise that good overall architects are very rare. It might be a good idea to spread these responsibilities over a number of people, and create an architecture function in an organization: a (possibly virtual) team made up of members with complementing skills and knowledge.

Conclusion

Architecture has five responsibilities: understanding context, making decisions, modelling, validating and fulfillment. Organizations need to pay proper attention to each of these responsibilities; what is ‘proper’ depends on the context. Dogmatically denying some of the responsibilities will lead teams astray into the waterfall wasteland or the agile outback. The breadth of knowledge and skills needed to fulfill these responsibilities is substantial, and hard to find in one individual; it almost always takes a team to produce good architecture.

(Image from Pexels at Pixabay)

2 thoughts on “A Map to Waterfall Wasteland and the Agile Outback

  1. Georg fleischer

    Hello Eltjo,
    your way of writing about architecture is tremendous. It really helps in communicating about a the various disciplines that architects need in order to fullfill their aim of building and designing good systems. Its exactly as you describe. If you miss one of the disciplines you mentioned above you either miss the needs of the stakeholder, model in an ivory tower or describe something which cannot be built, because important decisions were not taken. I think the easiest way to understand what architecture is about it is suitable to explain that architecture is the set of (important) decisions in software intensive system. From these decisions all other architectural views can be derived. What most developers and architects miss is that proper decision making takes time and a lot of communication in scenarios with different stakeholder (end users, product owners, leaders, security, operations, development teams).

    Currently we are working in agile development projects where the aspect of architecture is quite important, because we have security constraints, a strong customer interest in getting insight into what will be built and what technologies (and third party products) are used in ths system. Therefore we have adopted a way of working in making architecture decisions upfront for a certain release and building the release in the boundaries of these decisions. It is adopted from the scaled agile framework in the 4+1 sprint development cycle and the customer is fine with that.

    It also makes sense to understand the different viewpoints of an architect in an agile development team: giving guidance to developer through prototypes and code reviews as well as communicating with customers and preparing decisions.

    Keep going in writing about architecture! It really moves us and the projects forward.
    Kind regards, Georg

    Reply
    1. admin Post author

      Thank you for your kind reply, Georg! It’s very encouraging to get feedback that people find my words useful.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.