Category Archives: Idea

The occasional insight presented in a mini-article.

Architecture can be agile, too!

Ever since the introduction of “Agile” around the turn of the century, people have wondered how to combine agility with architecture. The difficulty of this combination is due in part to a fundamental difference in goals between the agile and architecture approaches: architects look for stability and future-proof-ness, while agilists want to embrace change, looking for a kind of future-“loose”-ness.

Some say architecture and agility are conflicting approaches. The agile proponents’ attitude towards Big Up-Front Design (BUFD) certainly seems to directly oppose the idea of architecture-driven development. This perception of conflict is increased by the agile movements’ tendency to behave like a religion, complete with dogmas and heretics, as wittily described by Philippe Kruchten. On blogs, agilists sometimes post rants against any suggestion towards up-front thinking about a solution’s architecture, or any hint that not all key (quality) requirements can be addressed afterwards by magically refactoring an IT solution. Fortunately, not all agilists scorn architecture, as witnessed by Scott Ambler’s essay on agile architecture.

Looking more closely, one sees that architecture and agility represent two ends on a spectrum.  Where in this spectrum is the optimal place for your project to live depends on the project context. As noted in an earlier blog post, Barry Boehm suggests that the ideal place on this spectrum depends on three factors that together determine the amount of architecture needed beforehand: the size of the project, the volatility of the environment and the business criticality of the solution.

Agilists can become more successful if they take the project context into account when assessing the usefulness of architecture, but what can architects do to cross the divide between agility and architecture? Judging by TOGAF, the Open Group’s popular architecture framework, the principles in the Agile Manifesto have long been neglected by the architecture community. ADM, the TOGAF architecture development method, still requires rather bulky documentation, produced by often heavy processes like Business Architecture, Information Systems Architecture and Technology Architecture. This type of enterprise architecture approach is not suitable for an agile environment. In the software architecture world, lighter architecture approaches like George Fairbanks’s “Just Enough Architecture” are gradually gaining ground. These more agile approaches no longer view architecting as mainly a design discipline, but also as a way to control risk and deal with uncertainty.

Risk- and Cost Driven Architecture (RCDA) is a relatively new approach in this arena. This approach was developed to close the gap between enterprise and software architecture. Existing software architecture practices are often too limited in scope for the solutions that need to be architected, but the enterprise architecture practices are too heavy for the agility required by time pressure and frequently occurring changes and uncertainty. The RCDA approach incorporates a number of aspects from agile software development practices, such as the use of a backlog of architectural concerns, to be frequently reprioritized based on economic factors like risk and cost.

arch workflow hiThe secret of making architecting agile is to change your view of the main deliverable of the work, just like with agile software development methods. An agile software development team does not deliver a “big-bang system”, but a continuous stream of improvements to a system. In the same way, an agile architect does not deliver a “big up-front design”, but a continuous stream of architectural decisions, step by step gaining control of the uncertainties and risks surrounding complex IT solutions. How much architecture to build in is determined not by agile dogmas like “You Ain’t Gonna Need It” (YAGNI), but by economic trade-offs taking into account the real value of architecture in context.

Architects can do much to cross the divide towards agility – in fact, they not only can, but must. If they don’t make this effort, architecture departments will lose touch with their IT development departments, where agile methods have become mainstream, and with their business stakeholders, who continually ask for faster and better responses to changing market requirements. The key change architects need to make is to no longer view architecture as a design document to be delivered to projects up-front, but as a continuous decision making process in order to gain control over cost, risks and uncertainty. Only then can architects deliver the added value and flexibility required from them by their business stakeholders.

Listen to Bett and Russ of architecturecast.net interviewing me about these ideas and RCDA.

Assumption-free architecture documentation

When reviewing architectures, I sometimes run into architecture description documents that contain long lists of assumptions. There are some negative connotations associated with assumptions. They often read as a “list of things other people should take care of.” They seem to imply that the author:

  • has not checked whether the assumptions are valid;
  • does not want any responsibility for the assumptions;
  • does not accept any blame if the assumptions do not hold;
  • avoids dealing with uncertainties.

It is better to avoid assumptions in architecture documentation: it is not coincidence that the RCDA Solution Definition template does not have a section called “assumptions”. There are several kinds of assumptions in architecture documentation, and they can usually be rephrased quite easily:

  • Scope limitation: “We assume that the client will take care of the network connection between the data centre and the Gouda office.” Rephrase: “The scope of the solution excludes network connectivity between DC and Gouda office.”
  • Interpretation: “We assume that a Spring/Hibernate framework on JBoss fits the client’s open source policy.” Rephrase: “Requirement RFP.OS1 Open Source is fulfilled by utilizing a Spring/Hibernate framework on JBoss. “
  • Dependency: “We assume that the PEAR application suite will run on a Websphere platform.” Rephrase: “There are no instances yet of PEAR application suite 4.5 running in production on a JBoss platform. Compatibility will be validated during the elaboration phase.”
  • Pending agreement: “We assume that the Testing Centre will  validate the performance criteria.” Rephrase: “The solution is based on cooperation with the Testing Centre for the validation of the performance criteria. Agreement about this cooperation is currently being negotiated by…., expected outcome October 15th at the latest.”
  • Pending decision: ”We assume that management will approve the necessary investment in the new Firewall “. Rephrase: “Management approval of the Firewall investment is expected November 10th. If it is not approved, the first three paragraphs of section 5.4 and figures 4, 5 and 6 of this document need to be revised.”

Care should be taken to place these rephrased assumptions in an appropriate section of the architecture documentation. Scope limitations and requirement interpretations belong in the Requirements section. Dependencies, pending agreements and decisions usually represent architectural concerns that have not yet been fully addressed, and belong in the Concern Register or equivalent section of the architecture document. Do not forget to document the implications of these. Many assumptions also lead to risks and may require mitigation measures, which should be added to the risk register and project plan.

Rephrasing assumptions this way has two clear benefits:

  • Clarifying what type of assumption we are making leads to more clarity what actions need to be taken to deal with them.
  • It avoids the negative “cover your behind” connotations many experience when reading assumptions, and gives a more pro-active impression of an architect who deals with, rather than avoids uncertainties.

What are your experiences with assumptions in architecture documentation? Are there more types of assumptions than the five categories listed above?

How much architecture up-front?

I recently read an interesting book chapter by Barry Boehm about what is the right amount of architecture to do  “up-front” before committing to the architecture. Boehm analyses his project database and concludes that for most solutions, a proper solution architecture validated up-front will eliminate many of the delivery overruns and shortfalls commonly experienced. The only exception is small, non-critical solutions in a volatile environment; in those situations, architecting generally has a negative ROI.

So what is “small”? Boehm’s analysis yields the following rough estimate of the optimum amount of architecture effort for various project sizes:

Project size Optimum architecture effort
$ 100,000 5%
$ 1,000,000 20%
$ 100,000,000 37%

My only addition to this table is a translation of Boehm’s Lines of Code to project budget, based on the assumption that the need for architecture in pure software solutions has roughly the same budget relationship as that in IT-related solutions with a wider scope. Boehm also nicely shows that the optimum architecture effort goes down for solutions with a less stable context, and goesup for more business critical solutions.

RCDA on how much up-front architecture

RCDA does not consider architecting to be an “up-front” activity that has to be completed before implementation of the solution can start. In most projects, however, there is a moment that can be identified as the Architecture Milestone. This is the moment after which reversing key architectural decisions becomes very costly and time-consuming, the moment at which the delivery team commits to the architecture. This is a key milestone for solution architects. Solution architects need to know how much architecture needs to be done before this milestone.

In RCDA, the answer to the “how much architecture?” question is based on the view of architecting as a risk- and cost management discipline. Deciding how much architecture needs to be done before committing to the solution is a risk management decision.

Less up-front architecting generally increases the following risks:

  • risk of not fulfilling architecturally significant requirements
  • risk of rework (refactoring, repairs)

More up-front architecting generally increases other risks:

  •  risk of overdesign (YAGNI)
  •  risk of idleness (resources waiting to go to work)

As you can see, both options increase risks that can lead to delays in delivery and cost overruns. Which outweighs the other is determined by the solution and project context. The context factors are the same as in Boehm’s analysis:

  • A volatile environment increases the probability of overdesign, making less up-front architecting better.
  • A highly business-critical solution increases the impact of not fulfilling architecturally significant requirements, making more up-front architecting better.
  • A large project increases the impact of both rework and idleness, these two risks compete with each other to determine whether more or less up-front architecting is better. When making the trade-off, take into account that the impact of idleness increases linearly with the project size, while the impact of rework generally increases at a superlinear rate, due to the extra coordination and dependencies between the elements of rework.

In short, RCDA’s view of architecting as a risk- and cost management discipline leads to the same conclusions as Boehm’s analysis, which adds a nice quantitative heuristic for solution architects to use.

References

Making Software: What Really Works, and Why WeBelieve It“, edited by Andy Oram and Greg Wilson. Chapter 10: “Architecture: how much and when?”, by Barry Boehm.

RCDA: Architecting as a Risk- and Cost Management Discipline” by Eltjo R. Poort and Hans van Vliet, Journal of Systems and Software (2012)