Last month, I was asked to give a second opinion on some key architectural decisions and the way they were working out in a client’s IT landscape. Such architecture assessments are some of my favorite engagements, especially when there is sufficient time to get to the bottom of things with a small team of experts. A thorough analysis, underpinned by firm evidence and fleshed out in concrete recommendations, can add tremendous value: value in terms of risk mitigation, prevention of rework (or failure) and freshly spotted opportunities.

The value of architecture assessments

Where does the added value come from? Of course, an outside reviewer can never have more depth of knowledge and experience about a system than the team that has been working on that system for months or years, no matter how experienced or erudite the reviewer is. There is, however, intrinsic value in what Daniel Kahneman calls the “outside view”.

When the same group of people make decisions together for a prolonged period of time, the quality of the decisions may start to suffer from tunnel vision. Tunnel vision is caused by the need to make progress and the desire to have harmony in the group. Time and peer pressure lead to participants not voicing, or prematurely rejecting, alternatives, and so see only information that confirms what they already thought, ignoring danger signals (confirmation bias). Asking someone from outside the team to provide a second opinion or perform an independent architecture assessment is a great way to obtain such an outside view, and combat tunnel vision as well as other forms of cognitive bias (such as “groupthink”, anchoring bias and cargo cult – see Philippe Kruchten’s presentation on cognitive biases in software architecture).

Can we quantify the value of the outside view? Almost 20 years ago, research suggested that a well-timed and well-executed architecture assessment saves 10% of a project’s costs on average. The same number probably applies to the delivery effort of epics, features and architecture runway in an agile context. It is likely the timing of independent assessments in an agile context will be different from more traditional up-front design situations. Regardless of where you are on the waterfall versus agile spectrum, independent architecture assessments should be timed when opportunity cost is low, and changes to the architecture are (still) relatively easy.

A good outside view

Providing outside views was one of the responsibilities of our “technical conscience” team at CGI (both for our own solutions and on-site for clients). For over a decade, we gathered many insights into how to add real value with our assessments. We certainly learned what not do. A  good architecture assessment is not:

• A cat and mouse game between reviewer and team
• A gate to pass or stamp of approval to obtain
• An opportunity for the reviewer to show how clever they are
• An assessment of the team’s performance
• An audit or compliance check

Avoid these mistakes, as they will significantly reduce the outside view’s added value. In our experience, most of that value comes from:

• Getting stakeholders together in a room: this is often the first time they hear each other talk about the solution or decision you’re assessing, which often leads to unexpected insights
• Uncovering the trade-offs that have been made, creating visibility of (assumptions about) priorities
• Identifying risks and opportunities when there’s still time to do something about them

As you can infer from the first bullet here, a good architecture assessment cannot be based on just a document describing the architecture. You need to talk to the team and their stakeholders to be able to fully understand the context and the drivers behind the choices that were made – ask the “why” questions.

Conclusion

Getting an “outside view” on architectural designs or decisions often yields good insights, and helps teams deal with risks and capitalize on opportunities at relatively low cost. Spot the best time, and ask someone who knows what they’re doing to assess your architecture. </begin shameless self-promotion> Let me know if I can help! </end>

Further resources: book Evaluating Software Architectures (SEI), interview with author Rick Kazman.