Shortening the architectural feedback loop

One of the things architects can learn from the Agile mindset is the importance of short feedback loops. The quicker an architect receives feedback on their output, the faster they learn about its effect in their specific solution context – and better informed architects make better decisions. Architecture is a matter of reducing uncertainty by gathering knowledge and making decisions, and a shorter architecture feedback loop speeds up that uncertainty reduction, leading to better architectures. On top of this, shorter loops lead to shorter reaction times when things change, which increases agility. In this blog post, I will share some tips for shortening the architectural feedback loop.

Architectural decisions are your primary deliverable

What does your organization consider the primary architectural deliverable? Chances are it is a document with a name like Project (Start) Architecture or Software Architecture Description – let’s call it The Architecture Document. It takes weeks or even months to produce, after which time all architectural models and decisions are approved and distributed in one foul swoop – The Architecture Document version 1.0. If you are in this situation, the first step to shortening your feedback loop is to start viewing individual architectural decisions as your primary deliverable. The finer granularity of these decisions, compared to The Architecture Document, will make it much easier to speed up the feedback loop.

Continuously share concerns and decisions

Do not be afraid to share your unfinished architecture output. The sooner you share, the faster you learn. I like to let my stakeholders (both business and technical) know what I think are the most critical architectural concerns, and share decisions as soon as I am aware that they need to be taken. (One word of caution: always make sure the status of the decision-in-progress is absolutely clear to prevent people acting on them prematurely.) This gives stakeholders the opportunity to contribute to the architectural process from the start. Make this information as easy as possible to access: don’t put it in text documents, but find a low-threshold platform like a Wiki or issue tracking system.

Invite immediate feedback from stakeholders

We are often hesitant to ask for feedback on something that we ourselves do not consider to be perfect already. Architects, however, cannot afford to wait that long. Most architectures emerge from a dynamic process of frequently identifying new concerns, repeatedly finding out new facts and continuously adjusting partial decisions that interact with each other and our context. The perfect architecture is only known after the solution has been delivered (if then).

So tell your stakeholders to give you feedback when they have it, and not to wait for ‘official’ review moments like a ‘version 0.9’. Make it as easy as possible for them: enable the comments box on your Wiki, invite people to email you. If big print-outs of your architectural models adorn the team’s war-room, make sure to put a pencil on a string next to them. Make sure everyone knows you welcome feedback by asking for it at the water cooler, or on your team chat platform if you are not co-located.

Simplify your architecture documentation template

Template bloat is one of the causes of long architectural feedback loops. Organizations often lack a decent repository for architectural knowledge, and abuse their architecture documentation templates as a dumping place for all lessons learned. Diverse stakeholder concerns for all types of solutions end up getting their own sections in the template. On top of that, architects for whom The Architecture Document is the only place to store knowledge about a solution cause even more bloat. There are two things you can do to fight this document obesity:

  • Create a template with sections for only the most common concerns at only the start of a solution’s lifecycle. Get rid of all ‘placeholder’ sections. Add views for other concerns only as they become significant later in the lifecycle, and only insofar as they are relevant to your context – in short, create living, minimal architecture documentation.
  • Make sure you have another repository for knowledge that is not immediately relevant at the current point in the solution lifecycle. Create a wiki or library for the organization’s lessons learned and documentation plug-ins for views to be added later in the lifecycle, and find a place outside of The Architecture Document for solution-specific insights that you have gathered.

Get involved in delivery

The final tip for an effective, short feedback loop is to get involved in the delivery of your solution. If you are a software architect, coding key parts of the solution yourself is a great way to get involved. If you do not have that opportunity, get involved in integration, quality attribute testing or other architecturally significant delivery activities. You will not only become your own feedback channel, but also stimulate other delivery team members to tell you about their concerns and help improve your architecture.

In my work of coaching organizations to approach architecting in an agile way, shortening the architectural feedback loop has proven to be one of the most effective ways to improve architects’ effectivity and business value. Especially its positive effect on the scability of architecture work and the start-up time of smaller projects is quickly noticed and appreciated by business stakeholders. Let me know if the five tips in this post prove to be useful to you as well.

SATURN 2017

Another great SATURN (SEI Architecture Technology User Network) Conference in Denver last week. The cast of interesting and directly applicable topics was led by two main characters: microservices and event sourcing. Apart from those leading roles, there were supporting roles for functional programming, and of course the popular returning role for good old uncle agile architecture. Below is a day-by-day account, with links to the video recordings for each of the sessions discussed.

Tuesday: details and crowdsourcing architecture

A great start with a keynote from @KevlinHenney, stressing the need for attention to details – even (or especially!) for architects. As Edsger Dijkstra wrote, “The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” Sometimes architects hide behind the “architecture as an abstraction” definition as an excuse not to go into details. However, sometimes the devil is in the details, meaning a tiny detail may have substantial risk and cost impact – meaning it is architecturally significant (according to Risk and Cost Driven Architecture).

The highest impact speaker this year was undoubtedly Sebastian @vonConrad, whose first session had some great tips about being an influential architect among 23 fully autonomous teams. If you have no decision authority, influence the decision makers (in the teams). And people are most likely to be influenceable when you help them. Sebastian did this by setting up an architecture guild, facilitating inter-team knowledge sharing and peer reviews based on very light-weight architecture documentation for projects (“project docs”) and architectural vision (“light on the hill”). Especially his approach for setting technology standards without stymieing innovations (“sensible defaults”) was refreshing. (I unfortunately had to miss Sebastian’s second session on event sourcing because it coincided with my own session on architecture life cycles – but I will certainly watch the recording once it is published.) Later that day, Stefan Toth (@st_toth) spoke about more or less the same topic under the title “the distributed architect”. Stefan presented a very nice model to decide how “crowd-sourced” your architecture can afford to be, with four levels of central roles: fully crowd-sourced (no architect), architecture marshalls, an architecture owner or the classic architect.

Wednesday: microservices and functional programming

This year’s highlight in terms of usefulness to me was Chris Richardson’s (@crichardson) keynote “There’s no such thing as a microservice“. I have seldom seen one of the field’s rockstars explain a concept so clearly, pragmatically and especially objectively.

Architectures are made to fulfill quality attributes. The quality attributes a microservices architecture is optimized for are all related to development velocity. The price you pay for this velocity is complexity – and it’s not cheap. Making the microservices themselves loosely coupled and simple means that all the remaining essential (coordination) complexity will end up in the surrounding architecture. Chris then went on to introduce his pattern language – a set of architectural tactics meant to address this complexity concern. The most interesting of these patterns was event sourcing, a tactic to avoid microservices having to access common data or using two-phase commits on transactions (often required by business logic). Chris also gave a very pragmatic rule for optimizing the size of a service in a microservices architecture (because “there’s no such thing as an individual microservice”):

  • small enough to be delivered by a small team
  • big enough to have meaningful business functionality

George Fairbanks (@GHFairbanks), one of SATURN’s organizers, won this year’s new directions award for his talk on functional programming, which appears to be coming back in fashion (remember LISP?). George explored the ramifications of this programming paradigm on the architectural level, in a fun way (which included a marbles game).

At the end of the day, I presented at a very interactive session in which we went looking for the common ground between architecting in an agile versus more traditional, project-governed context. Organizations sometimes can be rather sensitive about the words they use to describe elements of architecture work (e.g., “is the solution defined in a project intake form, an epic or a saga?”). In order to stay away from this loaded terminology, I used the metaphor of a human lifetime to analyze how architectural activities and decisions evolve over a solution’s life cycle. I am considering using the response I got in that session to write a little dictionary to help translate between “agilese” and “projectese” language…

Thursday: event storming and architectural decision records

The final conference day started with @RuthMalan‘s acceptance speech for winning this year’s Linda Northrop Software Architecture award. My successor made a passionate plea for visualizing designs that should be based on objective reasoning.

After a hilarious bout of slide roulette, I attended @ThePaulRayner‘s session on event storming, a workshop form for soliciting requirements from stakeholders built on the concept of events (rather than business objects) as the primary entity. It resonated nicely with my plea for time as a first-class citizen in architectural design. Paul’s method seems to be mainly focused on business scenarios, it would be nice to try it out as a way of soliciting change scenarios for architecture roadmapping.

Later, @michaelkeeling gave a nice report of his work introducing architectural decision records (ADRs) in his team at IBM Watson. He observed that modules that have documented ADRs appear to have less rework and greater general design awareness. He gave a series of nice tips in software development teams: — Store with the code, use plain text — Delegate ADR creation — Peer review as you would code — Foster a documentation habit — Make a decision, then document it — Not everything is an ADR! My own experience with ADRs is mostly in a higher level solution architecture context, but points to the same general principles: the threshold to documenting ADRs should be as low as possible, so try to store them in a tool or platform that your team is accessing on a daily basis already – and then coach them.

This was my fifth SATURN conference, and just like the previous years I returned home full of new ideas and inspiration, and with many useful new contacts. You can find recordings of all sessions on the SATURN YouTube playlist.I am already looking forward to the 2018 edition in Plano, TX.

Architecting transient solutions

When I teach architecture classes, I increasingly run into architects who never architect a “system”. They do all the things architects do: analyse stakeholder needs, identify ways to address these needs, (help) decide on the best way by making trade-offs, communicate these decisions and oversee the solution’s implementation. However, after all the work is done, the stakeholder needs are fulfilled, but there is nothing in the real world that they can point to and say: “that is the thing I architected”. At the end of the road, the implementation of their solution to the stakeholder needs consisted of a series of changes to existing systems, but did not require the creation of a new system. Their job is to architect transient solutions: changes to a domain or product they are responsible for.

One example was brought to me by the architect responsible for a transportation ticket vending machine. There was a stakeholder need to reduce the time and cost required to deploy new products and user interaction schemes, and the solution was to move most of the logic and data from the vending machine to a central place. Projects like this require significant (and careful) design effort, and substantial implementation time. A documented solution architecture is a must. However, at the end of the road, no new systems have been created (with the possible exception of minor components like hubs between previously unconnected systems).

Depending on the prevailing flavor of delivery governance, such transient solutions are defined in architecture epics, project documents or change requests. The architects designing them call themselves “domain architect”, “project architect”, “tech lead” or “master builder”.

What’s in a name

Is this really architecture? The ISO 42010 definitions of architecting and architecture both refer to a “system” of interest, e.g. “fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution”. All the other words seem to apply, so can we ignore the fact that there is no one “system” to point to after implementation of the architecture? Is this question really relevant, as long as the person designing these solutions knows how to apply architecting principles and practices?

If this is architecture, what type of architecture is it? Of all the different genres of architecture (software, infrastructure, system, enterprise architecture, etc.) the term solution architecture seems to best cover these transient solutions. This phenomenon actually helps us to create an appropriate definition for the term solution in solution architecture. Where previously we had to make do with something like “a solution is a way to solve a problem”, I would suggest the following:

Solution: a coherent set of changes delivered to address a defined set of stakeholder needs.

The changes referred to in this definition pertain to elements which can be newly created, modified or removed as part of the solution. The term coherent implies that the (changes to the) elements cooperate to achieve the fulfillment of the stakeholder needs (using Fred Brooks’ term, the solution has conceptual integrity). Coordination of a solution’s delivery depends on the governance model applied, this can be using agile or traditional delivery models, in a value stream, in a program or project, governed by a contract or otherwise. How and where the stakeholder needs are defined also depends on the governance model applied, this can be in an epic or set of (user) stories, but also in a program or project definition, contract or change request. From the perspective of this definition of solution, architecting a new system or service is just a special case where the “set of changes” consist of the creation of one new system or service.

Applying architecture practices

How does this way of looking at solution architecture affect the way we apply architecture practices? To explore this question, let’s take Risk- and Cost-Driven Architecture (RCDA) as our reference practices. As stated in the introduction, most core practices apply to both transient solutions and to system (or system of systems) solutions:

Architectural Requirements Prioritization No change.
Architectural Decision Making No change.
Applying Architectural Strategies System solutions: focus on decomposition.

Transient solutions: also focus on transformation of
elements.

Architecture Documentation System solutions: architecture documentation maintained
with system.Transient solutions: documentation dissipates into the
documentation of the systems affected by the solution.
Solution Costing System solutions: solution breakdown structure decomposes
into new elements to be created .Transient solutions: solution breakdown structure
decomposes into transformation of affected elements.
Architecture Evaluation No change.
Architecture Implementation System solutions: focus on build and integration.

Transient solutions: focus on dependencies between changes
to affected elements (and integration).

Technical Debt Control System solutions: technical debt controlled at system level
(and higher).Transient solutions: technical debt controlled at level of
changed systems and higher.
Architecture Evolution Not applicable to transient solutions after completion,
because the solution has then dissipated into the affected
elements.

As you can see from this brief analysis, the main differences when applying architecture practices occur after the solution has been delivered. The most visible difference is what happens to the architecture documentation, for which it makes no sense to remain “as a unit” after implementation of a transient solution, except for historic reference. Extra care must be taken that the relevant information (such as rationale of architectural decisions) remains available afterwards, either in the documentation of the systems affected, or at a higher level (e.g. in domain or enterprise architecture documentation).

Conclusion

As less and less green field systems are created, an increasing number of architects find themselves mainly designing transient solutions – solutions that dissipate into the landscape and cannot afterwards be identified as a product or system. Good architecture practices are still applicable in this situation. This realization leads to the idea for a defininition of Solution Architecture as the “architecture of a coherent set of changes delivered to address a defined set of stakeholder needs”.

Just Enough Anticipation: Architect Your Time Dimension

Eltjo Poort, IEEE Software, vol.33, no.6, pp. 11-15, Nov.-Dec. 2016

A bit of planning is indispensable to anticipate events that will affect your software’s risk profile. Traditional architectural planning emphasizes the spatial dimension (for example, components and connectors) but neglects the time dimension. Consulting IT architect Eltjo Poort makes the case for an explicit representation of architectural evolution along the time axis and reports on the experiences with architecture roadmapping where he works.

Official link

Pre-publication version

Architectuur krijgt nieuwe kans (Dutch)

Eltjo Poort, Computable, 20 September 2016

Official link

‘The best architectures, requirements and designs emerge from self-organizing teams’, is wellicht het meest misbruikte principe uit het Agile Manifesto. Door de jaren heen was de gangbare interpretatie van dit principe verworden tot: ‘Hoe minder je nadenkt over de toekomst, hoe beter je systeem wordt.” Het was de afgelopen tien jaar in de software-ontwikkeling niet bon-ton om het over architectuur te hebben. Maar het tij is langzaam aan het keren.

Modelleren wordt weer oogluikend toegestaan – zij het soms achter gesloten deuren, om het stigma ‘Big Up-front Design’ te vermijden. We durven het nog nauwelijks hardop architectuur te noemen, dus ontstaan er eufemismen als ‘Design thinking’ en ‘Master builder’. Maar het besef dat een beetje anticipatie in je systeemontwerp noodzakelijk is om risico’s beheersbaar te houden is nu wel doorgedrongen, getuige bijvoorbeeld het concept van de ‘Architectural Runway’ in het steeds populairder wordende Scaled Agile Framework.

Het is dus weer veilig voor ons, de anticiperende ontwerpers in de digitale wereld, om voorzichtig uit de kast te komen en ons bekend te maken als bijvoorbeeld enterprise-architect, solution-architect of software-architect. Daarmee is het ook tijd om de balans op te maken: wat staat ons te wachten, wat speelt er in het land van de digitale architectuur? In dit artikel een korte inventarisatie van de belangrijkste trends en ontwikkelingen.

SATURN 2016 – part 2

This is the second and last part of my SATURN 2016 conference report. Since I posted the first part, the organizers have posted video recordings of many of the best talks, and more will follow on the SATURN playlist on YouTube.

Ethical and social considerations in software architecture design featured in many talks, but most prominently in three short “DEV@SATURN” talks by Michael Keeling, João de Sousa and Bett Bollhoefer. DEV@SATURN was a new feature this year: 15 minute talks in the plenary sessions, which yielded some enjoyable and interesting food for thought; I would say a successful experiment. Michael presented a reasoning framework for ethical considerations in architectural decision making, including some reusable quality attribute scenarios for environmental sustainability and fairness to customers and developers. João reported on his work to develop an Internet of Things interaction framework (“Bezirk”) that shifts the power to control data exchanges from companies and application developers to end users – an architecture that gives app users more choices than just the two bad options we are generally given now: give access to everything the app demands or the app doesn’t work at all. João received the conference’s New Directions award for his talk. Bett gave us a glimpse of the insights into mindful software development she gathered in a “Zen of Software Development” booklet (and got the whole audience to do some yoga on the spot…).

Other highlights

For me, the most interesting keynote was Daniel Jackson’s “Rethinking Software Design”. It was mostly about user experience design, and the word “architecture” only appeared once – but Daniel presented some keen insights into the impact of bad user interaction design at the architectural level. Software and solution architects generally do not get very much involved in user interaction design, especially those schooled in the SEI architecture practices, which are mostly based on the notion that non-functional requirements and quality attributes drive architectural design. But isn’t usability also a quality attribute? Bad user interaction design has a substantial risk and cost impact, and can cause a project to fail – both strong indicators that user interaction design is architecturally significant, and thus should be addressed by architects.

Another highlight was Eoin Woods’ talk on Getting your system in production – and keeping it there. 90 minutes of great practical advice. Eoin is a star at systematically presenting solutions, and illustrates each and every point he makes with an amusing (or dramatic) real-life story. Like user interaction, production is another area architects sometimes underestimate, or at best have an ad-hoc approach to. If you have a few minutes, peruse Eoin’s slides, and if you have some more time, watch his presentation here.

getting production

IMG_0472At the close of the conference, I had the honor of receiving the first Linda Northrop Software Architecture Award, a few minutes after Linda herself gave a very nice overview keynote on the status of the software architecture practice. It felt great to get such international recognition for the hard work my CGI colleagues and I put into improving the way we do architecture, during a decade in which the whole idea of up-front design was widely discredited by the mainstream software development community. As noted above, the pendulum seems to be swinging back in the other direction now, and people are back modeling systems in order to build better software – hopefully, before long, with the door open again.
IMG_0470