My SATURN 2017 session on Agile and traditional architeture lifecycles
Another great SATURN (SEI Architecture Technology User Network) Conference in Denver last week. The cast of interesting and directly applicable topics was led by two main characters: microservices and event sourcing. Apart from those leading roles, there were supporting roles for functional programming, and of course the popular returning role for good old uncle agile architecture. Below is a day-by-day account, with links to the video recordings for each of the sessions discussed.
Tuesday: details and crowdsourcing architecture
A great start with a keynote from @KevlinHenney, stressing the need for attention to details – even (or especially!) for architects. As Edsger Dijkstra wrote, “The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” Sometimes architects hide behind the “architecture as an abstraction” definition as an excuse not to go into details. However, sometimes the devil is in the details, meaning a tiny detail may have substantial risk and cost impact – meaning it is architecturally significant (according to Risk and Cost Driven Architecture).
The highest impact speaker this year was undoubtedly Sebastian @vonConrad, whose first session had some great tips about being an influential architect among 23 fully autonomous teams. If you have no decision authority, influence the decision makers (in the teams). And people are most likely to be influenceable when you help them. Sebastian did this by setting up an architecture guild, facilitating inter-team knowledge sharing and peer reviews based on very light-weight architecture documentation for projects (“project docs”) and architectural vision (“light on the hill”). Especially his approach for setting technology standards without stymieing innovations (“sensible defaults”) was refreshing. (I unfortunately had to miss Sebastian’s second session on event sourcing because it coincided with my own session on architecture life cycles – but I will certainly watch the recording once it is published.) Later that day, Stefan Toth (@st_toth) spoke about more or less the same topic under the title “the distributed architect”. Stefan presented a very nice model to decide how “crowd-sourced” your architecture can afford to be, with four levels of central roles: fully crowd-sourced (no architect), architecture marshalls, an architecture owner or the classic architect.
— Eltjo Poort (@eltjopoort) May 2, 2017
Wednesday: microservices and functional programming
— Joseph Yoder (@metayoda) May 3, 2017
Architectures are made to fulfill quality attributes. The quality attributes a microservices architecture is optimized for are all related to development velocity. The price you pay for this velocity is complexity – and it’s not cheap. Making the microservices themselves loosely coupled and simple means that all the remaining essential (coordination) complexity will end up in the surrounding architecture. Chris then went on to introduce his pattern language – a set of architectural tactics meant to address this complexity concern. The most interesting of these patterns was event sourcing, a tactic to avoid microservices having to access common data or using two-phase commits on transactions (often required by business logic). Chris also gave a very pragmatic rule for optimizing the size of a service in a microservices architecture (because “there’s no such thing as an individual microservice”):
- small enough to be delivered by a small team
- big enough to have meaningful business functionality
George Fairbanks (@GHFairbanks), one of SATURN’s organizers, won this year’s new directions award for his talk on functional programming, which appears to be coming back in fashion (remember LISP?). George explored the ramifications of this programming paradigm on the architectural level, in a fun way (which included a marbles game).
— Bett Bollhoefer (@SASIcentral) May 3, 2017
At the end of the day, I presented at a very interactive session in which we went looking for the common ground between architecting in an agile versus more traditional, project-governed context. Organizations sometimes can be rather sensitive about the words they use to describe elements of architecture work (e.g., “is the solution defined in a project intake form, an epic or a saga?”). In order to stay away from this loaded terminology, I used the metaphor of a human lifetime to analyze how architectural activities and decisions evolve over a solution’s life cycle. I am considering using the response I got in that session to write a little dictionary to help translate between “agilese” and “projectese” language…
Thursday: event storming and architectural decision records
The final conference day started with @RuthMalan‘s acceptance speech for winning this year’s Linda Northrop Software Architecture award. My successor made a passionate plea for visualizing designs that should be based on objective reasoning.
— Eltjo Poort (@eltjopoort) May 4, 2017
After a hilarious bout of slide roulette, I attended @ThePaulRayner‘s session on event storming, a workshop form for soliciting requirements from stakeholders built on the concept of events (rather than business objects) as the primary entity. It resonated nicely with my plea for time as a first-class citizen in architectural design. Paul’s method seems to be mainly focused on business scenarios, it would be nice to try it out as a way of soliciting change scenarios for architecture roadmapping.
Later, @michaelkeeling gave a nice report of his work introducing architectural decision records (ADRs) in his team at IBM Watson. He observed that modules that have documented ADRs appear to have less rework and greater general design awareness. He gave a series of nice tips in software development teams: — Store with the code, use plain text — Delegate ADR creation — Peer review as you would code — Foster a documentation habit — Make a decision, then document it — Not everything is an ADR! My own experience with ADRs is mostly in a higher level solution architecture context, but points to the same general principles: the threshold to documenting ADRs should be as low as possible, so try to store them in a tool or platform that your team is accessing on a daily basis already – and then coach them.
This was my fifth SATURN conference, and just like the previous years I returned home full of new ideas and inspiration, and with many useful new contacts. You can find recordings of all sessions on the SATURN YouTube playlist.I am already looking forward to the 2018 edition in Plano, TX.
See my blog post on cgi.com.
When I teach architecture classes, I increasingly run into architects who never architect a “system”. They do all the things architects do: analyse stakeholder needs, identify ways to address these needs, (help) decide on the best way by making trade-offs, communicate these decisions and oversee the solution’s implementation. However, after all the work is done, the stakeholder needs are fulfilled, but there is nothing in the real world that they can point to and say: “that is the thing I architected”. At the end of the road, the implementation of their solution to the stakeholder needs consisted of a series of changes to existing systems, but did not require the creation of a new system. Their job is to architect transient solutions: changes to a domain or product they are responsible for.
One example was brought to me by the architect responsible for a transportation ticket vending machine. There was a stakeholder need to reduce the time and cost required to deploy new products and user interaction schemes, and the solution was to move most of the logic and data from the vending machine to a central place. Projects like this require significant (and careful) design effort, and substantial implementation time. A documented solution architecture is a must. However, at the end of the road, no new systems have been created (with the possible exception of minor components like hubs between previously unconnected systems).
Depending on the prevailing flavor of delivery governance, such transient solutions are defined in architecture epics, project documents or change requests. The architects designing them call themselves “domain architect”, “project architect”, “tech lead” or “master builder”.
What’s in a name
Is this really architecture? The ISO 42010 definitions of architecting and architecture both refer to a “system” of interest, e.g. “fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution”. All the other words seem to apply, so can we ignore the fact that there is no one “system” to point to after implementation of the architecture? Is this question really relevant, as long as the person designing these solutions knows how to apply architecting principles and practices?
If this is architecture, what type of architecture is it? Of all the different genres of architecture (software, infrastructure, system, enterprise architecture, etc.) the term solution architecture seems to best cover these transient solutions. This phenomenon actually helps us to create an appropriate definition for the term solution in solution architecture. Where previously we had to make do with something like “a solution is a way to solve a problem”, I would suggest the following:
Solution: a coherent set of changes delivered to address a defined set of stakeholder needs.
The changes referred to in this definition pertain to elements which can be newly created, modified or removed as part of the solution. The term coherent implies that the (changes to the) elements cooperate to achieve the fulfillment of the stakeholder needs (using Fred Brooks’ term, the solution has conceptual integrity). Coordination of a solution’s delivery depends on the governance model applied, this can be using agile or traditional delivery models, in a value stream, in a program or project, governed by a contract or otherwise. How and where the stakeholder needs are defined also depends on the governance model applied, this can be in an epic or set of (user) stories, but also in a program or project definition, contract or change request. From the perspective of this definition of solution, architecting a new system or service is just a special case where the “set of changes” consist of the creation of one new system or service.
Applying architecture practices
How does this way of looking at solution architecture affect the way we apply architecture practices? To explore this question, let’s take Risk- and Cost-Driven Architecture (RCDA) as our reference practices. As stated in the introduction, most core practices apply to both transient solutions and to system (or system of systems) solutions:
|Architectural Requirements Prioritization||No change.|
|Architectural Decision Making||No change.|
|Applying Architectural Strategies||System solutions: focus on decomposition.
Transient solutions: also focus on transformation of
|Architecture Documentation||System solutions: architecture documentation maintained
with system.Transient solutions: documentation dissipates into the
documentation of the systems affected by the solution.
|Solution Costing||System solutions: solution breakdown structure decomposes
into new elements to be created .Transient solutions: solution breakdown structure
decomposes into transformation of affected elements.
|Architecture Evaluation||No change.|
|Architecture Implementation||System solutions: focus on build and integration.
Transient solutions: focus on dependencies between changes
|Technical Debt Control||System solutions: technical debt controlled at system level
(and higher).Transient solutions: technical debt controlled at level of
changed systems and higher.
|Architecture Evolution||Not applicable to transient solutions after completion,
because the solution has then dissipated into the affected
As you can see from this brief analysis, the main differences when applying architecture practices occur after the solution has been delivered. The most visible difference is what happens to the architecture documentation, for which it makes no sense to remain “as a unit” after implementation of a transient solution, except for historic reference. Extra care must be taken that the relevant information (such as rationale of architectural decisions) remains available afterwards, either in the documentation of the systems affected, or at a higher level (e.g. in domain or enterprise architecture documentation).
As less and less green field systems are created, an increasing number of architects find themselves mainly designing transient solutions – solutions that dissipate into the landscape and cannot afterwards be identified as a product or system. Good architecture practices are still applicable in this situation. This realization leads to the idea for a defininition of Solution Architecture as the “architecture of a coherent set of changes delivered to address a defined set of stakeholder needs”.
Eltjo Poort, IEEE Software, vol.33, no.6, pp. 11-15, Nov.-Dec. 2016
A bit of planning is indispensable to anticipate events that will affect your software’s risk profile. Traditional architectural planning emphasizes the spatial dimension (for example, components and connectors) but neglects the time dimension. Consulting IT architect Eltjo Poort makes the case for an explicit representation of architectural evolution along the time axis and reports on the experiences with architecture roadmapping where he works.
Eltjo Poort, Computable, 20 September 2016
‘The best architectures, requirements and designs emerge from self-organizing teams’, is wellicht het meest misbruikte principe uit het Agile Manifesto. Door de jaren heen was de gangbare interpretatie van dit principe verworden tot: ‘Hoe minder je nadenkt over de toekomst, hoe beter je systeem wordt.” Het was de afgelopen tien jaar in de software-ontwikkeling niet bon-ton om het over architectuur te hebben. Maar het tij is langzaam aan het keren.
Modelleren wordt weer oogluikend toegestaan – zij het soms achter gesloten deuren, om het stigma ‘Big Up-front Design’ te vermijden. We durven het nog nauwelijks hardop architectuur te noemen, dus ontstaan er eufemismen als ‘Design thinking’ en ‘Master builder’. Maar het besef dat een beetje anticipatie in je systeemontwerp noodzakelijk is om risico’s beheersbaar te houden is nu wel doorgedrongen, getuige bijvoorbeeld het concept van de ‘Architectural Runway’ in het steeds populairder wordende Scaled Agile Framework.
Het is dus weer veilig voor ons, de anticiperende ontwerpers in de digitale wereld, om voorzichtig uit de kast te komen en ons bekend te maken als bijvoorbeeld enterprise-architect, solution-architect of software-architect. Daarmee is het ook tijd om de balans op te maken: wat staat ons te wachten, wat speelt er in het land van de digitale architectuur? In dit artikel een korte inventarisatie van de belangrijkste trends en ontwikkelingen.
My agile architecture roadmapping talk at SATURN 2016 can now be viewed on the SEI’s YouTube channel.
This is the second and last part of my SATURN 2016 conference report. Since I posted the first part, the organizers have posted video recordings of many of the best talks, and more will follow on the SATURN playlist on YouTube.
Ethical and social considerations in software architecture design featured in many talks, but most prominently in three short “DEV@SATURN” talks by Michael Keeling, João de Sousa and Bett Bollhoefer. DEV@SATURN was a new feature this year: 15 minute talks in the plenary sessions, which yielded some enjoyable and interesting food for thought; I would say a successful experiment. Michael presented a reasoning framework for ethical considerations in architectural decision making, including some reusable quality attribute scenarios for environmental sustainability and fairness to customers and developers. João reported on his work to develop an Internet of Things interaction framework (“Bezirk”) that shifts the power to control data exchanges from companies and application developers to end users – an architecture that gives app users more choices than just the two bad options we are generally given now: give access to everything the app demands or the app doesn’t work at all. João received the conference’s New Directions award for his talk. Bett gave us a glimpse of the insights into mindful software development she gathered in a “Zen of Software Development” booklet (and got the whole audience to do some yoga on the spot…).
For me, the most interesting keynote was Daniel Jackson’s “Rethinking Software Design”. It was mostly about user experience design, and the word “architecture” only appeared once – but Daniel presented some keen insights into the impact of bad user interaction design at the architectural level. Software and solution architects generally do not get very much involved in user interaction design, especially those schooled in the SEI architecture practices, which are mostly based on the notion that non-functional requirements and quality attributes drive architectural design. But isn’t usability also a quality attribute? Bad user interaction design has a substantial risk and cost impact, and can cause a project to fail – both strong indicators that user interaction design is architecturally significant, and thus should be addressed by architects.
Another highlight was Eoin Woods’ talk on Getting your system in production – and keeping it there. 90 minutes of great practical advice. Eoin is a star at systematically presenting solutions, and illustrates each and every point he makes with an amusing (or dramatic) real-life story. Like user interaction, production is another area architects sometimes underestimate, or at best have an ad-hoc approach to. If you have a few minutes, peruse Eoin’s slides, and if you have some more time, watch his presentation here.
At the close of the conference, I had the honor of receiving the first Linda Northrop Software Architecture Award, a few minutes after Linda herself gave a very nice overview keynote on the status of the software architecture practice. It felt great to get such international recognition for the hard work my CGI colleagues and I put into improving the way we do architecture, during a decade in which the whole idea of up-front design was widely discredited by the mainstream software development community. As noted above, the pendulum seems to be swinging back in the other direction now, and people are back modeling systems in order to build better software – hopefully, before long, with the door open again.
This year’s SATURN (Software Engineering Institute (SEI) Architecture Technology User Network Conference) was held in San Diego from May 2-5. Compared to last year, the geographical context was much improved – Southern California felt a lot more relaxed than Baltimore under curfew. The conference organizers had once again prepared an excellent technical program, and as always it was sometimes difficult to choose between tracks – everything was interesting. Fortunately, all presentations were video-recorded, and should appear on the conference website soon.
The fantastic opening keynote was by community guru Grady Booch, who gave us a peek into the future of architecting. He talked about “abstracting the unknown”: architecting the kind of systems that stretch us both technically, socially and ethically. His talk set the stage for a number of central themes that pervaded the program: evolutionary architecture, ethical considerations, and the renewed recognition of the role of modeling (the real world) and intentional design in software engineering. Slides 43-45 in Grady’s presentation give an elegant summary of the modern view of why and how we architect. As for architecting the unknown, it is clear that there is much less opportunity for reuse, and that we need a method that allows our intuition to play a much more prominent role.
Evolutionary architecture and roadmapping was the topic of talks by Patrick Kua, David Adsit, Alejandro Bianchi, Pierre Pureur (continuous architecture) and myself, and of a keynote by IoT frontrunner Joseph Salvo. The common message here seems to be that the “dot on the horizon”, the architected future state, is becoming less meaningful in a world that changes faster than architectures can be realized. We therefore need architectures and architecting practices that are flexible and resilient enough to maintain their value in a rapidly changing business or technological context. We all reported on our experiences implementing such practices, in my own case introducing an evolution viewpoint to help achieve just enough anticipation in your architecture.
Modeling the real world seems to be a prerequisite for an architecture to have that flexibility and resilience. I remember teaching this in Object Oriented Analysis and Design classes in the 90s, but it seems that insight is resurfacing now after having been deprecated by overzealous agilistas for a while. This was the topic of talks by Amber Haley, Arila Barnes and George Fairbanks, and played an important role in Daniel Jackson’s excellent Wednesday morning keynote. George talked about the stigma clinging to modeling in some agile teams, and “modeling with the door shut” to avoid being seen practicing such a shameful activity. It was also funny to see that modern modelers are developing their own, new languages for domain modeling, where we successfully used UML in the past. It seems UML is now so much associated with technical design that people don’t recognize it as a domain modeling language anymore, and feel the need to reinvent the wheel… for similar reasons, much of what we used to call architecture now seems to be relabeled “design thinking”.
There is much more to report on SATURN 2016 – next installment coming up soon!
The Software Engineering Institute at Carnegie Mellon has selected me as the winner of this year’s Linda Northrop Software Architecture Award. I feel honored to receive this award from the most respected research institute in our profession. It is a great recognition of the work done by the RCDA team at CGI in the past six years. Thank you Philippe Kruchten for nominating me, and Eoin Woods and Tom Verhoeff for endorsing my nomination.