Agile Software Architectural Governance in the Real World
June 13, 2015 Leave a comment
This article focuses on the use of the ‘Architect’ role within Agile environments by taking into consideration the experience of the author, as well as objective opinions from other Software professionals who have found their own version of successful software architecture via different means in the agile environment. The consideration of using the ‘architect’ role within an agile team is discussed along with how the architect can ensure the software does not become fragmented as well as ensuring architectural governance and accountability.
Architecture for any software project is the glue and foundation that keeps the software together, keeps it stable, keeps it maintainable, and keeps it performing well, among other things. How you “get to the architecture” depends on many factors, including, the structure of the project, the methodology used, the people on the project, and other factors. This article is going to focus on architecture as it is used specifically within Agile environments.
In addition to my own background and insight having worked in Architecture in Agile (as well as waterfall and others), recently, I put out a few questions to the architecture and agile communities hoping to gain additional opinions from those working with Architecture in Agile environments and how architecture is used and influenced in these different cultures. The response and encouragement has been overwhelming. Because this article is part opinion and part research, I take into account the experiences and opinions of myself as well as other top professionals. My goal is to be as objective as possible while sharing the experiences and opinions of other professionals weather or not I have shared similar experiences, use different methodologies, or agree or disagree.
Using the waterfall methodology, many software projects try to nail down all of the requirements up front, and these include both functional and non-functional requirements. Architecture documents are also created to various levels of detail describing the architecture. Waterfall typically can’t account well for variations in business or technical requirements along the way, so trying to get it right the first time (never happens) is typically important for many organizations. So typically, lots of time is spent up front tying to nail this down as best as possible.
Agile teams handle architecture differently, and depending on the team, there may be an initial iteration (or two, or three, etc) to nail down the initial architecture. Some agile teams will try to at least nail the most significant decisions as related to architecture (see, Introducing Significant Architectural Change within the Agile Iterative Development Process) hopefully trying to mitigate future architectural changes while understanding the costs associated with it. Other Agile teams let their software grow organically, as many Agile proponents promote YAGNI (You ain’t gonna need it), or also verbosely explained as “let’s not write anything until we actually have a business or technical reason to do it – aka, let’s not over-architect). So, YAGNI is great for a lot of things, but it doesn’t go far enough to account for significant architectural decisions that need to be baked into the software without significant cost down the road. The “Last Responsible Moment” principal tries to address this as well, and it works great for some teams, but it’s up to teams to determine when the “Last Responsible Moment” for creating and implementing the architecture is, and that becomes very subjective with the prospect that if you wait too long, the more re-engineering work is required.
So, how are Agile teams doing Architecture out in the field? My experience tells me how I’ve done it, as well as what has worked in the past for me and my teams. How does everybody else do it? Do they do it like me, do they have a better way, do they do it worse? Are they successful? Knowing that “the team” makes all the difference and what works for one team won’t necessarily work for another team, I have compiled what I think are the best responses, along with my commentary, to the questions put out to the community.
Dani Mannes – Agile Modellers & Developers
Dani Mannes is the Founder and Chief Architect at ACTL Systems Ltd. His work focuses on the defence industry where he is as a consultant and trainer to helping this predominately waterfall industry adopt agile.
In Dani’s approach, he uses the terms “agile developers” and “agile modellers” to define agile team members which do either development or design. He runs into a typical problem that I have seen in many Agile environments, and as Dani puts it “The teams are supposed to refactor, but they often don’t do it because of time pressure. This leads ultimately to spaghetti code/architecture and the velocity will eventually drop dramatically.”
The modellers will use a modelling tool to sketch out and ensure architecture documentation is up to date. “So in each sprint you have an architecture description of the sprint scope. But the architecture should not only focus on the current sprint but also take into consideration stories that will most probably be tackled in the next 2 sprints”. Dani emphasizes that taking into consideration the future stories is essential, but these should not be modelled at this point since “taking into consideration means only to think about them but not actually find a solution for them”.
In Dani’s world, ”the team acts as the architect”, and he states that there is no need for the architect role. But, he does keep one person in the role of architect, or “architect champion” just to keep discussions short, and to monitor the need for architectural change during each sprint.
“Our experience has shown that when the team applies a model based design process during first days of each sprint where focus is set on the sprint scope and attention is given to the scope of the next 2-3 sprints, the team is capable of coming up with a good architecture that serves as guidance during implementing the sprint scope.”, says Dani, and “Since the team has come up together with the architecture, all members are aware of the modules”.
Lee Fox – Architecture as Part of The Team
Lee Fox is a Software and Cloud Architect, Agilist, and Innovator, and he ensures that the architect is always a contributor and a team player. In his experience with waterfall projects, the architect is isolated from the rest of the team and isn’t necessarily even part of it, “As part of the team, I preach that the architect MUST be a contributing member and with some degree of consistency even contribute to team deliverables.”, and “the architect needs to really enhance the idea of empowerment and encourage the team to make architectural decisions.” Lee Fox also values that the architect must have the teams trust and maintain the big picture vision. This is a recurring theme in many agile processes when it comes to working within an agile team.
Lee’s vision is that “Agile architects work both in the low level with the team as well as the high level with the business. They use their broad exposure with the business to help guide a team’s decisions in the right direction. “ He is fine with architects working on multiple teams, but cautions that the “the architect must contribute to each team he is a member of and keep up with the big picture”. What Lee has seen through his approach and coaching is an increase in both velocity and code quality.
The Need for Governance – Dan’s Thoughts
In Agile, teams are self organized where there is no one on the team who should have additional responsibilities than anyone else as everyone is working together to achieve the same goal. Work is picked up and worked on by any team member and the expertise that is created is shared amongst all team members. However, I believe, as well as other respected Software Professionals, such as, Simon Brown, author of Software Architecture for Developers (https://leanpub.com/software-architecture-for-developers), that you need someone responsible for the Big Picture of the Software, and that includes the architecture. This creates accountability and governance for the architecture and all non-functional requirements (NFRs).
There are multiple ways to approach this, but ultimately realizing that having that architect who is responsible for the big picture can help ensure that the architecture is continually in-line with the functional and non-functional requirements. The architect will have increased access to both the technical side and the business side to ensure that the team(s) are continually aligning to not only the business/functional requirements, but the architecture is in alignment with both short and long term functional and non functional requirements, and that the existing architecture is followed, revised, and re-worked as necessary.
Ok, so I know some agile purists out there are thinking “Long term requirements are very subjective. Until a user story gets chosen by a ‘Product Manager’ and moves from the backlog to being actively worked on during a sprint, it’s not really a requirement yet”. Ok fine. I get that, and I understand the advantages here which is makes agile a process which helps you to change direction or add new to the project features midway through a development phase. User stories are (should be) always functional requirements though. When considering the architecture, we need to understand what is in the backlog or the general type of functional items that are in the backlog which helps the architect to create the technical vision. The technical vision can change as the project progresses, but ultimately understanding the grand vision will help ensure an architecture that takes into account current requirements and ensure it meets future product functionality with minimal re-work.
A drawback of Agile teams without architectural governance is that the system tends to fragment or suffers from too much rework and often non-functional requirements (such as performance, scalability, and others as related to architecture) get tossed out the window. Imagine a team has estimated a total of 20 points for the upcoming sprint. They have to consider, how can they get it done? Among these considerations is what is needed now, and unfortunately what is needed now, is often at the expense of technical accountability. Fragmentation of the system (or of the architecture) occurs when the now becomes the most important piece rather than ensuring we are adhering to sound architectural principals an meeting our NFRs as well as the long term architectural vision of the software. This is why architectural governance is important.
In agile planning meetings, the team will typically talk about design and may also discuss architectural changes. Ultimately, the team may still be on their own to make these decisions and ensure that the architecture they have will meet the existing business and technical requirements of the sprint, but it’s up to the architect to ensure that the decisions and approaches undertaken by the team are in fact in line and consistent with the architectural vision. The architect is accountable, and accountable for the team’s decisions related to architecture and any architectural changes that come about.
The architect is also a team member who may also code but ultimately has responsibility for the continuous evolvement of the architecture. Typically the architect needs to work in the trenches alongside the development and business teams to ensure constant technical team and business communication. The architect should be involved in all of the agile processes from planning, development, to retrospectives. If the architecture has failed, or we spent too much time on a sprint worrying about “the now” and compromised our architecture, it needs to be brought up and addressed (think retrospective). The whole team should be able to come up with reasons why it failed and how we can improve.
When using agile teams, the need for the role of the architect needs to have a high level of consideration. It’s important for this role to work alongside the team and have a very sound vision of how the architecture will meet short and long term objectives with minimal re-work. The architect should be part of all regular agile processes and held to accountability while allowing the entire team to propose and come up with architecture, but ensuring the architect is accountable. Having the architect maintain accountability provides a level of governance to the project from a technical architecture perspective that would otherwise get lost and lead to fragmentation in most agile environments.