Scaling Software Agility

An Agile Architectural Epic Kanban System: Part 1 – Introduction

I’ve been building up an internal “head of steam” and some new content for the architecture chapter for my upcoming book on Agile Requirements. It’s one of the later chapters, but an important one as the topic of building ever-larger scale systems with agile methods is becoming increasingly important in the enterprise setting.

I’ve written about the need for some amount of intentional “architectural runway” in agile development in the SSA book and in this blog, so I won’t repeat it here. For that, you can look at the Agile Architecture category, or the whitepaper (Principles of Agile Architecture) I wrote with Mauricio Zamora and Ryan Martens.

By some almost-intentional-serendipity (yes, an oxymoron, but apropos nonetheless), I’ve been working with an enterprise who is puzzling on the same issue. The company is F-Secure, from Helsinki Finland, and they have been implementing and extending agile methods across their enterprise (hundreds of developers) for over four years. Fortunately, they also are inclined to talk publicly about their agile experiences and that means that we can both work together, and publish together – a perfect opportunity for each us to develop and implement some new lean-agile practices.

At F-Secure, Chief Architect Santeri Kangas leads a small team of system architects who are responsible for analyzing and addressing architectural concerns that are cross cutting, big vision things. These are the “architectural epics” that span development teams – consumer smart phone and PC point-application products, corporate server and gateway products, hosted security as service offerings and online storage services operated by operations teams  - as well as a growing suite of web applications.

As they describe it, new architectural epics are driven by four primary business factors:

1. New product or service opportunities. New, ideas, from the portfolio backlog (big things they’d like to do) that provide opportunity for growth of revenue and market share.
2. Changes in technologies that affect multiple products and services.
   Examples: new platforms and operating systems, mobile technologies, 64 bit chip sets, etc.
3. Some architecturally-centric problem within the existing solution set. Examples: performance, size, security, usability, upgradability, compatibility, etc.
4. Common infrastructure and avoidance of over-investment.
   This last factor is driven by the need to make sure that certain components are built only once and reused by many teams, thereby avoiding having multiple teams invest in the same code. Even if “time-to-market trumps potential-for-reuse”, as it often can in agile, some common, reusable components can and do provide substantial long-term user (usability, fitness for use) and business (economic) benefits.

Clearly, this is an important charter for the F-Secure architecture team. Moreover, since the development teams are agile (indeed they are now implementing an Agile Release Train at the product line level), the identification, analysis and implementation of large-scale architectural epics must be agile too. Otherwise, at best we’ll have a clash of mental models; at worst, the company might not achieve the full benefits of top-to-bottom enterprise lean and agile practices.

Therefore, we’ve agreed to collaborate on a set of architectural practices that bring their system architectural efforts to a “lean and agile par” with that of the development teams. We’ll agree on a common framework, implement it, and eventually, report on the results. The output will likely be a whitepaper that we’ll submit to one or more US and Scandinavian agile and lean conferences. The tentative title is: A Lean Method for Analyzing and Implementing Architectural Epics in the Agile Software Enterprise

In a series of blog posts, starting with this one, I’ll be describing our initial thinking. Eventually, we’ll be able to report on the actual results.

Objectives of the Method

A lean, flow-based model for moving from architecture to implementation would accomplish three objectives:

1. Make architectural work in process (AWIP) visible.
   Lean thinking drives us to make sure that all work is visible. As Reinertsen points out, invisible, development work in process is WIP nonetheless. Worse, since it can’t be seen, it has “no natural predators” and therefore there is natural tendency to overload those involved in such work. (Since we can’t see it or quantify it, why not do some more of it?). Our kanban system must make AWIP visible so that it can be owned and managed responsibly. Architectural backlogs, queues and analysis work in process must be made visible, creating a shared understanding of current and future work and workload
2. Establish AWIP limits to help assure product development flow. Limiting wip helps achieve flow by making sure that input matches output, thereby avoiding the turbulence and trashing caused by overload. First, we’ll limit local AWIP to only that work that the architecture teams can actually do, thereby assuring that the architects are not thrashing across too broad a workload – starting many projects – but finishing far fewer. That will increase the efficiency, productivity and quality outputs of the architecture team. Secondly, in doing so, we will also be consciously limiting global WIP. This includes upstream, portfolio WIP (projects that drive new architectures) and downstream, development WIP. (Projects that consume new architecture). In this manner, we’ll match input objectives to implementation constraints, all across the enterprise.
3. Drive an effective collaboration with the development teams. The tension between architecture and development (implementation) is obvious. Eventually, all approved architectural epics are going to be implemented by the development teams. The last thing we want to do is to either surprise them with new stuff (“if we would only have known that sooner, we wouldn’t have spent all this time ….”) or hold them accountable for implementing plans that they don’t feel are actually workable.  If our model drives effective communication between these teams, things will naturally flow more smoothly.

Summary

So there you have the context and objectives for this blog series. Stay tuned for content.