Welcome!

@DevOpsSummit Authors: Lori MacVittie, TJ Randall, Liz McMillan, David Sprott, Flint Brenton

Related Topics: Agile Computing, Microservices Expo, @CloudExpo, @DevOpsSummit

Agile Computing: Blog Feed Post

Agile Governance | @DevOpsSummit #Agile #DevOps #APM #Microservices

In many organizations governance is still practiced by phase or stage gate peer review

Last year the decision was finally made to mandate Agile across our enterprise. The decision was taken, even though there were many unanswered questions. The assumption was that forcing the migration, along with adoption of popular “enterprise Agile methods” would ensure resolution of the outstanding questions. In practice, Agile methods have been very effective in delivering specific digital business initiatives. But almost inevitably the distribution and delegation of architecture has resulted in duplication, inconsistency and increased complexity, across all project types including legacy and new projects. We are now concerned that we no longer have an effective governance capability. The question is how do we fix this without losing the undoubted benefits of Agile methods?“ Enterprise Architect, F2000 company.

Over the past few months I have heard this message over and over again. While Agile is being successful, it is increasingly in conflict with broader goals. And this is clearly becoming a major issue, manifest in increased complexity, horizon of change and coordination issues as well as inconsistent customer experience.  I am now regularly advising a practical approach to resolution by addressing from the governance perspective.

In many organizations governance is still practiced by phase or stage gate peer review, and Agile projects are forced to accommodate, which leads to WaterScrumFall or worse. But governance criteria and policies are often very weak anyway, out of date or non-existent. Consequently governance is frequently a matter of opinion and experience, highly dependent upon the experience of individual reviewers. As we all know, a basic principle of Agile methods is delegation of responsibility, and ideally we need to delegate governance to the Agile practitioners and teams. So the question is how to implement self-governance and ensure quality and consistency of governance?

I think it was an old John Cleese training film in which Cleese himself plays the part of a manager telling a subordinate that he is now empowered, and he scatters magic dust over him and shouts some magic words saying, “You are now empowered!” Clearly this isn’t any more useful than telling an Agile team that they are now self-governing! Rather we need to go back to basics and define and communicate what governance is required and provide Agile teams with guidance on what is expected.

That sounds good in theory, except that in practice no one is going to be able to accurately define all the governance requirements; certainly not in a fast changing, Agile business and technology environment; nor will Agile development teams be able to keep up with a bureaucratic regime that continually issues edicts that everyone is expected to adopt.

What’s required is a governance system that works in an Agile environment. The parameters of the Agile governance system comprise:
a) A defined Agile governance model
b) Defined principles and reference architecture that establish ways of working together with articulation of business value
c) Automation systems that progressively incorporate the principles and reference architecture into frameworks, tooling, design time platforms, deliverable profiles and knowledge management systems.

d) A Community of Interest (CoI) responsible for the governance system and communications
e) A communication system that ensures Agile projects are fully implementing the governance system, and providing at least retrospective feedback to the CoI that contributes to a common asset base as well as practice maturity.

Agile Governance Model


This is a much simplified Agile governance model. Key points to note are:
1. the centricity of the architecture runway, and the tight relationships between policy, reference architecture, reusable assets and the automation platform.
2. variants guided by various dimensions of scope, which may include applicability, business or technology domain, or even the maturity of the business or technology domain to achieve compliance.

Principles are a great place to start. Self-governance is going to be a key part of Agile governance, and if we can’t articulate and communicate what’s important then we are dead in the water. Some examples shown here:


Reference Architecture is a critical capability, defining the architecture styles and patterns and applicability. But reference architecture shouldn’t stop at models, it must be realized in code in the Design Platform – which progressively realizes the reference architecture as application level infrastructure reusable across multiple projects. The design platform is typically managed by the CoI, a collaboration of senior developers and architects that decide what should be in the platform and develop the models and code as exemplars that succeeding projects will be happy to use, customize and or extend. The design platform is therefore a critical governance tool that actively evolves, managed by the CoI and constantly challenged by project developers  to provide optimal solutions to delivering on the principles and reference architecture. As a by-product the mature design platform will also be a major productivity tool; for example in the Everware-CBDI Agile Service Factory, over 80% of the code for typical large projects will be automatically inherited from the platform.

As shown Principles are generic patterns or techniques that guide strong solutions to business problems. The application of Principles is achieved by Policies. But not your father’s policies! In many (most?) organizations Policies are outdated lists of standards. In Agile governance, policies should be a context based record of how the principles and reference architecture have been realized. Like principles, policies are not mandates from senior management, they are transparent  communications of pragmatic decisions made by the CoI on the best way of delivering an optimal result in a particular context, reusing tried and tested methods supported by existing architecture and design assets. This is therefore a continuously evolving body of knowledge, specifically tailored for one enterprise’s needs. Examples below. Note in particular the Policy Context that highlights applicability and exemption.

Many Agile teams are now using the Scrum of Scrums approach to coordination of multiple projects. This is a highly effective mechanism to manage the pan-Scrum backlog. However this coordination must not be confused with architecture realization. The Community of Interest is not a Scrum of Scrums, it is a group of the most respected architects and developers who will be active practitioners in architecture and development projects, who coordinate the realization of the architecture, the models and implementing code, typically in direct response to project demand, but involving CoI members as appropriate to review, refine and contribute to improve the solution, to be optimal, generic, principle and policy compliant. The Architecture Scrum may therefore on occasions be a series of architecture specific sprints, perhaps at commencement of new programs, or in response to significant new areas of reference architecture or design platform requirement. But in momentum situations the Architecture Scrum is more likely to be integral to multiple development Scrums.

In a generic sense, governance is concerned with ensuring the integrity of the delivered product. This requires a strong focus on the architecture and how it is realized. As many organizations are now realizing, delegation of architecture in an uncontrolled manner is high risk. The approach outlined actually encourages delegation of architecture but to a coordinating body, the CoI, which itself is charged with supporting project demands and broader organizational objectives. But the approach outlined also recognizes that there needs to be explicit documentation of architecture principles and policies and their application in order to allow communication and review, and justification of business value. This is a necessary level of documentation needed to communicate to the many stakeholders involved.

Summary

Reliance on opinions expressed on a case by case basis, or architect resource involvement in projects without the backing of strong, defined reference architecture, gives programs or projects far too much discretion. Whilst we may laugh at John Cleese’s magic dust, in practice the embedding of key architectural code into the platform layer actually does make governance considerably more effective. But even if it’s incredibly effective, it’s not magic. An effective CoI comprising the very best architect and developer skills available means all projects have access to optimal solutions as well as automatic compliance. Agile governance as described in this post is therefore not an extension of Agile methods per se, rather it is a bridge between Agile methods and agile architecture that defines and ensures desired outcomes, without compromising the integrity of the Agile process.


More Stories By David Sprott

David Sprott is a consultant, researcher and educator specializing in service oriented architecture, application modernization and cloud computing. Since 1997 David founded and led the well known think tank CBDI Forum providing unique research and guidance around loose coupled architecture, technologies and practices to F5000 companies and governments worldwide. As CEO of Everware-CBDI International a UK based corporation, he directs the global research and international consulting operations of the leading independent advisors on Service Oriented Application Modernization.

@DevOpsSummit Stories
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, will discuss how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at Dice, he takes a metrics-driven approach to management. His experience in building and managing hi...
DevOps is speeding towards the IT world like a freight train and the hype around it is deafening. There is no reason to be afraid of this change as it is the natural reaction to the agile movement that revolutionized development just a few years ago. By definition, DevOps is the natural alignment of IT performance to business profitability. The relevance of this has yet to be quantified but it has been suggested that the route to the CEO’s chair will come from the IT leaders that successfully make the transition to a DevOps model. If this still seems foreign to you, I recommend reading up on D...
19th Cloud Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS – software, platform, and infrastructure as a service.
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
Today every business relies on software to drive the innovation necessary for a competitive edge in the Application Economy. This is why collaboration between development and operations, or DevOps, has become IT’s number one priority. Whether you are in Dev or Ops, understanding how to implement a DevOps strategy can deliver faster development cycles, improved software quality, reduced deployment times and overall better experiences for your customers.
SYS-CON Events announced today that SoftNet Solutions will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. SoftNet Solutions specializes in Enterprise Solutions for Hadoop and Big Data. It offers customers the most open, robust, and value-conscious portfolio of solutions, services, and tools for the shortest route to success with Big Data. The unique differentiator is the ability to architect and implement a complete Big Data architecture specific to your use case. SoftNet Solutions is a premier...
@ThingsExpo has been named the Top 5 Most Influential M2M Brand by Onalytica in the ‘Machine to Machine: Top 100 Influencers and Brands.' Onalytica analyzed the online debate on M2M by looking at over 85,000 tweets to provide the most influential individuals and brands that drive the discussion. According to Onalytica the "analysis showed a very engaged community with a lot of interactive tweets. The M2M discussion seems to be more fragmented and driven by some of the major brands present in the M2M space. This really allows some room for influential individuals to create more high value inter...
Join IBM November 2 at 19th Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how to go beyond multi-speed it to bring agility to traditional enterprise applications. Technology innovation is the driving force behind modern business and enterprises must respond by increasing the speed and efficiency of software delivery. The challenge is that existing enterprise applications are expensive to develop and difficult to modernize. This often results in what Gartner calls "Bimodal IT," where business struggle to apply modern tools and practices to traditional monolithic...
@DevOpsSummit has been named the ‘Top DevOps Influencer' by iTrend. iTrend processes millions of conversations, tweets, interactions, news articles, press releases, blog posts - and extract meaning form them and analyzes mobile and desktop software platforms used to communicate, various metadata (such as geo location), and automation tools. In overall placement, @DevOpsSummit ranked as the number one ‘DevOps Influencer' followed by @CloudExpo at third, and @MicroservicesE at 24th.
More and more companies are looking to microservices as an architectural pattern for breaking apart applications into more manageable pieces so that agile teams can deliver new features quicker and more effectively. What this pattern has done more than anything to date is spark organizational transformations, setting the foundation for future application development. In practice, however, there are a number of considerations to make that go beyond simply “build, ship, and run,” which changes how developers and operators work together to streamline cohesive systems.
As the world moves toward more DevOps and Microservices, application deployment to the cloud ought to become a lot simpler. The Microservices architecture, which is the basis of many new age distributed systems such as OpenStack, NetFlix and so on, is at the heart of Cloud Foundry - a complete developer-oriented Platform as a Service (PaaS) that is IaaS agnostic and supports vCloud, OpenStack and AWS. Serverless computing is revolutionizing computing. In his session at 19th Cloud Expo, Raghavan "Rags" Srinivas, an Architect/Developer Evangelist at IBM, will discuss the Microservices archite...
So you think you are a DevOps warrior, huh? Put your money (not really, it’s free) where your metrics are and prove it by taking The Ultimate DevOps Geek Quiz Challenge, sponsored by DevOps Summit. Battle through the set of tough questions created by industry thought leaders to earn your bragging rights and win some cool prizes.
Bert Loomis was a visionary. This general session will highlight how Bert Loomis and people like him inspire us to build great things with small inventions. In their general session at 19th Cloud Expo, Harold Hannon, Architect at IBM Bluemix, and Michael O'Neill, Strategic Business Development at Nvidia, will discuss the accelerating pace of AI development and how IBM Cloud and NVIDIA are partnering to bring AI capabilities to "every day," on-demand. They will also review two "free infrastructure" programs available to startups and innovators.
There is growing need for data-driven applications and the need for digital platforms to build these apps. In his session at 19th Cloud Expo, Muddu Sudhakar, VP and GM of Security & IoT at Splunk, will cover different PaaS solutions and Big Data platforms that are available to build applications. In addition, AI and machine learning are creating new requirements that developers need in the building of next-gen apps. The next-generation digital platforms have some of the past platform needs and many new requirements. He will provide reference architectures for next-gen platforms and sample ...
By now most people have either created their configuration management solution or are just embarking on this journey. In his session at @DevOpsSummit at 19th Cloud Expo, Marco Ceppi, a DevOps Engineer working at Canonical, will discuss how to take configuration management to the next level with modelling and orchestration. He will also discuss how and why people are moving from a machine-centric view to a service/application-oriented view of deployments, and how you can leverage the knowledge and tools used at the machine level to expand to the scale-out, service-oriented architecture.
operations aren’t merging to become one discipline. Nor is operations simply going away. Rather, DevOps is leading software development and operations – together with other practices such as security – to collaborate and coexist with less overhead and conflict than in the past. In his session at @DevOpsSummit at 19th Cloud Expo, Gordon Haff, Red Hat Technology Evangelist, will discuss what modern operational practices look like in a world in which applications are more loosely coupled, are developed using DevOps approaches, and are deployed on software-defined, and often containerized, infras...
All clouds are not equal. To succeed in a DevOps context, organizations should plan to develop/deploy apps across a choice of on-premise and public clouds simultaneously depending on the business needs. This is where the concept of the Lean Cloud comes in - resting on the idea that you often need to relocate your app modules over their life cycles for both innovation and operational efficiency in the cloud. In his session at @DevOpsSummit at19th Cloud Expo, Valentin (Val) Bercovici, CTO of SolidFire, will discuss how to leverage this concept to seize on the creativity and business agility t...
“Media Sponsor” of SYS-CON's 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. CloudBerry Backup is a leading cross-platform cloud backup and disaster recovery solution integrated with major public cloud services, such as Amazon Web Services, Microsoft Azure and Google Cloud Platform.
Digitization is driving a fundamental change in society that is transforming the way businesses work with their customers, their supply chains and their people. Digital transformation leverages DevOps best practices, such as Agile Parallel Development, Continuous Delivery and Agile Operations to capitalize on opportunities and create competitive differentiation in the application economy. However, information security has been notably absent from the DevOps movement. Speed doesn’t have to negatively impact security. Container-to-container communication should not be in clear text, yet it is. ...
Whether they’re located in a public, private, or hybrid cloud environment, cloud technologies are constantly evolving. While the innovation is exciting, the end mission of delivering business value and rapidly producing incremental product features is paramount. In his session at @DevOpsSummit at 19th Cloud Expo, Kiran Chitturi, CTO Architect at Sungard AS, will discuss DevOps culture, its evolution of frameworks and technologies, and how it is achieving maturity. He will also cover various styles and stacks in DevOps with examples and live demos – on AWS using tools/techniques for continuou...
SYS-CON Events announced today that LeaseWeb USA, a cloud Infrastructure-as-a-Service (IaaS) provider, will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. LeaseWeb is one of the world's largest hosting brands. The company helps customers define, develop and deploy IT infrastructure tailored to their exact business needs, by combining various kinds cloud solutions.
In his session at 19th Cloud Expo, Claude Remillard, Principal Program Manager in Developer Division at Microsoft, will contrast how his team used config as code and immutable patterns for continuous delivery of microservices and apps to the cloud. He will show the immutable patterns helps developers do away with most of the complexity of config as code-enabling scenarios such as rollback, zero downtime upgrades with far greater simplicity. He will also have live demos of building immutable pipelines in the cloud using both containers and VMs.
SYS-CON Events announced today that Enzu will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Enzu’s mission is to be the leading provider of enterprise cloud solutions worldwide. Enzu enables online businesses to use its IT infrastructure to their competitive advantage. By offering a suite of proven hosting and management services, Enzu wants companies to focus on the core of their online business and let Enzu manage their IT hosting infrastructure.
Without lifecycle traceability and visibility across the tool chain, stakeholders from Planning-to-Ops have limited insight and answers to who, what, when, why and how across the DevOps lifecycle. This impacts the ability to deliver high quality software at the needed velocity to drive positive business outcomes. In his general session at @DevOpsSummit at 19th Cloud Expo, Eric Robertson, General Manager at CollabNet, will discuss how customers are able to achieve a level of transparency that enables everyone from Planning-to-Ops to make informed decisions based on business priority and lever...
Rapid innovation, changing business landscapes, and new IT demands force businesses to make changes quickly. In the eyes of many, containers are at the brink of becoming a pervasive technology in enterprise IT to accelerate application delivery. In this presentation, you'll learn about the: The transformation of IT to a DevOps, microservices, and container-based architecture What are containers and how DevOps practices can operate in a container-based environment A demonstration of how Docker and Kubernetes reduce software delivery cycle times, drive automation, and increase efficiency Ho...