Business Issues, Technology, Applications

DevOps now: Best practices for greater business agility

Blog-post by,

(Author’s note:  This blog entry is a slight rewrite of an article for Discover Performance where I was interviewed about DevOps.)

Agile and iterative development methods have promised greater business agility by letting developers create applications faster. But IT infrastructure and operations teams aren’t always able to deploy the changes / applications at the same speed. Thus, IT experiences the bottlenecks that occur when Agile’s need for speed clashes with operations’ need for stability. The result is non-agile Agile–a feedback loop with a huge delay in the middle.

The key is bringing both agile principles such as iterations and cross functional teams together with operations fundamentals automation, to the entire application lifecycleassessment, development, management and monitoring.  Because DevOps includes a strategy and set of principles, backed up by tooling, for promoting collaboration between development and operations DevOps can help solve the problem of non-agile Agile by producing leaner, more responsive teams that ultimately deliver business value back to the customer. There are a number of factors to consider when bringing DevOps to the enterprise. 

Upfront assessment: In a DevOps environment, you’re looking at new applications holistically, including assessing cost of the infrastructure you’ll deploy in the cloud or on-site. Additionally, you need to decide how you’ll manage releases: whether you’re going to adopt continuous delivery (in which the development team hands the application to operations to deploy) versus continuous deployment (in which development releases incremental changes to production without a formal hand-off to operations)—or both, or as we see in most teams a mixture (normally dependent on the time of change).

Application development: DevOps requires developers to take on additional responsibilities or simply different approaches in the development phase (with the help of their operations counterparts of course). Look to adopt practices such as continuous integration to catch flaws early and increase the level of automation in testing. Use production data to help in the application development and testing phases, and maybe even build monitoring and diagnostics into the application so that it is ready for deployment.

Ongoing maintenance and monitoring: With DevOps, traditional operations concerns about reliability and stability get raised earlier in the process. Victor Miller, senior manager of systems management at Blue Cross and Blue Shield of Florida, stresses the importance of moving monitoring further back into the application lifecycle. “The challenge there is reeducating people and making sure that they understand that they have to develop their app with monitoring in mind,” Miller said in an interview at HP Discover 2011.

Best Practices

Creating a successful DevOps environment requires widespread changes, which can be mapped to people, process and technology solutions.

People: You need open collaboration and education between your operations and development teams as well as a way for experts to share their knowledge. Two key DevOps best practices in terms of people and culture are:

  • No finger-pointing
  • Focus on mean time to recovery instead of mean time between failures

Process: Most of the customers I speak with are using some form of iterative development. Operations teams are often very focused on ITIL, which provides a set of rules around service delivery. The challenge is to take processes for continuous delivery and incorporate ITIL’s additional standards. Creating the Dev Ops version of “WaterScrumFall” (the combination of Agile and Waterfall processes). The two can exist together!

Technology: Development and operations use different tools. But there’s meaningful information in both, and DevOps creates the opportunity for cross-hybridization. For instance, developers may need to become more aware of configuration management systems, server automation systems and monitoring tools etc that are traditionally associated with operations. Especially valuable are social collaboration tools that help development and operations share information, collaborate and ultimately work more like a single unit.

KPIs for DevOps

DevOps encompasses key ITIL service delivery processes, such as configuration, change and release management. To monitor your progress on the path to DevOps, pay attention to these KPIs:

  • Time to set up an environment
  • Time from change request to release
  • Number of deployments per week / month
  • Mean time to resolution

Steve Katz, manager of Software Performance and Quality at Seagate, talked about his company’s experience with DevOps in an interview conducted at HP Discover 2011. For his organization, putting hard metrics around expectations for performance has been key. “That’s been a huge boon for us,” Katz said, “because it’s helped us script [requirements for performance] early in the project and actually look at the unit-level pieces, especially in each different iteration of the Agile process. We can break down the performance and do testing to make sure that we’ve optimized that piece of it to be as good as possible.”

The next phase: NoOps and the cloud

Just as DevOps is an outgrowth of Agile, NoOps is sometimes seen as the next phase of DevOps. NoOps doesn’t mean you’re getting rid of operations. With NoOps you’re automating the process of releasing the application into the dev /test environment or production.  

I typically see customers who embark on the DevOps journey go through one of two paths they either think it through and progress rapidly to continuous delivery or the other path (especially in more traditional enterprises) is they follow a maturity model, starting with ad hoc manual provisioning that requires lots of back and forth between the Dev and Ops teams.

The next step is identifying repeatable processes and automating them. For example, when development checks in a build, the build manager can automatically communicate with the testing tool and kick off a series of regression tests. The same applies to provisioning test environments. With automation in a DevOps environment, a QA person can open a ticket, kicking off the multi-step provisioning process. As a result, the time it takes to set up the infrastructure to test a new application can go from weeks to minutes.

This is only the start of the journey, there is more to learn. However, what (we hope) we know is that long release cycles are a thing of the past. With most organizations using some form of Agile and iterative development, operations leaders can make their teams leaner and more responsive by embracing DevOps.   

 I’d like to hear your experiences with DevOps. Feel free to share them hear on the Executive CIO Forum.

For more information, read the white paper “Agile Development Meets IT Operations–In the Cloud,” (registration required).  For more articles like this, join Discover performance for IT leaders

Related link:

Paul Muller Blog:  IT is like salad dressing, sometimes you need to shake it up

https://h41183.www4.hp.com/inflexion/?jumpid=re_r10351_us/en/large/tsg/disper_blog_ops_oct2011 

(6) (6)

Discussion
Would you like to comment on this content? Log in or Register.
dbobke
Daniel Bobke 7 Points | Wed, 11/02/2011 - 03:36

You could rename DevOps "nothing exists in a vacuum". The best description uses the word "holistic" (as the article and comments have). Application development has a process (be it Agile or other methods) and the infrastructure deployments have their processes and requirements. DevOps is a framework that promotes communication between the two teams and helps them to understand that they can serve each other and the customer better if they climb up a few rungs and look at the whole IT process.

 

I don't think this is too "technical" for the CIO - a CIO should be intimate with the concept of process improvement and a process-driven environment vs. a people-driven environment.

aphillips
Andrew Phillips 0 Points | Wed, 11/02/2011 - 02:32

@Genefa: Thank you for an interesting section about KPIs for DevOps. One of the challenges we've encountered is trying to find KPIs that don't simply reflect stability (MTTR) or throughput (# deployments), but which capture the business value delivered and especially any increases in the rate of business value delivery.
One of the challenges here, which goes a little against traditional Ops metrics, is that there is also a trade-off between change frequency and stability involved (cf. the following blog post).
Recognising and finding a balance between these two in our projects has been an important lesson that seems quite central to DevOps. Automation, especially of testing, as you mention, has been key to maintaining stability for us. Deployment automation has been pretty much a prerequisite for automated testing, too.

@Paul: "In some ways DevOps feels too technical for the CIO?"

I think we will see this start to change as DevOps goes from being a largely practitioner-driven methodology to a topic written about by the Gartners and Forresters. The focus on end-to-end KPIs will help here, especially if they are able to capture how effective an organisation is in bringing new features to market whilst maintaining quality of service.
Ultimatly, the tooling and process changes that are mentioned in "DevOps" or even "NoOps" conversations should all be trying to facilitate this.

Genefa Murphy
Genefa Murphy 39 Points | Sat, 12/17/2011 - 21:53

Good point Andrew – this is part of the Dev Ops conundrum – the want to deliver more faster and the need to ensure that it is delivered in a stable way which doesn’t affect the service health of the organization. A good piece to read in my opinion which doesn’t necessarily give these KPI but which does ask us as practitioners to think differently is Eric Reis’s Minimum Viable Product (the chapter on accounting and actionable metrics is a good one). On a similar note myself and some of my colleagues are currently looking into meaningful and balanced KPI for Dev Ops and other trends – when we have completed our work I will be sure to post.

PaulM
Paul Muller 119 Points | Wed, 11/02/2011 - 20:20

Well put Andrew - there's no reason why DevOps shouldn't foster increased availability AND throughput. It's a question of changing peopel and process to accomodate a "build, test, stage, fix, deploy" capability.

One thing is certain, if you don't have an automated regression test environment and a "like for like" automated deployment model for dev, test, staging, production, you're going to have a lot of downtime. Dev/Ops or Devops... 

cbaart
coert baart 1 Point | Tue, 11/01/2011 - 17:38

many thanks for this great post.

Maybe this webinar by XebiaLabs is interesting.

Learn how to accelerate your application delivery to middleware and cloud environments, with a turn-key application release automation platform, Deployit 3.5.

Understand why application release automation is an essential part of a comprehensive DevOps strategy and hear how large enterprises have approached the implementation of application release automation tooling in their environments.

Webinar Details:
Date and Time: Nov 10th, 12pm (EST)
http://www.xebialabs.com/Deployit_3.5_BestPractices_Webinar

pcalento
Paul Calento 256 Points | Mon, 10/31/2011 - 13:12

Interesting to see the downside to demands for IT agility, specifically "Agile’s need for speed clashes with operations’ need for stability." In your opinion, who drives the DevOps trend? Dev and Ops teams themselves? Project manager? In some ways DevOps feels too technical for the CIO? What am I missing?

--Paul Calento

(note: I work on projects sponsored by EnterpriseCIOForum.com and HP)

Genefa Murphy
Genefa Murphy 39 Points | Sat, 12/17/2011 - 22:27

Interesting question Paul – so here I go with some potential answers (my take only of course based on the customers I work with and a lot of talk with the industry analysts):

Who Drives Dev Ops – Dev Ops has tended to be a grass roots movement primarily driven from the bottom up by developers who are looking to enhance and expand the benefits they have achieved through the application of agile in the SDLC to operations. They are sometimes but not always (this is part of the problem) in cohorts with some of the sys admins on the ops side who are looking to facilitate flexibility in the operations world. For Dev Ops to be successful it has to be embraced or at least understood by the various different stakeholders within a project – from the developer, to the tester, to the release manager to the ops guys. To your point about Dev Ops being too technical for the CIO – I think the term sometimes seems overwhelming but every CIO wants an optimized process for delivering business value which allows flexibility, uses the latest technology to operational advantage and helps reduce costs – this in my opinion is what is at the heart of Dev Ops and explained like this might make things more digestible. However, as Andrew points out there is still a lot of education on Dev Ops (in particular in the enterprise) to be done for it to become main stream approachable (at a recent Gartner conference in an audience of approx. 800 around 64% of the audience were still in the exploratory phase of Dev Ops i.e. finding out what it meant and how they could apply it). I think like Agile before it Dev Ops will start with the purists and over time the masses will borrow from it what works and adopt it into the enterprise. After all, if the “agileistas” came out and send agile is great, but well not everyone needs it, I’m sure we wouldn’t see the mainstream adoption as it is today, but by saying Agile is great, everyone needs it – you start a movement to change people’s thinking

 

pearl
Pearl Zhu 90 Points | Fri, 10/28/2011 - 16:50

Hi, Genefa , as usual. great posting, especially I like you drive the readers to envision the progressive IT development/operation environment, from Agile application methodology to DevOps-to manage the holistic IT enviroment with bybrid methodology and approach, and a couple of KPIs listed there make sense, plus, the goal is to automate operation as possbile or move more stuff into the cloud envelop, the final destination is to framework the Agile IT --focus on business innovation, integration and intelligence. thanks. 

Genefa Murphy
Genefa Murphy 39 Points | Sat, 12/17/2011 - 21:56

Thanks Pearl - I am a big believer in the lean approach to IT, continuous improvment, embracing a hybrid world and underlying all of this the key role of automation and another forgotten element, governance. The key though is that the automation and governance are hidden from the end-user.