I know I'm a little late in posting my follow up from the April Devops Meetup in Atlanta, but it was a great morning and I wanted to share some of the things discussed. Not everything in this post was explicitly discussed in the meetup, but some are thoughts I had related to the topics.
Artisan Server Crafting
First, I put out a challenge/request for John Christian to record his routine on "Artisan Server Crafting". He talks about how the traditional system administrator "crafts" each server, gives them personal names, and treats them like family. "Oh Look! Gandalf has been up for 365 days, let's throw a party!" The cloud makes the family too big that you can't give each of your children the attention they "deserve". I guess we just have to treat our machines like, well, machines. Automation, not art.
Devops, bottom-up
Next, John talked about how Devops got started inside his company. They started with one person from Dev and one from Ops working together on some automation to improve their lives. This is a common vector for Devops in companies. Start small as a skunkworks project, produce some results that show business value, then get management buy-in to continue the work and hopefully dedicate more time to the project, show more business value from your incremental success, then the business is hooked and you can't go back. I'll ask John to write a guest post to go into more detail.
The challenge of the bottom up process is that it is hard to get past the 1.0. You start out with a few energetic people that get things going, but scaling up can only happen with management's support. How do you cope with the original team moving on to something else? Are the new people going to be able to sustain progress on energy and enthusiasm alone? To move beyond 1.0 you have to show business value and show how the goals of Devops are aligned with the goals of the company. Also, you need to maintain the balance of Dev and Ops. A dominant personality can sway projects one way or another and alienate one side of the team.
Bottoms-up does work and has the potential to create a great deal of cohesion between Dev and Ops. Just be aware that at some point someone from the team is going to have to sell the story to management and get the business bought in. Devops is not complete unless the alignment goes all the way to the top of the management chain.
The Devops Team
A second way of introducing Devops occurred at my company. We unintentionally, through a series of reorgs, created a "Devops Department" without really knowing it. We created an Ops team inside Dev to take care of the non-production (Dev, QA, Load Test, CM, etc..) systems. Since this team reported up to the Dev executive and was chartered to take care of Dev's needs, there was a natural alignment of goals. This team and the Configuration Management Team got together and started automating deployments. After about a year building up Control Tier to deploy the code and succeeding in automating all deployments from Dev through Production the focus went to configuration. We have a suite of Java apps that are "overconfigured". Our current project is automating configurations of applications.
Automation and "Devops" got started with a standalone team inside Dev, but through a reorg that team merged with Production Operations. This was actually the best thing that could possibly happen because the biggest risk of a dedicated Devops Team in an organization with separate executives for Dev and Ops is that the team must naturally report up one silo and not have an affinity to the other silo. Also there is a more subtle factor that comes to play. The Devops team is not a part of any one Dev team, and not a part of any one Ops team, so Dev and Ops both think the team is an outsider. The whole point of Devops is lost. Dev doesn't have any ownership and Ops doesn't have any ownership. The team spends a lot of time trying to sell to the bottom and to the top.
Now, we were fortunate that the original Devops Team was populated with some of the senior people in the company that had deep relationships inside Dev and Ops so the selling wasn't too hard, but if you are considering a Devops team, the team will have to have strong support from both Dev and Ops executives with the ability to roam freely within both organizations. (This assumes Dev and Ops have different executives.)
So now our Devops 1.0 was a standalone team inside of Dev, but after the reorg the members are in Ops. But we have the benefit that the first project was to automate deployments which helps both teams, the second was to automate configuration which simplifies life in Ops and gives more ownership to Dev (that's just how things have been over here). Our third phase now fully branches with Ops embracing automation for system configuration and Dev thinking of the code in terms of operational impact and how it can be run and maintained easier and with automation.
But, you argue, Devops is about People first, then process, then tools. I think if we analyze the stories from companies we will find that the tools are the gateway drug to bring in Devops. You can stop at tools and just have some automation, or you can show the business value your tools bring and start a revolution that will ultimately encompass people and process. We have our tools now, we are in the long stage of battle to align the people and process.
I'll conclude with my wholehearted agreement that I believe that Devops will be most successful if Dev and Ops report to the same executive before the CEO. If you have two silos and they don't share goals, Devops will remain a bottoms-up battle.
