Building a DevOps Culture

Chapter 5. Mentoring the New Culture

As your pilot project progresses, you’ll be mentoring the product team in the values that foster a DevOps environment: communication, responsibility, respect, and trust.

Fostering open communication channels is the foundation of a successful DevOps culture. One of the most common complaints from many Operations teams is that they weren’t aware of an impending change, or weren’t told about a new product feature. When I have asked these folks if they attend the Development team’s standup meetings, the answer I typically hear is “No, we don’t go to their meetings, they never talk about anything that concerns Operations.” Open communication leads to better collaboration throughout the entire team. Every team member should know that he or she can go to any other member of the team to ask a question, and expect to get an answer or some kind of help in finding one.

Your new DevOps-style standup meeting includes everyone. Your developers, your QA engineers, your system administrators, all of them get together to discuss what they’re working on. The benefit of this single change is immense, and even if you’re reading this article and thinking “This isn’t for me,” bringing a full team together to intentionally talk about the product on a regular basis is going to help you build a better product.

Maybe your developers are working on how the product is going to use a new third-party API. They’re talking about credentials and authorization with that third party. QA will need to know how to write tests to flex the features; do they need fake accounts or dummy users? Your sysadmins will have questions about the SLA of the API, how failures will be monitored and rolled up for reporting. What if the API has an outage? The whole team has worked on the metrics and information necessary for the product manager to make an informed decision on disabling the feature. When everyone knows up front what new features are in the pipeline, it will be less likely for any of these topics to be missed. This makes your product more reliable and gives your customers a better experience.

Another aspect of opening communications is logistical. Your best bet for successful communications is with people who sit together or have a common physical space that allows for chance interactions and conversations. Think about the conferences you have attended. The “hallway track” is often just as awesome as the talks. People get to meet in person and talk in real time about interesting things. The product your team is building benefits from this unfettered communications. Get your folks some dedicated white boards to work on and share ideas. Facilitate all avenues of communication.

If your team is not permanently collocated, bring them together for as long as you can at the beginning of the project. You want them to get to know each other, talk to each other, and begin to gel as a team. A week is probably the minimum; two is better.

Your next hurdle is realigning the responsibilities of your team. The scariest part of this step is fixing your broken on-call schedule. Guaranteed, if only your system administrators have ever been paged about your product, your responsibility model needs to be updated. It should be obvious; if the desire is to have the product be as awesome as it possibly can be, every bump, burp, or error in the code would report back to the person who wrote that code and is most likely to understand what is wrong in order to start the process of fixing it. Fault recovery processes in siloed teams are full of wasted time while additional input is requested from people who didn’t think they needed to be available.

Similarly, Dev and QA shouldn’t be the only ones creating tests for the product. If your team is adopting DevOps practices in order to put more product features in front of customers more often, you want to be confident that the way you’re installing the product’s requirements or new infrastructure will behave properly with your product. This requires testing. How you test varies with the availability of hardware resources and the tools your team might be using. Making it a priority to test infrastructure changes is assigning value to a new behavior, changing the incumbent culture.

Mentoring your team around respect derives from how you resolve interpersonal conflicts among team members. Team members need to feel that everyone is being treated fairly and has a voice. Sometimes teams fight, and fighting it out over a difficult topic isn’t always a bad thing. But it’s possible to have a good fight and a bad fight. Personal attacks and name-calling are obviously not the kinds of behaviors you want in a collaborative team, and you have to deal with them from a managerial standpoint. These are the difficult discussions for many managers, and it is tempting to sit back and let people deal with their conflicts. Do not be tempted to ignore destructive behaviors; engage the people who are fighting in a constructive discussion, work through the sticking points, and mediate a compromise if one is warranted. This is a good place to ask one of your SMEs to help mediate and act as an impartial judge.

Finally, your team members will learn to trust each other over time. The changes to behaviors in communication, responsibility, and respect build an open, collaborative environment. Your team will build new relationships that they wouldn’t have had in an otherwise siloed environment. It may take a while for your team to heal from whatever post-deploy PTSD may be haunting their nightmares.