DevOps is a way of delivering software that includes people, process, technology, and automation, all working together to reduce time to market features with quality.
The traditional models of delivering software (e.g. ITIL, PRINCE2) are very old (>20 years). These traditional models also involve multiple handovers and silos in duties. The software space has changed a lot since the inception of these tools (the concept of Cloud, i.e. infrastructure as code did not exist back then, today many open source tools exist to take care of monitoring, implement CICD, agile process for quicker software delivery, etc.).
DevOps encourages teams to think in a model we refer to as “from concept to cash”. This is a core principle in lead product development. The concept of this model, implores teams to own their product/service from ideation to retirement and not just write some code and deploy into production.
DevOps and Digital Transformation
In the world of digital transformations, every industry is open to disruption. AWS disrupted data centers, Amazon disrupted retail businesses, Uber disrupted taxi businesses, Airbnb disrupted the hotel industry, data disrupted traditional telco business, to name a few. The disruption is possible with the availability of high speed Internet, new mobile hardware, and the ease of which high quality software is delivered through applications. Customers demand more from their service providers and are able to share their opinion about a product or a company through social media (Facebook, twitter, customer reviews, etc.) impacting businesses’ brand value. Organizations should be equipped to provide what their customers want and do so quickly.
Enterprises who still follow the traditional model of releasing new features once in six months or every year will lose out to competition even though their ideas may be very sound and great. DevOps is a core contributor to the success of digital transformations. DevOps concepts such as Agile, CICD and Lean, enable new ideas to be built and put into the hands of customers with feedback available almost instantly. Teams using DevOps structuring have proven to push less bugs due to better testing through automation, improved reliability of the software, and ultimately, improved team morale and efficiency. This enables confident and creative solutions to be released more frequently. Organizations who do not take advantage of these modern processes and change the way they deliver service to their customers will be overtaken by their competition.
In order to embrace the DevOps journey at scale, enterprises need to change their team structures, and the entire software delivery flow. Changes such as requirements gathering, architecture, design, development, testing, operations, security, and support, are necessary, but oftentimes difficult to implement at the enterprise level.
There are many factors that are barriers to enterprise level DevOps transformations. Some of the common ones are:
1. Fear of change – Success of DevOps lies with willingness of different teams in organization to change their traditional model of working. Some people are afraid of the change and do not want to give up their status quo. They are afraid of uncertainties that come along with the changes and/or are not willing to learn new things. With the risk of implementing a change which fails, teams are slow to act on implementing a better structure. Management has to provide support to teams to take risks and not penalize them for failure and allow them to experiment in agile structuring for the ultimate betterment of the company.
2. Skills – DevOps encourages as much automation as possible in all possible steps. This means employees have to have the skills to inherently build automated systems, or quickly learn. If a team is lacking these skills, it can make it hard for enterprises to change to DevOps fast. Organizations should always look to invest in their employees and allow them to acquire new skills in order for long-term success in DevOps to be found.
3. Inflexible Command and Control – Some organizations have a rigid process which prevents teams from doing things fast. Examples: Projects and funding must go through different approval processes, which may be time consuming. A Software has to be reviewed and approved by operations before put in production, which may take a long time to turn around. Development team cannot deploy software into production, etc.
4. Misunderstanding of DevOps – A common misconception is that DevOps means eliminating all the processes and functions, and teams move forward to do whatever they want. This is not what DevOps aims to do (refer to DevOps definition in the top). To practice good DevOps at the enterprise level, some processes must be strictly followed to keep things running smoothly at all times. At the same time, the waste and wait times between different stages of software delivery should be eliminated. Automation should be employed as much as possible so that errors can be eliminated and speed can be achieved.
5. Missing Sense of Urgency – Some organizations may have vision and strategy to move towards Digitization and DevOps but they may lack the sense of urgency. In today’s digital world investors expect firms to make profit every quarter while customers value low costs and flexible solutions. In order to meet the demands of both, organizations have to act quickly, continuously innovate and question their processes constantly. Some enterprises engage outside consultants for long periods (in greater of 6 months to 1 year) to define a strategy for the company’s Digital and DevOps journey. While outside consultants may bring new knowledge and an outside view, changes have to be acted upon quickly.
To add to the complexity, enterprises need to make these necessary changes while continuing to meet their customer demands. Being said, it doesn’t mean it is impossible to implement Devops in big enterprises. There are big enterprises who have adopted DevOps practice at enterprise level and have seen huge success. ING, Verizon, Netflix, Amazon, GE, Walmart, and Target, are reaping the benefits of DevOps. Many articles are available online referencing such success stories. Here are a few:
Ideas for Enterprise Level DevOps
Big enterprises who want to follow DevOps guidelines have to address the barriers listed above. In the below sections, we will look into some ideas on how to practice DevOps at scale to get the maximum benefit.
In order to achieve efficiency with DevOps, organizations should eliminate the silos in roles and teams like Dev, test, ops, security, product, etc. The lesser the number of teams of different functions and lesser the hierarchy, the better. Time will be saved between hand-offs and work can be shared among team members easily. A team (a group of people who own a service/product/application from concept to retirement) should be given some defined SLAs and held accountable to meet those SLAs. Some examples of the SLAs could be: less than one percent error/roll back or roll forward changes for new deployments or changes.
Common Vision and Roadmap for Senior Leadership and Teams
As we have said earlier, Senior Leadership sponsor and support is a must for getting the utmost benefits from DevOps. A common vision and strategy should be defined between executives and teams. Examples of common strategies could be “No more Enterprise releases”, “There is only one layer between business and the engineering team”, “No more manual testing”, etc. A team achieves this common goal by using creativity and problem solving to lead to a joint, envisioned mission.
Get outside help
Organizations should look for outside help to augment the internal skills. Neither outsourcing the entire DevOps work nor fully depending on the existing in-house talent will give any company the maximum benefit. In order to move fast and take advantage of external knowledge, companies should find the right partner to work with. This will allow them to either delegate the day to day work to the partner and the internal resources spend time in adopting the new DevOps journey related work, or make the partner do all the foundational DevOps work, while teams inherit said system.
Invest in people
Enterprises who desire a digital transformation and practice DevOps should invest in their people first. Not all members will understand DevOps and it will take time before the majority of an organization practices DevOps. Finding and sourcing talent is costly and time consuming. Investing in the existing team will provide a better ROI with increased team morale, better software, and better inherent processes. Companies should provide the necessary training (some could be mandatory), opportunities for upward mobility, and time for people to experiment new ideas (e.g. Google provides time for their engineers to work on anything outside their regular project work, pushing for overall growth rather than specific growth objectives based on company projects).
Enforcement vs Autonomy for teams
Strictly enforcing one tool or language inside an organization will always limit innovation and make it difficult to find a large pool of accessible talent. At the same time, allowing each team to use their own unique tool or their own specialized process will make it difficult for large enterprises to keep track of overall DevOps health. The right solution is to find a balance between control versus innovation and freedom. Enterprises should find a few tools (a small set, e.g. Java, golang, node for back-end programming, angular, react, for front-end, etc) and allow teams to pick what they need from the finite list. A simple and clean process needs to be defined for teams to bring additional tools or languages for their work so that improvements can be made in the current solutions.
The legacy model of software life cycle management such as ITIL, Waterfall, PRINCE2 are focused on being risk averse and have many control options built in. This allows for different teams accountability in their respective functions. With the DevOps model, all the hierarchies and layers are demolished and the team is given more autonomy. At the enterprise level, not all teams will showcase this higher level of responsibility and maturity. Customer expectations cannot be compromised at any time due to change in way of doing things. Organizational level controls need to be established using technology and process to help teams to provide superior product and customer service.
Enterprises can use in-house or COTS solutions to enforce their controls.
For example, Netflix built their own solution (chaos monkey) to randomly delete servers to make sure their applications are resilient to failure of underlying cloud servers. https://medium.com/netflix-techblog/the-netflix-simian-army-16e57fbab116
Texas A&M university uses Machine Learning and Artificial Intelligence to identify and triage security incidents. They were able to bring security incident resolution from a few hours to 10 – 20 mins. ISCACA has 10 DevOps controls recommendations for companies who wish to embrace DevOps practice. http://www.isaca.org/About-ISACA/Press-room/News-Releases/2015/Pages/ISACA-Report-10-Important-Dev-Ops-Controls.aspx
Some companies follow dual model IT. Some legacy COTS products follow old model SDLC and modern software (e.g. inhouse built services, web applications, mobile apps, etc). It is strongly recommended that Enterprises follow only a single model. In order to provide great products to customers, different systems (Legacy COT, SaaS, inhouse solutions) have to work together. Following dual speed IT can create a disconnect in the architecture of systems, different team structures, and the rate of software delivery. Dual speed IT will confuse business and create complexity for them to figure out which part of IT to work with. It will also create roadblocks for collaboration between different parts of the organization, lowering employee morale.
Right Tools and Technology
Enterprises can gain the most out of DevOps by using the right tools and technology. Though teams have autonomy to try new tools and techniques to foster innovation, at the enterprise level certain tools need to be decided upon early, and with intention. Some of the areas where organizations have to choose proper tools are; source code control systems, requirements gathering, central logging, central monitoring and dashboards, and team collaboration (communication and knowledge sharing).
- A single source control system will allow different teams to search and reuse others code without duplication. It will also allow security teams to scan the repos for any security violation and notify teams on a timely basis.
- Using a strong collaboration software will enable teams to share knowledge and easily reach others.
- A central dashboard can enable everybody in the organization to see what is going on with their systems and software. This will bring awareness to the entire organization, also boosting team morale when progress is tracked.
- Whatever tools and technologies are chosen, ease of use should be a huge factor in the decision making process. Tools can be chosen based on the inputs from different teams and based on a preferred consensus.
At the same time solutions should be there to catch teams who violate their SLAs. E.g. for all the issues raised for a given team, what is the violation in SLA to resolve them? Based on the threshold of the teams’ manager, a duly and frequent message to notify the team leader of missing SLAs is necessary.
High performing teams
A high performing team will always exceed expectations, have less conflict, fantastic team morale, higher quality of products, fast delivery and lower associated costs. In order for a team to be performing at their best, three factors need to work in tandem: People, process and technology.
Understanding the current state of teams would be the first step towards starting the DevOps journey. Tools like doitrigt (https://beta.doitright.io) will organizations to capture the situation in their teams and plan action for improving the maturity.
Adopting DevOps practice at an enterprise level is a big task. It is not an impossible task, and is often well worth the time spent in implementation. When enterprises adopt certain fundamentals such as changing their team structures, automating repeatable work, supporting teams to take risks and fail fast, simplifying SDLC process, etc., DevOps practice can be implemented at scale much easier. Enterprises also need to define and build controls, use the right technology and tools, as to not impede customer satisfaction and usability. If done properly, correct use of these systems can lead to sustained growth in any enterprise.
IS YOUR COMPANY LOOKING FOR DIGITAL SOLUTIONS?