The State of DevOps at the Start of 2017
By Reanna Miranda

The word—or rather, the contraction—is simple: DevOps. But DevOps, which refers to the increased communication and collaboration between development and IT operations, is an ever-changing, sometimes complicated term. While “dev” and “ops” used to be siloed into separate philosophies, practices, tools, and work flows, they’re merging into one. The result? A more efficient, reliable process and product.

Though the basics aren’t exactly new, the DevOps world is still trying to figure out how it will evolve and integrate into already established infrastructures. Whether you head a company that relies heavily on DevOps or you’re just now dipping your toes into the DevOps pond, there’s plenty to learn—because this movement and culture are anything but set in stone. To get a better feel for what’s in store for DevOps in 2017, we reached out to the pros. Read on for challenges and changes you may encounter in the coming year.

The Experts:

Justin Vaughan-Brown: Director of Product Marketing at AppDynamics

Kevin Gillis: Software Product Management Leader (former DevOps leader at SmartBear)

Grant Smith: DevOps consultant at Just Eat and author of The Next Gen DevOps Transformation Framework

Stephen Thair: Co-Founder and CTO of DevOpsGuys

How has DevOps changed the IT landscape in the last few years?
There are increasing signs of improved collaboration between Dev and Ops teams to work on a shared goal, which is the continuous delivery of high-quality applications to customers, employees, and partners. The days of Dev throwing code over the wall to Ops and each of them blaming each other for either bad code or a poor infrastructure are hopefully over. There is a sense of responsibility amongst Dev teams that they cannot work in isolation and need to partner with Ops to not only troubleshoot any issues, but also ensure that these do not occur again. —Justin Vaughan-Brown

DevOps has completely changed how the technology industry tackles engineering problems. DevOps turned the conversation away from a narrow focus on software and SDLCs to services and engineering. 2009 was a landmark year; more than a decade after we began running online services we finally started considering how we build, launch, manage, and support online services as a whole rather than just how we build the software. —Grant Smith

Dramatically. Departments, budgets, roles responsibilities, and above all—expectations. Annual product releases gave way to quarterly, then monthly, weekly, daily, and now several times a day. Now there are micro code check-ins, micro builds, micro testing, micro releases, and micro updates. Customers, the market, and executive management want everything now.

The lines have been completely blurred and that creates uncertainty and confusion as responsibilities go from Ops to Developers. To enable this shift in responsibilities and dramatic speed to release, it means automation of all stages in the development, testing, and production process, and that dev teams lead the charge and decision-making on tools and technology. My front row seat in particular showed how QA departments are driving toward more and more automation—some 100 percent—which means no manual/UX testing. This creates new risks around quality and user experience and has led to some serious philosophical debates in the industry. —Kevin Gillis

I think DevOps has fundamentally changed the IT landscape, and that change was long overdue. When you look at the stats on project delivery, you can see that most large enterprise IT departments are struggling to deliver. At best, 25 percent of the IT spend is wasted on failed projects and rework (some estimates have that as high as 50 percent or more). When you add the huge changes required across the organization to deal with digital disruption and digital transformation, it’s obvious something has to change. We need a new operating model for IT, and that new model is DevOps. Organizations are seeking to restructure their IT delivery models away from the short-term, cost-focused, project-based models to full lifecycle, innovation-focused, product-centric models based around cross-functional DevOps-enabled teams, using the newest cloud-hosted automated tools and technologies. This change isn’t easy. It is a cultural shift from command and control management to true leadership, surrendering centralized control to autonomous teams, as well as learning a raft of new technical skills in order to stay relevant in this new world. —Stephen Thair

What are some challenges you think the DevOps world will face in 2017?
I have been a DevOps evangelist for some time now but I had a sobering experience recently. I attended a Gartner Data Centre Summit in London at the end of November, and during the DevOps and I & O breakout, analyst Ian Head suggested that around 87 percent of those surveyed at each Gartner conference said they were not getting what they hoped for from DevOps. DevOps is just past Gartner’s peak of inflated expectations and I believe it now has to push past the hype and deliver.

This past year was dominated by data breaches, hacking stories, and a general sense of disquiet at how vulnerable our personal information is to a multitude of threats. Going into 2017, security needs to be an in-built consideration in any new product release, not an afterthought. There is huge partnership potential between DevOps and Security, which is largely unrealized. —Justin Vaughan-Brown

Availability of good engineers is the single biggest challenge facing the technology industry today—and it shows no sign of improving. Many businesses are investing in bootstrapping graduates where five years ago they would have demanded three or five years experience. Some universities have started offering DevOps courses or modules, but many computer science graduates still leave university having never configured the environments that run their code. —Grant Smith

There’s a notion, particularly with the ease of updating SaaS and Mobile-first applications, that speed wins—but at what expense? Quality, customer experience, brand erosion? Companies are struggling with this. At one conference, I heard Uber’s Denali Lumma talk about having thousands of code repos and getting build cycles in an intense CI/CD environment means everyone—even developers—are on call. Defects in production may go straight to the developer at 3AM! Still, fox-in-the-hen-house debates persist about developers picking up more and more of the testing. Companies like Facebook and Expedia are moving away from formal QA and more toward testing by developers and tools like Jenkins. I suspect that outside of really talented engineers, commercial-grade testing is beyond most developers. Automated testing is good regardless of dev methodology, but with DevOps, it’s a requirement and not an elective. —Kevin Gillis

I think it’s inevitable that in 2017 we’ll start to see a DevOps backlash. As we move through the hype cycle from the peak of inflated expectations into the trough of disillusionment—as happens with any new technology or methodology—there will be articles saying “DevOps is Dead,” and that’s fine. What doesn’t change is that the old, siloed, hierarchical models are still broken. We need new ways to deliver IT products and services, particularly as cloud SaaS and PaaS services evolve, which in turn means software architecture evolves (e.g. the current hype over server-less computing). The other biggest challenge the DevOps world will face is a massive skills shortage. Currently there is somewhat of a zero-sum game going on with experienced DevOps people in high demand, which means people jump from company to company. This isn’t sustainable in the long run, which is why we (DevOpsGuys) are investing heavily in 2017 to expand our education arm and set up our own DevOps Academy to train people in the theory and cultural aspects of DevOps as well as practical hands-on tools and technology. —Stephen Thair

How will DevOps advance in 2017?
Organizations are increasingly data centric in nearly all respects: from where to invest to win new customers to smart Algorithmic IT Operations (AIOps) and data-driven development. DevOps will be no different. As industries continue to adopt DevOps, KPIs will be shared and enable benchmarking with peers. Insights from the application performance perspective will remain critical in understanding the customer experience and the exact nature of each business transaction.

There are also a ton of events for DevOps practitioners to gather and share: MeetUps, summits, conferences, virtual shows, and more. Learning from people who have initiated change and had a measurable impact reassures people at the start of the DevOps journey.

And finally, DevOps is having a direct impact on businesses in terms of customer satisfaction, retention, and acquisition. This gives Development and Operations heads the mandate to engage business colleagues and understand where they can focus efforts next, and be truly aligned. AppSphere saw the launch of Business iQ, which shows the correlation between the customer online experience and the bottom line. This is just one example of how IT teams can share actionable insights with business unit colleagues and foster deeper collaboration. —Justin Vaughan-Brown

Server-less is going to be part of every progressive organization operating model. The benefits are too huge to ignore given how easy it is to implement. —Grant Smith

The entire Software Development Lifecycle (SDLC) is shrinking. And this requires more controls to measure and track effectiveness and productivity. I love the idea of Facebook using a scoring system where developers lose points for bad releases/defects that make it to production. Test automation and test parallelization will receive more attention. Until code automation/AI is more prevalent, the bottleneck is testing and deployment—and that can be addressed with automation (e.g. testing, code check-ins, builds, integration), APIs, and deployment. —Kevin Gillis

DevOps will advance as we expand the practitioner knowledgebase. As more and more organizations adopt DevOps, we’ll see more case studies and real-world examples of how to effectively manage the DevOps transition—especially for large enterprise organizations. There is a difference between organic and transformational DevOps, and the challenges a globally-distributed enterprise with thousands of staff faces are completely different than those faced by a small start-up implementing DevOps as they grow. That said, the goal is to get all organizations to communicate more effectively, better manage their products, and cope with the rates of change we’ll see over the next five years. And for good or bad, we will also see more DevOps certification and standardization efforts in 2017. We’ve already seen at least three organizations claim they’re producing a “Certified DevOps” curriculum. Even Axelos, the commercial owners of the ITIL standard, are getting in on the act. —Stephen Thair

What are some DevOps practices you think will become mainstream in 2017?
I hope to see some form of reorganization and restructuring that breaks down traditional developmental divides. As organizations evolve to become more agile, expect traditional org structures to be challenged, revised, and even abandoned entirely.

“Shadow IT” and “citizen developers” are just two terms being shared to describe the growing impact the business will have on tech spend decisions, and IT teams need to have a plan for this. Equally, each employee’s field of vision is becoming extended, as their touch points with the wider organization increase. We will see a parallel loss of direct control and increase in influence further afield as the extended IT ecosystem grows. The days of empire building are fast receding.

And expect to see a new role emerge in the coming years called the Site Reliability Engineer (SRE). A term initially created by Benjamin Treynor Sloss, who leads global ops, networking, and engineering within the Google team, the SRE is sometimes a developer who now sit in an Ops role and is focused on availability, stability, latency, and performance. —Justin Vaughan-Brown

Anomaly detection. Innovations in logging and monitoring capabilities have now got us to the point where we can apply machine learning algorithms to metrics directly. You still have to identify your primary constraint metrics, but once that’s done you can have the computers stare at the graphs for you. —Grant Smith

Many larger, more mature companies are still transforming their product delivery teams to Agile and Scrum, and DevOps is still just a glimmer. For those currently running DevOps, expect CI/CD and daily micro releases to become more and more mainstream. Waterfall was measured in quarters, agile in weeks. But DevOps is now measured in days—even hours. —Kevin Gillis

The low hanging fruit of DevOps automation practices (continuous integration, test automation, automated provisioning, configuration management, application release automation, etc.) will be increasingly adopted. They can buy you some time and breathing space and help you pay back enough technical debt so you can focus on the wider digital and DevOps transformation agenda. Measurement-driven approaches to DevOps will be huge in 2017. We are already seeing organizations collecting and analyzing huge amounts of data about their customers, applications, and infrastructure. We’re also seeing more organizations use this information to guide decision making and product strategies, and where they invest their time and effort. Both open source and vendor solutions, such as AppDynamics, will play a role here. —Stephen Thair

Do you think DevOps will become more standardized?
DevOps has different implications for each organization, in how it is adopted, evangelized, measured, and grown. There are some common metrics such as release velocity, Mean Time to Recovery, Mean Time to Resolution, and so forth, but the overarching vision will be determined by the organization’s industry, culture, business goals, and market positioning rather than a pre-defined program. That said, the build, test, and deployment approaches should be standardized within each organization. The DevOps Institute offers a training and certification process (which has its supporters and detractors). Naturally, any accreditation needs to be backed by real-world experience for a candidate to be considered seriously. —Justin Vaughan-Brown

DevOps is actually highly standardized. By my reckoning, DevOps is in its third generation now. For those early adopters, standardization is based around their own internal code bases. Facebook, Netflix, Google, and Etsy have rich, publicly available code repositories, which show how they’ve developed. The second generation of DevOps organizations all standardized around AWS and CI tools like Jenkins and Teamcity, and have invested heavily in ELK and visualization tools. The third generation of DevOps organizations have standardized around containers, microservices, and server-less functions. And they’ve fully embraced SAAS offerings for non-strategic requirements like log and metric analysis. —Grant Smith

Yes, but it will take one or two key, industry-wide initiatives to drive this. For instance, Sonatype helps companies with the quality, security, and speed of their software supply chains, and SmartBear’s SwaggerHub helps standardize the building, documentation, and deploying of APIs. —Kevin Gillis

The IEEE are already working on a DevOps Standard 2675 to “… specify practices for groups including development, operations, and other key stakeholders to collaborate and communicate effectively to build, package, and deploy software and systems in a secure and reliable way.” Note that the cultural aspects of DevOps, which are harder to measure and define, are out of scope for 2675. ISO are also working closely with IEEE on the DevOps standard.  —Stephen Thair

Why should organizations who haven’t yet adopted DevOps do so in 2017?
Quite simply, they risk of falling behind the competition and becoming dinosaurs. The application experience is a huge determinant of the brand (even more so if you are an online only business), and if the competition delivers better quality and richer customer experience than you, your customer base is at huge risk. The good news: DevOps is being adopted as an approach in all sectors, and success stories are out there. If you know little about DevOps, a good starting point would be to watch some of the excellent videos on YouTube from luminaries such as Jez Humble, Gene Kim, Dave Farley, and Dan North, and read key books such as The Phoenix Project and the more recently released DevOps Handbook. —Justin Vaughan-Brown

The reason for adopting DevOps working practices are the same now as they were when the term was first coined. It leads to better solutions that offer customers better products and delivers innovation faster than the alternative. If your organization doesn’t need to improve how it delivers online services, then you can continue as you have been. —Grant Smith

Because probably most of their competitors have! More seriously, I see this shift being very similar to what I saw with Agile/Scrum. About 10 years ago, my entire product management and R&D team was trained on Scrum by Mountain Goat Software. It was a company-wide initiative and there was great resistance from the one high-performing team that was shipping on time with the right quality and scope.

I asked Ken Schwaber (Co-Founder of Scrum) to speak to my team and he told me “don’t switch to Scrum, because what you have is working!” It’s very similar for DevOps—if your customers, competitors, and the market are not demanding it, don’t switch. But if you need a more responsive methodology and quicker time to market, your organization should start with small, focused, and non-critical projects in 2017 before making company-wide institutional changes. —Kevin Gillis

If you don’t significantly improve your ability to deliver digital products in the next two to three years, your organization will go out of business. The IDC predicts that by 2018, 66 percent of global 2000 enterprises will have digital transformation at the center of their corporate strategy. Doing things the old way just isn’t good enough anymore. —Stephen Thair

Do you have any additional thoughts on the state of DevOps in the coming year?
It’s going to be one of hard work, but hopefully real progress. After all the excitement and noise DevOps has created, it’s time to roll up your sleeves and tackle the hard stuff, if you haven’t already done so. This means expanding initial wins to other teams (geographically and business units), promoting the tangible benefits, and being brave enough to challenge established thinking. It takes real belief to do something different and challenge the status quo, but if there was a time to do so, it is now. —Justin Vaughan-Brown

We in DevOps need to get better at delivering the basics quickly. The exciting and challenging problems aren’t building infrastructure, deploying containers, or exposing metrics and logs through query engines. Data is the next great frontier. We can manage software with software, infrastructure with software, and networks with software—and yet when it comes to managing data we’re still scrambling around for flint to make fire. We either need to find very sophisticated solutions to version, deploy, roll back, and delete records in large data structures, or we need to get much smarter about how we store and manipulate customer data. (My bet is figure out the latter before we get the former.) —Grant Smith

Watch for the “Ops” movement to become contagion: DevOpos, DevSecOps, BizOps, and for you beer lovers, even HopOps! As with the shift from waterfall to modified waterfall to various flavors of Agile, expect to see many DevOps variants. As your company assesses, don’t be too prescriptive or pedantic, but instead match up with what you need to accomplish your company’s business goals and then figure out where DevOps fits in your organization. —Kevin Gillis

Tech Companies already lead the Fortune 500 by market capitalization, and the speed at which unicorns like Uber, AirBnB, WhatsApp, Nest, and Beats reached multi-billion dollar valuations is phenomenal. A new entrant can disrupt established companies just as quickly, so you need strong digital capabilities to be able to react to changes in your market. DevOps is the new operating model for IT that sits at the heart of your digital capabilities. —Stephen Thair

More About Our Experts

Justin Vaughan-Brown
Justin Vaughan-Brown is Director of Product Marketing at AppDynamics, with a specific focus on DevOps. He has previously held DevOps evangelist and digital transformation global lead roles, and was CMO of DevOps consultancy Contino. He has 20 years of technology marketing experience with Software AG, CA Technologies, Microsoft, BusinessObjects, and SAS, amongst others. He has also held senior European marketing roles with data integration, document security, and high-performance database vendors. He is the author of “The Digital Transformation Journey: Key Technology Considerations” paper, hosted the quarterly DevOps Influencer Dinners, and is responsible for the DevOps Simulation Experience, an interactive online workshop that explains core DevOps principles.

Kevin Gillis
Kevin has more than 20 years of executive product management experience and is credited with more than 100 product releases, ranging from 1.0 products and services to sourcing. He successfully integrated 10 mergers and acquisitions, together generating more than $600 million in sales. More recently, he served DevOps leader SmartBear Software and Network Management leader Ipswitch, where he held product management leadership positions and helped drive rapid growth. He was Director of Product Management at Trellix/Interland and Into Networks and Product Manager at Lotus/IBM. Kevin graduated with honors from the H. Milton Stewart School of Industrial & Systems Engineering (ISyE) at the Georgia Institute of Technology.

Grant Smith
Grant has created and led high-performance operations teams in some of the largest and fastest growing companies in the U.K. over the last 15 years, and has been at the forefront of the DevOps movement for the last four years. Grant has driven real collaboration between Operations and Development teams at AOL, Electronic Arts, and British Gas by implementing Infrastructure as Code and driving application integration from continuous build systems. Grant has delivered game platforms running in the cloud, enjoyed by millions of players per day, and websites serving one billion page views per month. Most recently he delivered a high-performance, scalable, Internet-of-Things platform for British Gas.

Stephen Thair
Stephen has 25 years of IT experience working for top government and corporate organizations in Australia and the U.K., including BNP Paribas, Vodafone, and Credit Suisse. Prior to co-founding DevOpsGuys, Stephen was the Web Operations Manager for Totaljobs and was responsible for running their online job board platform. Stephen’s passion is creating operationally excellent platforms for web applications that are secure, scalable, and manageable. He is an expert across a wide range of technologies, with Microsoft web server infrastructure being a specialty. In recognition of this expertise, Stephen has been made a Regional Director by Microsoft. Stephen blogs extensively and presents regularly at numerous meetups, webinars, and conferences around the world to evangelize the benefits of DevOps. In 2014 he co-founded the WinOps Conference and Meet-Ups, which are dedicated to promoting DevOps in Windows environments.

The post The State of DevOps at the Start of 2017 appeared first on Application Performance Monitoring Blog | AppDynamics.