The Wayback Machine - https://web.archive.org/web/20161213120412/http://java.sys-con.com/node/3954962

Welcome!

Java IoT Authors: Liz McMillan, Elizabeth White, William Schmarzo, Peter Silva, Eric Robertson

Related Topics: Java IoT, Linux Containers, Containers Expo Blog, @DevOpsSummit

Java IoT: Article

DevOps #Monitoring | @DevOpsSummit #DevOps #IoT #APM #AI #ML #DL

Central to the idea of DevOps is collaboration between all of the teams involved in IT applications and infrastructure

Monitoring in the World of DevOps
By Twain Taylor

Central to the idea of DevOps is collaboration between all of the teams involved in IT applications and infrastructure. Developers, Operations, Quality Assurance, Security, and more, all have stakes in the delivery of a product or service. So, what does collaboration mean when it comes to monitoring in the world of DevOps?

How DevOps Changes The Game
In the past, different teams involved in creating or maintaining an application would first finish their portion completely before passing it on to the next team. For example, the Development team would first write code for the entire app, or specific features in the app, before passing it to the Quality Assurance (QA) team. The QA team would then do their testing and analysis before sending it forward to Operations, and so on.

This is similar to an Olympic relay team where each person runs 100 meters before passing the baton on to the next runner. Now, think of a team using DevOps ideas as all of those runners running simultaneously, in four 100 meter sprints instead of one 400 meter sprint. Instead of waiting for one team to hand off work to the next, each is working concurrently on the application in their own area of focus. The DevOps runners have more opportunities to adjust strategy, and iteratively improve.

Cross-Platform and Cloud-Based Agility
DevOps concepts are commonly referenced alongside other software development practices such as Agile. The ideas behind Agile and DevOps are similar - break down work into smaller increments, iteratively improve in each increment, eliminate toil where it makes sense, and share learnings across the organization.

Of course, too many opinions can lead to its own set of problems. Well architected applications tend to have multiple components and clearly defined areas of concern. This is where distributed architectures such as microservices often come into the picture. This is also why teams that adopt the DevOps methodology produce a lot more releases, as they can address issues more quickly.

waterfall-devops

The Importance Of Workflows
Two minds are better than one and a dozen minds are even better. Transparency and end-to-end visibility are key factors within DevOps for good communication. End-to-end visibility means having everyone on the same page through the entire development process. When everyone is on the same page, the development process becomes a lot smoother, and there are fewer chances of having to redo things.

Faster Iterations Make For Happier Users
What's important to remember among all the mayhem that goes into building an application is that the ultimate goal is to refine and enhance the end-user experience. Since most cloud-based applications need to react swiftly to changing user needs and market shifts, sometimes it is necessary to quickly release updates for these applications. Software developers using DevOps ideas are able to keep up with this requirement easier, as there are fewer bottlenecks in the cross-team workflow.

Apart from the fact that this makes for happy customers, it also makes for happy developers who are much less likely to lose sleep over any bugs, crashes or mishaps. With good communication as a key difference between the traditional waterfall style of development and DevOps methods, this also influences the needs of the tools.

The DevOps style of monitoring, then, is by no means easier than the traditional Waterfall method where teams often just needed a couple of tools for the entire process and a lot of humans to make up the difference. Having a single team work across a wide variety of areas (development, QA, operations, and so on) requires numerous tools that need to work well with each other. Finding a group of tools that are all compatible can be challenging, but the results are well worth it.

One way to consider these tools is to split them into four groups of software-Application Performance monitoring, Infrastructure monitoring, Log analysis, and last but not least, Incident Management.

monitoring

Monitoring Your Applications
Monitoring applications is a key factor in identifying issues (performance, regression, or otherwise) and fixing them quickly as part of a team's iteration. Three of the more popular application performance monitoring (APM) tools are New Relic, Dynatrace, and AppDynamics. Apart from letting teams monitor and manage their software, they also allow for end-user monitoring, which is crucial to ensuring an application is delivering the best experience.

Monitoring Your Infrastructure
The other side of the coin to application performance monitoring is infrastructure monitoring. There are many, many tools for this, including SaaS solutions such as DataDog, LogicMonitor, and SignalFx, as well as hybrid solutions such as Zenoss. Though it's a favorite of teams transitioning from legacy development processes, Nagios alone isn't enough for your DevOps monitoring strategy.

Monitoring Your Logs
Analyzing logs is a crucial part of the iterative improvement cycle and is a critical part of system troubleshooting, debugging, and security incident response. Popular log analysis tools include Splunk, LogEntries, and Loggly. They allow you to study user behavior based on log file analysis. They also let you collect a fast amount of data from various sources in one centralized log stream. This is a convenient way to maintain and analyze log files. Apart from debugging, Log analysis plays a key role in helping you comply with security policies, and regulations and is vital in the process of auditing and inspection.

Quick and Efficient Incident Management
DevOps really outshines the traditional Waterfall approach in the field of incident management. It is with the help of incident management platforms like PagerDuty that cloud-based applications can now encourage users to resolve most of their issues themselves. PagerDuty is unique in its capabilities as it not only allows you to integrate with just about any tool you can imagine, it allows you to customize your incident management workflows to suit different teams. This approach avoids bottlenecks, and helps achieve the ultimate goal of providing the best user experience.

Enhancing the End-User Experience
Following DevOps practices makes applications more resilient as it makes it easier to uncover implementation issues, and discover deficiencies. With infrastructure monitoring tools, a sudden influx of users that would normally crash a system can easily be detected, with application monitoring tools engineers can dive into the specifics of how performance is being impacted, and any issues that have escaped through all the different levels of quality and monitoring often require closer inspection through log analysis tooling. All the while, these tools all tie into an incident management platform like PagerDuty, to quickly enable engineers to triage and collaborate on incidents

As far as enhancing the user experience goes, DevOps is light years ahead of the traditional waterfall method and those looking to keep up with the pace of technology will have to adopt at least some aspects of this approach. Users of applications that are built and monitored by teams that use DevOps methodologies are accustomed to having their problems solved quickly and efficiently. These same users will seldom tolerate long wait times to fix bugs and catastrophes like system crashes, and anyone who wants to make an impact on this market will have to step up their game or be left behind.

The post Monitoring in the World of DevOps appeared first on PagerDuty.

More Stories By PagerDuty Blog

PagerDuty’s operations performance platform helps companies increase reliability. By connecting people, systems and data in a single view, PagerDuty delivers visibility and actionable intelligence across global operations for effective incident resolution management. PagerDuty has over 100 platform partners, and is trusted by Fortune 500 companies and startups alike, including Microsoft, National Instruments, Electronic Arts, Adobe, Rackspace, Etsy, Square and Github.

@ThingsExpo Stories
An IoT product’s log files speak volumes about what’s happening with your products in the field, pinpointing current and potential issues, and enabling you to predict failures and save millions of dollars in inventory. But until recently, no one knew how to listen. In his session at @ThingsExpo, Dan Gettens, Chief Research Officer at OnProcess, discussed recent research by Massachusetts Institute of Technology and OnProcess Technology, where MIT created a new, breakthrough analytics model for s...
What happens when the different parts of a vehicle become smarter than the vehicle itself? As we move toward the era of smart everything, hundreds of entities in a vehicle that communicate with each other, the vehicle and external systems create a need for identity orchestration so that all entities work as a conglomerate. Much like an orchestra without a conductor, without the ability to secure, control, and connect the link between a vehicle’s head unit, devices, and systems and to manage the ...
Complete Internet of Things (IoT) embedded device security is not just about the device but involves the entire product’s identity, data and control integrity, and services traversing the cloud. A device can no longer be looked at as an island; it is a part of a system. In fact, given the cross-domain interactions enabled by IoT it could be a part of many systems. Also, depending on where the device is deployed, for example, in the office building versus a factory floor or oil field, security ha...
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at 20th Cloud Expo, Ed Featherston, director/senior enterprise architect at Collaborative Consulting, will discuss the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
"We build IoT infrastructure products - when you have to integrate different devices, different systems and cloud you have to build an application to do that but we eliminate the need to build an application. Our products can integrate any device, any system, any cloud regardless of protocol," explained Peter Jung, Chief Product Officer at Pulzze Systems, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
SYS-CON Events has announced today that Roger Strukhoff has been named conference chair of Cloud Expo and @ThingsExpo 2017 New York. The 20th Cloud Expo and 7th @ThingsExpo will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "The Internet of Things brings trillions of dollars of opportunity to developers and enterprise IT, no matter how you measure it," stated Roger Strukhoff. "More importantly, it leverages the power of devices and the Internet to enable us all to im...
"ReadyTalk is an audio and web video conferencing provider. We've really come to embrace WebRTC as the platform for our future of technology," explained Dan Cunningham, CTO of ReadyTalk, in this SYS-CON.tv interview at WebRTC Summit at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Everyone knows that truly innovative companies learn as they go along, pushing boundaries in response to market changes and demands. What's more of a mystery is how to balance innovation on a fresh platform built from scratch with the legacy tech stack, product suite and customers that continue to serve as the business' foundation. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, discussed why and how ReadyTalk diverted from healthy revenue and mor...
The Internet of Things (IoT) promises to simplify and streamline our lives by automating routine tasks that distract us from our goals. This promise is based on the ubiquitous deployment of smart, connected devices that link everything from industrial control systems to automobiles to refrigerators. Unfortunately, comparatively few of the devices currently deployed have been developed with an eye toward security, and as the DDoS attacks of late October 2016 have demonstrated, this oversight can ...
Cognitive Computing is becoming the foundation for a new generation of solutions that have the potential to transform business. Unlike traditional approaches to building solutions, a cognitive computing approach allows the data to help determine the way applications are designed. This contrasts with conventional software development that begins with defining logic based on the current way a business operates. In her session at 18th Cloud Expo, Judith S. Hurwitz, President and CEO of Hurwitz & ...
"Once customers get a year into their IoT deployments, they start to realize that they may have been shortsighted in the ways they built out their deployment and the key thing I see a lot of people looking at is - how can I take equipment data, pull it back in an IoT solution and show it in a dashboard," stated Dave McCarthy, Director of Products at Bsquare Corporation, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Bert Loomis was a visionary. This general session will highlight how Bert Loomis and people like him inspire us to build great things with small inventions. In their general session at 19th Cloud Expo, Harold Hannon, Architect at IBM Bluemix, and Michael O'Neill, Strategic Business Development at Nvidia, discussed the accelerating pace of AI development and how IBM Cloud and NVIDIA are partnering to bring AI capabilities to "every day," on-demand. They also reviewed two "free infrastructure" pr...
"At ROHA we develop an app called Catcha. It was developed after we spent a year meeting with, talking to, interacting with senior citizens watching them use their smartphones and talking to them about how they use their smartphones so we could get to know their smartphone behavior," explained Dave Woods, Chief Innovation Officer at ROHA, in this SYS-CON.tv interview at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place June 6-8, 2017, at the Javits Center in New York City, New York, is co-located with 20th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry p...
As data explodes in quantity, importance and from new sources, the need for managing and protecting data residing across physical, virtual, and cloud environments grow with it. Managing data includes protecting it, indexing and classifying it for true, long-term management, compliance and E-Discovery. Commvault can ensure this with a single pane of glass solution – whether in a private cloud, a Service Provider delivered public cloud or a hybrid cloud environment – across the heterogeneous enter...
SYS-CON Events announced today that Fusion, a leading provider of cloud services, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Fusion, a leading provider of integrated cloud solutions to small, medium and large businesses, is the industry’s single source for the cloud. Fusion’s advanced, proprietary cloud service platform enables the integration of leading edge solutions in the cloud, including cloud...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life sett...
Unsecured IoT devices were used to launch crippling DDOS attacks in October 2016, targeting services such as Twitter, Spotify, and GitHub. Subsequent testimony to Congress about potential attacks on office buildings, schools, and hospitals raised the possibility for the IoT to harm and even kill people. What should be done? Does the government need to intervene? This panel at @ThingExpo New York brings together leading IoT and security experts to discuss this very serious topic.
In this strange new world where more and more power is drawn from business technology, companies are effectively straddling two paths on the road to innovation and transformation into digital enterprises. The first path is the heritage trail – with “legacy” technology forming the background. Here, extant technologies are transformed by core IT teams to provide more API-driven approaches. Legacy systems can restrict companies that are transitioning into digital enterprises. To truly become a lead...