
By SmartBear Blog | Article Rating: |
|
December 11, 2017 03:45 PM EST | Reads: |
207 |
AI and Advanced Time Series Analysis for Monitored Applications
By Abraham Nevado
While having a reliable monitoring solution for your application is important, being able to parametrize and configure thresholds and alerting is even more critical.
No matter what kind of market your business is in, your web applications have seasonal patterns. For example, the load of a conventional airline ticketing system fluctuates heavily over the hours of the day, the day of the week, the month, and eventually a particular day of the year. Think about the travel difference between New Year's Eve and the day before Thanksgiving.
Your application may experience some normal fluctuations that are dynamic in nature, preventing you to define appropriate thresholds with a classical static threshold based solution. Additionally, some fluctuations can be very minimal or hidden in the vast amount of data your monitoring solution may be collecting. This information could be a good pseudo predictive indicator that something weird has been triggered but if you're unable to see it, you're surely not using it.
However, despite this criticality, defining convenient thresholds is not an easy task, sometimes even impossible. Proliferation of machine learning techniques leveraged by serious economic interests, like banking fraud prevention, has created numerous collection techniques to detect unusual data across any time series data.
From operation's perspective, these techniques alone do not offer significant benefits as seasonality plays a critical role as well. There are monitoring solutions that can be configured to apply anomaly detection algorithms after you specify a seasonality pattern. However, this approach has some limitations:
- You are required to know the seasonality of all your applications
- Your application seasonality may change over the time, enforcing you to periodically check your "expected" seasonality still is valid.
It is usual while doing time series analysis to decompose any dataset into three components:
- Seasonal Component - includes the expected recurrent behavior of the time series
- Trend Component - includes the observed trend over time
- Remainder Component - includes specific usage not part of the trend or the periodic behavior
These components can be used for a variety of different purposes including predictive alerting, triggering a message every time the remainder component exceeds a fixed dimension, and capacity planning based on the trend component.
To illustrate all these concepts lets used the dataset of monthly totals of international airline passengers between the years 1949 to 1960:
As indicated earlier, a particular time series can be considered the aggregation of three different series that include the seasonality, the trend and of course the remainder components.
It is obvious that at any given point in time the actual data is equal to the sum of its three decomposed components:
data(t) = seasonal(t) + trend(t) + remainder(t)
Depending on the kind of analysis to be performed one of these components may perform better than the others. For example, the "remainder" component could have been used by the airline companies to detect lower than usual passenger volume:
The blue arrows are pointing to the anomalies in the remainder data that were hidden from the "data" time series chart at the top.
Time series analysis decomposition has been available for a long time, however to perform this analysis you need to manually infer the periodicity. Using our airline example, you would have to first identify the seasonality and then manually enter it into the algorithm.
Requiring this knowledge "a priori" has prevented most monitoring solutions to include this analysis in a completely unassisted way. At SmartBear we see the immense potential of time series analysis and are working to incorporate this in an automated way into AlertSite.
Rather than going into an exhaustive description of the process, here's a simple glimpse of how this works.
The problem we want to address is to automatically discover the seasonality of any of the data gathered by our customer's monitors. Fourier Analysis allows to transform any "periodic" dataset into an equivalent frequency representation. This is like what you see on an equalizer when playing your favorite music.
The main idea behind Fourier Analysis is to represent a function as a finite sum of essential frequencies, called harmonics. If the energy of all different harmonics is depicted, we get what is called a periodogram. Here's a periodogram for our airline travel example:
There is a clear outstanding frequency flagged with the blue arrow, it stands out from the rest of them because despite not being the absolute biggest, it is the biggest across its neighbors. It is the third one ordered by amplitude and belongs to frequency: 0.083333333 Hz. This means our data has a strong seasonality every (1/0.083333333= 12) 12 samples. Knowing that our data has a sample every month, it is clear that our airline data has a strong 12 months seasonality pattern.
In summary, once seasonality is discovered the time series can be decomposed, and the variety of available functions are enormous, especially when analyzing the "remainder" component.
This is part one of my three-part series, stay tuned for the next installment where I'll discuss how this "remainder" component can be used employing statistics techniques and machine learning to flag abnormalities automatically that can be used for predictive alerting, for example.
CloudExpo | DXWorldEXPO have announced the conference tracks for Cloud Expo 2018, introducing DXWorldEXPO.
DXWordEXPO, colocated with Cloud Expo will be held June 5-7, 2018, at the Javits Center in New York City, and November 6-8, 2018, at the Santa Clara Convention Center, Santa Clara, CA.
Digital Transformation (DX) is a major focus with the introduction of DXWorld Expo within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term.
A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes.
Cloud Expo | DXWorldEXPO 2018 New York
(June 5-7, 2018, Javits Center, Manhattan)
Cloud Expo | DXWorldEXPO 2018 Silicon Valley
(November 6-8, 2018, Santa Clara Convention Center, CA)
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
DX World EXPO, LLC., a Lighthouse Point, Florida-based startup trade show producer and the creator of DXWorldEXPO® - Digital Transformation Conference & Expo has announced its conference agenda, with three major themes:
* Technology - The Revolution Continues
* Economy - The 21st Century Emerges
* Society - The Big Issues
"DX encompasses the continuing technology revolution, and is addressing society's most important issues throughout the entire $78 trillion 21st-century global economy," said Roger Strukhoff, Conference Chair. "DXWorldExpo has organized these issues along 10 tracks with more than 500 of the world's top speakers coming to Istanbul to help change the world."
There are 10 tracks running throughout the event and following the major themes. More than 500 breakout sessions will be featured, along with keynotes and general sessions from some of the world's top technology, business, and societal leaders. The event will be held over five days in Istanbul, reflecting the global nature of Digital Transformation and the city's long and historic role as a key business and intellectual center and linchpin between East and West.
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
DX World Expo's Global Themes and Tracks are as follows:
Technology - The Revolution Continues
DX Tech: Data-Driven Global 2000
DX Tech: The Blockchain Challenge
DX Tech: AI and Cognitive
DX Tech: The Global Cloud
Economy - The 21st Century Emerges
DX Econ: Software is Rewriting the World
DX Econ: Smart Cities, Nations, and Regions
DX Econ: FinTech and the Token Economy
DX Econ: The Industrial Internet and Industrie 4.0
Society - The Big Issues
DX Society: Environment
DX Society: Education
DX Society: Agriculture
DX Society: Health Care
Call for Papers -speaking opportunities- as well as sponsorship and exhibit opportunities will open on November 1, 2017.
World's Most Important Tech Event
DXWorldEXPO® will be the world's most important tech event with 1,000 exhibitors in its first year and 2,000 exhibitors in its second year, as it guides Global 2000 companies through their Digital Transformation journey for the next two decades. The global event is set to launch September 17-20, 2018 in Istanbul. It will be sponsored by Fortune 50 companies, and more than 30 international banks will be among sponsors of its FinTech/InsurTech track.
Gaining a better understanding of customers and acting upon this information is the foundation of Digital Transformation (DX) in the enterprise. Applying the latest technologies in this area is the key to driving new topline revenue opportunities.
Global 2000 companies have more than US$40 trillion in annual revenue - more than 50% of the world's entire GDP. The Global 2000 spends a total of US$2.4 trillion annually on enterprise IT. The average Global 2000 company has US$11 billion in annual revenue. The average Global 2000 company spends more than $600 million annually on enterprise IT.
Governments throughout the world spend another US$500 billion on IT - much of it dedicated to new Smart City initiatives. There are more than a dozen Global 2000 companies in Turkey, including Isbank, Garanti Bank, other financial institutions, Turk Telecom, Turkcell, Turkish Airlines, and ENKA.
Cloud Expo | DXWorldEXPO 2018 New York
(June 5-7, 2018, Javits Center, Manhattan)
Cloud Expo | DXWorldEXPO 2018 Silicon Valley
(November 6-8, 2018, Santa Clara Convention Center, CA)
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
More Management Quotes
"For the past 10 years at Cloud Expo, we've helped drive the migration to modern enterprise IT infrastructures, built upon the foundation of cloud computing. Today's hybrid, multiple cloud IT infrastructures integrate Big Data, analytics, blockchain, the IoT, mobile devices, and the latest in cryptography and enterprise-grade security," said Fuat Kircaali, Chairman and founder of DX World Expo, LLC.
"As a report from the World Economic Forum and Accenture recently stated, 'Companies need to fundamentally change the way they identify, develop and launch new business ventures.' We agree," said Carmen Gonzalez, president of DX World Expo, LLC.
"Digital Transformation is the key issue driving the global enterprise IT business," said Roger Strukhoff, Conference Chair and Executive Director of the Tau Institute for Global ICT Studies. "DX is most prominent among Global 2000 enterprises and government institutions. Our new event in Istanbul brings together the top companies and delegates from around the world, who are transforming the world."
2018 Conference Agenda and Tracks, June 5-7, Javits Center
Track 1 | Cloud Expo - Enterprise Cloud
Track 2 | DXWorld Expo - Digital Transformation (DX)
Track 3 | The API Enterprise | Mobility & Security
Track 4 | DevOps | Containers & Microservices
Track 5 | Cognitive Computing | AI, ML, DL
Track 6 | Big Data | Analytics
Track 7 | IoT | IIoT | Smart Cities
Track 8 | Hot Topics | FinTech | WebRTC
Cloud Expo covers all of these tools, with the most comprehensive program and more than 120 top world-class speakers throughout our Industry presenting Keynotes, General Sessions, Breakout Sessions along eight focused tracks, as well as our signature Power Panels. Our expo floor brings together the world's leading companies throughout the world of Cloud Computing, DX, and all they entail.
As your enterprise creates a vision and strategy that enables you to create your own unique, long-term success, learning about all the technologies involved is essential. Companies today not only form multi-cloud and hybrid cloud architectures, but create them with built-in cognitive capabilities. Cloud-native thinking is now the norm in financial services, manufacturing, telco, healthcare, transportation, energy, media, entertainment, retail and other consumer industries, as well as the public sector.
Cloud Expo is the world's most important, independent event where technology buyers and vendors meet to experience and discuss the big picture of Digital Tranformation and all of the strategies, tactics, and tools they need to realize their goals.
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
Only Cloud Expo brings together all this in a single location:
- Cloud Computing
- Big Data & Analytics
- Software-Defined Infrastructure
- Industrial IoT
- Industry 4.0
- Artificial Intelligence
- Cognitive Computing
- Microservices
- Machine Learning
- DevOps
- WebRTC
- FinTech
- Digital Transformation
Attend Cloud Expo. Build your own custom experience. Learn about the world's latest technologies and chart your course to Digital Transformation.
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
Download Show Prospectus ▸ Here
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some form of XaaS - software, platform, and infrastructure as a service.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo, October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
2018 Conference Agenda and Tracks, June 5-7, Javits Center
Track 1 | Cloud Expo - Enterprise Cloud
Track 2 | DXWorld Expo - Digital Transformation (DX)
Track 3 | The API Enterprise | Mobility & Security
Track 4 | DevOps | Containers & Microservices
Track 5 | Cognitive Computing | AI, ML, DL
Track 6 | Big Data | Analytics
Track 7 | IoT | IIoT | Smart Cities
Track 8 | Hot Topics | FinTech | WebRTC
Cloud Expo | DXWorldEXPO 2018 New York
(June 5-7, 2018, Javits Center, Manhattan)
Cloud Expo | DXWorldEXPO 2018 Silicon Valley
(November 6-8, 2018, Santa Clara Convention Center, CA)
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
Download Show Prospectus ▸ Here
Every Global 2000 enterprise in the world is now integrating cloud computing in some form into its IT development and operations. Midsize and small businesses are also migrating to the cloud in increasing numbers.
Companies are each developing their unique mix of cloud technologies and services, forming multi-cloud and hybrid cloud architectures and deployments across all major industries. Cloud-driven thinking has become the norm in financial services, manufacturing, telco, healthcare, transportation, energy, media, entertainment, retail and other consumer industries, and the public sector.
Cloud Expo is the single show where technology buyers and vendors can meet to experience and discus cloud computing and all that it entails. Sponsors of Cloud Expo will benefit from unmatched branding, profile building and lead generation opportunities through:
- Featured on-site presentation and ongoing on-demand webcast exposure to a captive audience of industry decision-makers.
- Showcase exhibition during our new extended dedicated expo hours
- Breakout Session Priority scheduling for Sponsors that have been guaranteed a 35-minute technical session
- Online advertising in SYS-CON's i-Technology Publications
- Capitalize on our Comprehensive Marketing efforts leading up to the show with print mailings, e-newsletters and extensive online media coverage.
- Unprecedented PR Coverage: Editorial Coverage on Cloud Computing Journal.
- Tweetup to over 75,000 plus followers
- Press releases sent on major wire services to over 500 industry analysts.
For more information on sponsorship, exhibit, and keynote opportunities, contact Carmen Gonzalez by email at events (at) sys-con.com, or by phone 201 802-3021.
The World's Largest "Cloud Digital Transformation" Event
@CloudExpo | @ThingsExpo 2017 Silicon Valley
(Oct. 31 - Nov. 2, 2017, Santa Clara Convention Center, CA)
@CloudExpo | @ThingsExpo 2018 New York
(June 12-14, 2018, Javits Center, Manhattan)
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
Sponsorship Opportunities
Sponsors of Cloud Expo | @ThingsExpo will benefit from unmatched branding, profile building and lead generation opportunities through:
- Featured on-site presentation and ongoing on-demand webcast exposure to a captive audience of industry decision-makers
- Showcase exhibition during our new extended dedicated expo hours
- Breakout Session Priority scheduling for Sponsors that have been guaranteed a 35 minute technical session
- Online targeted advertising in SYS-CON's i-Technology Publications
- Capitalize on our Comprehensive Marketing efforts leading up to the show with print mailings, e-newsletters and extensive online media coverage
- Unprecedented Marketing Coverage: Editorial Coverage on ITweetup to over 100,000 plus followers, press releases sent on major wire services to over 500 industry analysts
For more information on sponsorship, exhibit, and keynote opportunities, contact Carmen Gonzalez (@GonzalezCarmen) today by email at events (at) sys-con.com, or by phone 201 802-3021.
Secrets of Sponsors and Exhibitors ▸ Here
Secrets of Cloud Expo Speakers ▸ Here
All major researchers estimate there will be tens of billions devices - computers, smartphones, tablets, and sensors - connected to the Internet by 2020. This number will continue to grow at a rapid pace for the next several decades.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend @CloudExpo | @ThingsExpo, October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-4, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Delegates to Cloud Expo | @ThingsExpo will be able to attend 8 simultaneous, information-packed education tracks.
There are over 120 breakout sessions in all, with Keynotes, General Sessions, and Power Panels adding to three days of incredibly rich presentations and content.
Join Cloud Expo | @ThingsExpo conference chair Roger Strukhoff (@IoT2040), October 31 - November 2, 2017, Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, for three days of intense Enterprise Cloud and 'Digital Transformation' discussion and focus, including Big Data's indispensable role in IoT, Smart Grids and (IIoT) Industrial Internet of Things, Wearables and Consumer IoT, as well as (new) Digital Transformation in Vertical Markets.
Full Conference Registration "Gold Pass" and Exhibit Hall ▸ Here (Register ▸ Here via EventBrite)
Financial Technology - or FinTech - Is Now Part of the @CloudExpo Program!
Accordingly, attendees at the upcoming 21st Cloud Expo | @ThingsExpo October 31 - November 2, 2017, Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, will find fresh new content in a new track called FinTech, which will incorporate machine learning, artificial intelligence, deep learning, and blockchain into one track.
Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expensive intermediate processes from their businesses.
FinTech brings efficiency as well as the ability to deliver new services and a much improved customer experience throughout the global financial services industry. FinTech is a natural fit with cloud computing, as new services are quickly developed, deployed, and scaled on public, private, and hybrid clouds.
More than US$20 billion in venture capital is being invested in FinTech this year. @CloudExpo is pleased to bring you the latest FinTech developments as an integral part of our program, starting at the 21st International Cloud Expo October 31 - November 2, 2017 in Silicon Valley, and June 12-14, 2018, in New York City.
@CloudExpo is accepting submissions for this new track, so please visit www.CloudComputingExpo.com for the latest information.
Speaking Opportunities
The upcoming 21st International @CloudExpo | @ThingsExpo, October 31 - November 2, 2017, Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY announces that its Call For Papers for speaking opportunities is open.
Submit your speaking proposal today! ▸ Here
About @CloudEXPO and @DXWorldEXPO
SYS-CON Media (www.sys-con.com) has since 1994 been connecting technology companies and customers through a comprehensive content stream - featuring over forty focused subject areas, from Cloud Computing to Web Security - interwoven with market-leading full-scale conferences. The company's internationally recognized brands include among others Cloud Expo® (@CloudExpo), Big Data Expo® (@BigDataExpo), DevOps Summit (@DevOpsSummit), @ThingsExpo® (@ThingsExpo), and DXWorldEXPO® (@ExpoDX).
@CloudExpo® and @ThingsExpo® are registered trademarks of CLOUD EXPO INC.
DXWorldEXPO® is a registered trademark of DX WORLD EXPO LLC.
Read the original blog entry...
Published December 11, 2017 Reads 207
Copyright © 2017 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By SmartBear Blog
As the leader in software quality tools for the connected world, SmartBear supports more than two million software professionals and over 25,000 organizations in 90 countries that use its products to build and deliver the world’s greatest applications. With today’s applications deploying on mobile, Web, desktop, Internet of Things (IoT) or even embedded computing platforms, the connected nature of these applications through public and private APIs presents a unique set of challenges for developers, testers and operations teams. SmartBear's software quality tools assist with code review, functional and load testing, API readiness as well as performance monitoring of these modern applications.
![]() Dec. 11, 2017 04:30 PM EST Reads: 670 |
By Elizabeth White ![]() Dec. 11, 2017 04:15 PM EST Reads: 246 |
By Elizabeth White ![]() Dec. 11, 2017 04:15 PM EST Reads: 714 |
By Liz McMillan ![]() Dec. 11, 2017 02:45 PM EST Reads: 1,051 |
By Pat Romanski ![]() Dec. 11, 2017 02:00 PM EST Reads: 744 |
By Pat Romanski ![]() Dec. 11, 2017 01:56 PM EST Reads: 227 |
By Liz McMillan ![]() Dec. 11, 2017 01:45 PM EST Reads: 662 |
By Pat Romanski ![]() Dec. 11, 2017 01:30 PM EST Reads: 824 |
By Liz McMillan ![]() Dec. 11, 2017 01:15 PM EST Reads: 653 |
By Elizabeth White ![]() Dec. 11, 2017 01:00 PM EST Reads: 690 |
By Liz McMillan ![]() Dec. 11, 2017 12:00 PM EST Reads: 1,075 |
By Liz McMillan ![]() Dec. 11, 2017 11:45 AM EST Reads: 308 |
By Elizabeth White ![]() Dec. 11, 2017 10:45 AM EST Reads: 1,007 |
By Elizabeth White ![]() Dec. 11, 2017 09:00 AM EST Reads: 718 |
By Pat Romanski ![]() Dec. 11, 2017 08:30 AM EST Reads: 945 |
By Liz McMillan ![]() Dec. 11, 2017 08:30 AM EST Reads: 690 |
By Pat Romanski ![]() Dec. 11, 2017 08:30 AM EST Reads: 1,391 |
By Elizabeth White ![]() Dec. 11, 2017 08:30 AM EST Reads: 1,624 |
By Elizabeth White ![]() Dec. 11, 2017 08:00 AM EST Reads: 1,052 |
By Elizabeth White ![]() Dec. 11, 2017 07:45 AM EST Reads: 1,081 |