
By Sematext Blog | Article Rating: |
|
December 19, 2014 11:45 PM EST | Reads: |
4,273 |

Apache Spark is an open-source, large-scale data processing engine built on top of the Hadoop Distributed File System (HDFS) and enables applications in Hadoop clusters to run up to 100x faster in memory, and 10x faster even when running on disk. So it’s not surprising the usage of Spark is booming as this Google Trends graph shows.
And while Spark usage has been going through the roof, Engineers and DevOps handling Spark have not had a good monitoring tool at their disposal. Well, that is, until now. By releasing the first Spark monitoring product to market Sematext has, with the addition of Spark monitoring to SPM Performance Monitoring, Alerting and Anomaly Detection, just filled a big hole in the Spark ecosystem.
Having just been added — along with other goodies — to the latest SPM release, SPM for Spark monitors all Spark metrics. It includes alerting, anomaly detection, log correlation, custom dashboards, events graphing, custom metrics, and a ton more. SPM can be installed On Premises or one can use the Cloud version run by Sematext, in which case the setup takes less than 5 minutes before graphs with performance metrics start appearing in real-time.
Enough with the words – Show me what Spark Monitoring looks like!
Have a look at a few screenshots to see how we graph Spark metrics in SPM. While we don’t use Spark at Sematext at this time and thus don’t have a live demo to show you, you can check out SPM’s live demo and see some other types of apps we monitor, such as Hadoop, HBase, Cassandra, Kafka, Storm, ZooKeeper, Elasticsearch, Solr, NGINX and NGINX Plus, Apache, MySQL, Redis, Java webapps and generic Java applications, as well as custom metrics.
Screenshot – Spark Executor metrics [click to enlarge]
Screenshot – Spark Worker metrics [click to enlarge]
And One More Thing…
SPM now works hand-in-hand with Logsene Log Management and Analytics. This makes the integration of performance metrics, logs, events and anomalies more robust for those of you looking to combine performance monitoring and centralized log management in one place — not only knowing that SOMETHING affected performance of your Spark cluster when you look at your performance metrics graphs or get an alert, but also exactly WHAT happened with the cluster by having immediate access to all relevant Spark event logs right there!
Take a Test Drive — It’s Easy and Free to Get Started
Like what you see here? Sound like something that could benefit your organization? Then try SPM and/or Logsene for Free for 30 days by registering here. There’s no commitment and no credit card required.
Filed under: Monitoring
Tagged: announcement, Apache Spark, monitoring, performance, performance monitoring, Spark, spm
Read the original blog entry...
Published December 19, 2014 Reads 4,273
Copyright © 2014 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.
More Stories By Sematext Blog
Sematext is a globally distributed organization that builds innovative Cloud and On Premises solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), and search analytics (SSA). We also provide Search and Big Data consulting services and offer 24/7 production support for Solr and Elasticsearch.
![]() Nov. 16, 2017 12:00 PM EST Reads: 880 |
By Pat Romanski ![]() Nov. 16, 2017 11:45 AM EST Reads: 548 |
By Elizabeth White ![]() Nov. 16, 2017 10:00 AM EST Reads: 496 |
By Elizabeth White ![]() Nov. 16, 2017 08:45 AM EST Reads: 749 |
By Liz McMillan ![]() Nov. 16, 2017 08:15 AM EST Reads: 848 |
By Liz McMillan ![]() Nov. 16, 2017 07:45 AM EST Reads: 611 |
By Yeshim Deniz ![]() Nov. 15, 2017 10:45 PM EST Reads: 1,921 |
By Elizabeth White ![]() Nov. 15, 2017 03:45 PM EST Reads: 2,707 |
By Elizabeth White ![]() Nov. 12, 2017 01:45 PM EST Reads: 988 |
By Pat Romanski ![]() Nov. 12, 2017 10:00 AM EST Reads: 2,173 |
By Roger Strukhoff ![]() Nov. 10, 2017 12:00 PM EST Reads: 2,631 |
By Yeshim Deniz ![]() Nov. 8, 2017 10:15 AM EST Reads: 1,963 |
By Yeshim Deniz ![]() Nov. 7, 2017 11:00 AM EST Reads: 2,194 |
By Elizabeth White ![]() Nov. 5, 2017 05:15 PM EST Reads: 2,028 |
By Yeshim Deniz ![]() Nov. 3, 2017 12:30 AM EDT Reads: 2,092 |
By Liz McMillan ![]() Nov. 2, 2017 12:00 PM EDT Reads: 1,473 |
By Elizabeth White ![]() Nov. 2, 2017 10:00 AM EDT Reads: 1,599 |
By Yeshim Deniz ![]() Nov. 2, 2017 08:30 AM EDT Reads: 1,705 |
By Pat Romanski Nov. 2, 2017 08:15 AM EDT Reads: 2,890 |
By Yeshim Deniz ![]() Nov. 2, 2017 04:00 AM EDT Reads: 1,972 |