Amazon Web Services has released Amazon Lookout for Metrics, a fully managed service that detects anomalies in metrics and helps determine their root cause.
The service helps customers monitor the most important metrics for their business like revenue, web page views, active users, transaction volume, and mobile app installations with greater speed and accuracy. The service also makes it possible to diagnose the root cause of anomalies like unexpected dips in revenue, high rates of abandoned shopping carts, spikes in payment transaction failures, increases in new user sign-ups, and many more—all with no machine learning experience required.
Organisations of all sizes and across industries gather and analyze metrics or key performance indicators (KPIs) to help their businesses run effectively and efficiently. Traditionally, business intelligence (BI) tools are used to manage this data across disparate sources (e.g. structured data stored in a data warehouse, customer relationship management data residing on a third-party platform, or operational metrics kept in local data stores) and create dashboards that can be used to generate reports and alerts if anomalies are detected. But effectively identifying these anomalies is challenging.
Traditional rule-based methods are manual and look for data that falls outside of numerical ranges that have been arbitrarily defined (e.g. provide an alert if transactions per hour fall below a certain number), which results in false alarms if the range is too narrow, or missed anomalies if the range is too broad. These ranges are also static, and don’t change based on evolving conditions like the time of the day, day of the week, seasons, or business cycles. When anomalies get detected, developers, analysts, and business owners can spend weeks trying to identify the root cause of the change before they can take action.
Machine learning offers a compelling solution to the challenges posed by rule-based methods because of its ability to recognize patterns in vast amounts of information, quickly identify anomalies, and dynamically adapt to business cycles and seasonal patterns. However, developing a machine learning model from scratch requires a team of data scientists that can build, train, deploy, monitor, and fine-tune a machine learning model over time.
Amazon Lookout for Metrics is a new machine learning service that automatically detects anomalies in metrics and helps customers identify the root cause. It puts the same technology used by Amazon internally to detect anomalies in its business metrics into the hands of every developer. Customers can connect it to 19 popular data sources, including Amazon Simple Storage Solution (S3), Amazon CloudWatch, Amazon Relational Database Service (RDS), and Amazon Redshift, as well as SaaS applications like Salesforce, Marketo, and Zendesk.
The service automatically inspects and prepares the data, selects the best-suited machine learning algorithm, begins detecting anomalies, groups related anomalies together, and summarizes potential root causes. For example, if a customer’s website traffic dropped suddenly, it can help them quickly determine if an unintentional deactivation of a marketing campaign is the cause. The service also ranks the anomalies by predicted severity so that customers can prioritize which issue to tackle first.
It connects to notification and event services like Amazon Simple Notification Service (SNS), Slack, Pager Duty, and AWS Lambda, allowing customers to create customised alerts or actions like filing a trouble ticket or removing an incorrectly priced product from a retail website. As the service begins returning results, customers also have the ability to provide feedback on the relevancy of detected anomalies via the AWS console or the Application Programming Interface (API), and the service uses this input to continuously improve its accuracy over time.
“From marketing and sales to telecom and gaming, customers in all industries have KPIs that they need to be able to monitor for potential spikes, dips, and other anomalies outside of normal bounds across their business functions,” says Swami Sivasubramanian, vice president of Amazon Machine Learning for AWS. “But catching and diagnosing anomalies in metrics can be challenging, and by the time a root cause has been determined, much more damage has been done than if it had been identified earlier. We’re excited to deliver Amazon Lookout for Metrics to help customers monitor the metrics that are important to their business using an easy-to-use machine learning service that takes advantage of Amazon’s own experience in detecting anomalies at scale and with great accuracy and speed.”
DevFactory is a Dubai-based provider of software and services solutions for global enterprises. “Our flagship product, Quantum Retail, offers intelligent retail-focused supply chain management and inventory optimization solutions to thousands of retail customers,” says Rahul Subrananiam, CEO, DevFactory. “Our customers have volatile sales data that is affected by millions of daily events across categories like stores, products, and departments which fluctuates according to yearly, monthly, and daily seasonality. Understanding the sales patterns and separating anomalous sales from seasonal variations is critical to accurate forecasting and downstream inventory planning. Our existing solution relied on statistical models and often failed to detect anomalous sales behaviours across stores, leading to over or under allocation of inventory to stores, which in turn significantly impacted the overall revenue and customer satisfaction. With Lookout for Metrics, we are able to automatically monitor data across all the important categories with a few clicks and identify anomalous events in nearly 40% of cases that we missed earlier. By quickly identifying such cases, we are able to adjust our inventory planning and distribution across all stores in an optimal way.”
Since its founding in 2001, Slalom has grown into a $1 billion company with over 5,000 employees. Its clients include more than half the Fortune 100, along with startups, nonprofits, and innovative organizations of all kinds. “By leveraging Amazon Lookout for Metrics, our clients will be able to unlock critical data insights quickly and accurately,” says David Frigeri, senior director of data and analytics, Slalom. “Giving our clients the ability to respond to near real-time anomaly detection, adapt rapidly, and anticipate future disruptions and opportunities is a key step towards embracing a modern culture of data.”
Wipro is a global IT consulting and system integration services firm that develops and implements solutions for enterprises across the globe in industries such as financial services, retail, consumer goods, and more. “For us, Amazon Lookout for Metrics is an autonomous service that provides customers with critical insights into security and business data, helping them excel in the cloud,” says Manish Govil, general manager and global head, Wipro AWS Business Group. “Lookout for Metrics has not only reduced our development efforts, but also significantly lowered the time it takes to employ anomaly detection on customer workloads. It has also empowered us to analyze historical and continuous data streams in near real-time, enabling us to find and eliminate anomalies from our customer’s operational and business data. We are excited to bring this AWS service to our customers to help them achieve AI-driven business outcomes in the cloud at scale.”