Amazon Web Services (AWS) has announced nine new capabilities for its machine learning (ML) service, Amazon SageMaker, to make it easier for developers to automate and scale ML workflow.
Key among these is an increased effort to eliminate the kind of racial and demographic bias that has damaged the reputation of artificial intelligence and machine learning tools in recent years.
Making the announcement during the AWS re:Invent conference, being held virtually this year, the company said that the new features bring together powerful new capabilities like faster data preparation, a purpose-built repository for prepared data, workflow automation, greater transparency into training data to mitigate bias and explain predictions, distributed training capabilities to train large models up to two times faster, and model monitoring on edge devices.
With all the attention machine learning has received, it seems like it should be simple to create machine learning models, but it isn’t. In order to create a model, developers need to start with the highly manual process of preparing the data. Then they need to visualise it in notebooks, pick the right algorithm, set up the framework, train the model, tune millions of possible parameters, deploy the model, and monitor its performance.
This process needs to be continuously repeated to ensure that the model is performing as expected over time. In the past, this process put machine learning out of the reach of all but the most skilled developers.
However, says AWS, Amazon SageMaker has changed that. Amazon SageMaker is a fully managed service that removes challenges from each stage of the machine learning process, making it easier and faster for everyday developers and data scientists to build, train, and deploy machine learning models.
Customers utilising Amazon SageMaker include 3M, ADP, AstraZeneca, Avis, Bayer, Bundesliga, Capital One, Cerner, Chick-fil-A, Convoy, Domino’s Pizza, Fidelity Investments, GE Healthcare, Georgia-Pacific, Hearst, iFood, iHeartMedia, JPMorgan Chase, Intuit, Lenovo, Lyft, National Football League, Nerdwallet, T-Mobile, Thomson Reuters, and Vanguard.
Read more on the next page about the new capabilities of AWS.
AWS provided the following information on the new features, which it says build on more than 50 new Amazon SageMaker capabilities that AWS has delivered in the past year:
- Amazon SageMaker Data Wrangler automated data preparation: Amazon SageMaker Data Wrangler provides the fastest and easiest way to prepare data for machine learning. Data preparation for machine learning is a difficult process. This difficulty arises from the fact that data attributes (known as features) used to train a machine learning model often come from different sources and exist in various formats. This means that developers must spend considerable time extracting and normalising this data so it’s consistently easy to use with machine learning. Customers might also want to combine features into composite features to give the machine learning model more helpful inputs.
- For example, a customer might want to create a feature that describes a group of customers that are prolific spenders so they can be offered loyalty program rewards by combining features for items previously purchased, the amount spent, and frequency of purchases. The work associated with transforming data into features is called feature engineering, and it consumes a lot of time for developers when they’re building machine learning models. Amazon SageMaker Data Wrangler radically simplifies the process of data preparation and feature engineering.
- With Amazon SageMaker Data Wrangler, customers can choose the data they want from their various data stores and import it with a single click. Amazon SageMaker Data Wrangler contains over 300 built-in data transformers that can help customers normalise, transform, and combine features without having to write any code, while managing all of the processing infrastructure under the hood. Customers can preview and inspect that these transformations are what was intended by viewing them in SageMaker Studio (the first end-to-end Integrated Development Environment for machine learning). Once the features have been engineered, Amazon SageMaker Data Wrangler will save them for reuse in the Amazon SageMaker Feature Store.
- Amazon SageMaker Feature Store feature storage and management: Amazon SageMaker Feature Store provides a new repository that makes it easy to store, update, retrieve, and share machine learning features for training and inference. Today, customers can save their features to Amazon Simple Storage Service (S3). This works well for a simple set of features that are mapped to a single model, but most features are not mapped to only one model. Most features are used repeatedly by multiple models and multiple developers and data scientists, and as new features are created, developers also want to be able to reuse them repeatedly.
- This leads to multiple S3 objects to manage, which can quickly become difficult to manage. Developers and data scientists try to solve this by using spreadsheets, paper notes, and emails. Sometimes they even try to build a custom application to keep track of the features, but this is a lot of work and error-prone. Further, developers and data scientists need the same features not only to train multiple models with all of the data available and where this activity can happen over hours, but also to use during inference when the predictions need to be returned in milliseconds and often use just a subset of the data in relevant features.
- For example, a developer might want to create a model that predicts the next best song in a playlist. To do this, developers would train the model on thousands of songs and then provide the model the last three songs played during inference to predict the next song. Training and inference are very different uses cases. During training, the models can access the features offline and in batch, but for inference, the model needs only a subset of the features in near real-time. Since machine learning models have a single source of features that need to be consistent, these different access patterns make it challenging to keep the features consistent and up to date.
- The Amazon SageMaker Feature Store solves this problem by providing a purpose-built feature store where developers can access and share features that make it faster to name, organise, find, and share sets of features among teams of developers and data scientists. Since the Amazon SageMaker Feature Store resides in Amazon SageMaker Studio—close to where machine learning models are run—it provides single-digit millisecond latency for inference. Amazon SageMaker Feature Store makes it simple and easy to organise and update large batches of features for training and smaller instantiations of them for inference. That way, there’s one consistent view of features for machine learning models to use and it becomes significantly easier to generate models that produce highly accurate predictions.
- Amazon SageMaker Pipelines workflow management and automation: Amazon SageMaker Pipelines is the first purpose-built, easy-to-use continuous integration and continuous delivery (CI/CD) service for machine learning. As customers can see with feature engineering, machine learning comprises multiple steps that can benefit from orchestration and automation. This is not dissimilar to traditional programming, where customers have tools like CI/CD to help them develop and deploy applications more quickly. However, with machine learning, CI/CD tools are rarely used because they don’t exist or because they are hard to set up, configure, and manage. With Amazon SageMaker Pipelines, developers can define each step of an end-to-end machine learning workflow.
- These workflows include the data-load steps, transformations from Amazon SageMaker Data Wrangler, features stored in Amazon SageMaker Feature Store, training configuration and algorithm set up, debugging steps, and optimisation steps. With Amazon SageMaker Pipelines, developers can easily re-run an end-to-end workflow from Amazon SageMaker Studio, using the same settings to get the exact same model every time, or they can re-run the workflow on a regular schedule with new data to update a model. Amazon SageMaker Pipelines logs each step in Amazon SageMaker Experiments (an Amazon SageMaker capability that organises and tracks machine learning experiments and model versions) every time a workflow is run. This helps developers visualise and compare machine learning model iterations, training parameters, and outcomes. With Amazon SageMaker Pipelines, workflows can be shared and re-used between teams, either to recreate a model or to act as a starting point for making improvements through new features, algorithms, or optimisations.
- Amazon SageMaker Clarify bias detection and explainability: Amazon SageMaker Clarify provides bias detection across the machine learning workflow, enabling developers to build greater fairness and transparency into their machine learning models. Once developers have prepared data for training and inference, they need to try to ensure the data is free from statistical bias and that model predictions are transparent, so they can explain how the model features are contributing to predictions. Today, developers sometimes try to use open source tools to detect statistical bias in their data, but these tools require a lot of manual effort and coding and are typically error-prone.
- With Amazon SageMaker Clarify, developers can now more easily detect statistical bias across the entire machine learning workflow and provide explanations for predictions their machine learning models are making. Amazon SageMaker Clarify integrates with Amazon SageMaker Data Wrangler where it runs a set of algorithms on features to identify bias during data preparation with visualisations that include a description of the sources and severity of possible bias. This way, developers can take steps for mitigation. Amazon SageMaker Clarify also integrates with Amazon SageMaker Experiments to make it easier to check trained models for statistical bias. It also details how each feature inputted into the model is affecting predictions. Finally, Amazon SageMaker Clarify integrates with Amazon SageMaker Model Monitor (an Amazon SageMaker capability that continuously monitors the quality of machine learning models in production) to alert developers if the importance of model features shifts and causes model behaviour to change.
- Deep Profiling for Amazon SageMaker Debugger model training profiler: Deep Profiling for Amazon SageMaker Debugger now enables developers to train their models faster by automatically monitoring system resource utilisation and providing alerts for training bottlenecks. Today, developers don’t have a standard way to monitor system utilisation (e.g. GPU, CPU, network throughput, and memory I/O) to identify and troubleshoot bottlenecks in their training jobs. As a result, developers can’t train models as quickly and cost-effectively as possible.
- Amazon SageMaker Debugger solves this problem with Deep Profiling’s newly announced capabilities, which provide developers with the ability to visually profile and monitor system resource utilisation in Amazon SageMaker Studio. This makes it easier to root cause issues and reduce the time and cost of training machine learning models. With these new capabilities, Amazon SageMaker Debugger expands its scope to monitor the utilisation of system resources, send out alerts on problems during training in Amazon SageMaker Studio or via AWS CloudWatch, and correlate usage to different phases in the training job or a specific point in time during training (e.g. 28 minutes after the training job started). Amazon SageMaker Debugger can also trigger actions based on alerts (e.g. stop a training job when irregularities in GPU usage are detected). Amazon SageMaker Debugger’s Deep Profiling works across frameworks (PyTorch, Apache MXNet, and TensorFlow) and collects the necessary system and training metrics automatically without requiring any code changes in training scripts. This allows developers to visualise how their system resources were used during training in Amazon SageMaker Studio.
Read more on the next page about further capabilities of SageMaker.
- Distributed Training on Amazon SageMaker accelerates training times: New Distributed Training on Amazon SageMaker makes it possible to train large, complex deep learning models up to two times faster than current approaches. Today, advanced machine learning use cases—such as natural language processing for intelligent assistants, object detection and classification for autonomous vehicles, and image classification for large-scale content moderation—demand increasingly large datasets and more graphics processing unit (GPU) memory for training.
- However, some of these models are too big to fit in the memory provided by a single GPU. Customers can attempt to split models across multiple GPUs, but finding the best way to split the model and adjusting training code can often take weeks of tedious experimentation. To overcome these challenges, Distributed Training on Amazon SageMaker offers two distributed training capabilities that enable developers to train large models up to two times faster at no additional cost. Distributed Training with Amazon SageMaker’s Data Parallelism engine scales training jobs from one GPU to hundreds or thousands by automatically splitting data across multiple GPUs, improving training time by up to 40%.
- The reduction in training time is possible because Amazon SageMaker’s Data Parallelism engine manages GPUs for optimal synchronisation using algorithms that are purposefully built to fully utilise AWS infrastructure with near-linear scaling efficiency. Distributed Training with Amazon SageMaker’s Model Parallelism engine can efficiently split large, complex models with billions of parameters across multiple GPUs by automatically profiling and identifying the best way to partition models. They do this by using graph partitioning algorithms to optimally balance computation and minimise communication between GPUs, resulting in minimal code changes and fewer errors caused by GPU memory constraints.
- Amazon SageMaker Edge Manager model management for edge devices: Amazon SageMaker Edge Manager allows developers to optimise, secure, monitor, and maintain machine learning models deployed on fleets of edge devices. Today, customers use Amazon SageMaker Neo to create optimised models for edge devices that run up to twice as fast, with less than a tenth of the memory footprint and no loss in accuracy. However, after deployment on edge devices, customers still need to manage and monitor the models to ensure they continue to perform with high accuracy. Amazon SageMaker Edge Manager optimises models to run faster on target devices and provides model management for edge devices, so customers can prepare, run, monitor, and update deployed machine learning models across fleets of devices at the edge.
- Amazon SageMaker Edge Manager gives customers the ability to cryptographically sign their models, upload prediction data from their devices to Amazon SageMaker for monitoring and analysis, and view a dashboard that tracks and visually reports on the operation of the deployed models within the Amazon SageMaker console. Amazon SageMaker Edge Manager extends capabilities that were previously only available in the cloud by sampling data from edge devices and sending it to Amazon SageMaker Model Monitor for analysis, so developers can continuously improve model quality by retraining them when their accuracy declines over time.
- Amazon SageMaker JumpStart enables the machine learning journey: Amazon SageMaker JumpStartprovides developers an easy-to-use, searchable interface to find best-in-class solutions, algorithms, and sample notebooks. Today, some customers that lack experience with machine learning have difficulty getting started with machine learning deployments, while more advanced developers find it difficult to adopt machine learning for all of their use cases. With today’s launch of Amazon SageMaker JumpStart, customers can now quickly find relevant information specific to their machine learning use cases. Developers new to machine learning will be able to select from several complete end-to-end machine learning solutions (e.g. fraud detection, customer churn prediction, or forecasting) and deploy them directly in their Amazon SageMaker Studio environments. And, experienced users will be able to choose from more than a hundred machine learning models to quickly get started on building and training models.
“Hundreds of thousands of everyday developers and data scientists have used our industry-leading machine learning service, Amazon SageMaker, to remove barriers to building, training, and deploying custom machine learning models,” said Swami Sivasubramanian, vice president of Amazon Machine Learning at AWS. “One of the best parts about having such a widely-adopted service like SageMaker is that we get lots of customer suggestions which fuel our next set of deliverables.”