SageMaker Autopilot
Amazon SageMaker Autopilot is a feature-set that simplifies and accelerates various stages of the machine learning workflow by automating the process of building and deploying machine learning models (AutoML).
Autopilot performs the following key tasks that you can use on autopilot (hence the name) or with various degrees of human guidance:
-
Data analysis and preprocessing: Autopilot identifies your specific problem type, handles missing values, normalizes your data, selects features, and overall prepares the data for model training.
-
Model selection: Autopilot explores a variety of algorithms and uses a cross-validation resampling technique to generate metrics that evaluate the predictive quality of the algorithms based on predefined objective metrics.
-
Hyperparameter Optimization: Autopilot automates the search for optimal hyperparameter configurations.
-
Model Training and Evaluation: Autopilot automates the process of training and evaluating various model candidates. It splits the data into training and validation sets, trains the selected model candidates using the training data, and evaluates their performance on the unseen data of the validation set. Lastly, it ranks the optimized model candidates based on their performance and identifies the best performing model.
-
Model Deployment: Once Autopilot has identified the best performing model, it provides the option to deploy the model automatically by generating the model artifacts and the endpoint exposing an API (Application Programming Interface). External applications can send data to the endpoint and receive the corresponding predictions or inferences.
Autopilot supports building machine learning models on large datasets up to hundreds of GBs.
The following outlines the tasks of this AutoML process managed by Autopilot.
Depending on your comfort level with the machine learning process and coding experience, you can use Autopilot in different ways:
-
Fully automated mode without any additional human input with Amazon SageMaker Studio UI. This approach requires minimal coding or technical knowledge. Only experiments created from tabular data for problem types such as regression or classification are available via the Studio UI.
-
Human input with specific option and parameter selection in Amazon SageMaker Studio UI. Users can better customize and control the creation of their machine learning experiment by selecting specific options and parameter values.
-
Code implementation using SDKs. If you have coding experience, you can use available software development kits (SDKs) to create an Autopilot experiment into your own code. This approach provides greater flexibility and customization options.
Autopilot currently supports the following problem types:
-
Regression, binary, and multiclass classification with tabular data formatted as CSV or Parquet files in which each column contains a feature with a specific data type and each row contains an observation. The column data types accepted include numerical, categorical, text, and time series that consists of strings of comma-separated numbers.
-
Text classification with data formatted as CSV or Parquet files in which a column provides the sentences to be classified, while another column should provide the corresponding class label.
-
Image classification with images formats such as PNG, JPEG or a combination of both.
Note
For regression or classification problems involving tabular data, users can choose between two options: using the Studio user interface or the API Reference.
Tasks like text and image classification are exclusively available through the SageMaker API. However, users who prefer the convenience of a user interface can use Amazon SageMaker Canvas to access pre-trained models or create custom models tailored to specific image or text classification needs.
Additionally, Autopilot helps users understand how models make predictions by automatically generating reports that show the importance of each individual feature. This provides transparency and insights into the factors influencing the predictions, which can be used by risk and compliance teams and external regulators. Autopilot also provides a model performance report, which encompasses a summary of evaluation metrics, a confusion matrix, various visualizations such as receiver operating characteristic curves and precision-recall curves, and more. The specific content of each report vary depending on the problem type of the Autopilot experiment.
The explainability and performance reports of Autopilot are available for all problem types for the best model candidate.
For tabular data use cases such as regression or classification, Autopilot offers additional visibility into how the data was wrangled and how the model candidates were selected, trained, and tuned, by generating notebooks that contain the code used to explore the data and find the best performing model. These notebooks provide an interactive and exploratory environment to help you learn about the impact of various inputs or the trade-offs made in the experiments. You can experiment further with the higher performing model candidate by making your own modifications to the data exploration and candidate definition notebooks provided by Autopilot.
With Amazon SageMaker, you pay only for what you use. You pay for the underlying compute and storage
resources within SageMaker or other AWS services, based on your usage. For more information about
the cost of using SageMaker, see Amazon SageMaker
Pricing
Topics
