Introduction to AWS Deep Learning
The digitization of data has paved the way for large volumes of data being processed in areas ranging from financial to the medical domain. Thus, there is a paradigm shift from traditional to cloud-based platforms. AWS is one such platform that is seamlessly integrated with Deep Learning and Machine Learning APIs.
AWS Deep Learning Framework
The framework of AWS deep learning is explained below:
AWS provides AMIs (Amazon Machine Images), which is a virtual instance with a storage cloud. It can be used to launch Amazon EC2 instances which can be used to train complex deep learning models or to experiment with deep learning algorithms. It is also compatible with the Linux Operating System and NVIDIA based graphic accelerator libraries like CUDA and CuDNN. It also supports popular deep learning frameworks like MXNet and Gluon, Caffe, Caffe2, Keras, Microsoft Cognitive Toolkit, PyTorch, TensorFlow, Theano, etc. that come as preinstalled packages in the AMI instance. There are several preconfigured AMIs available or a custom AMI can be created by the user. The list of popularly available AMIs used in Deep Learning is as follows:
It helps the user to switch between different deep learning contextual environments. It also comes preconfigures with popular pip packages used by deep learning frameworks. It is fast and can install, run and update the packages and their dependencies on the fly.
It provides the infrastructure to set up a private deep learning repository or a custom deep learning engine. It supports GPU drivers and acceleration libraries to deploy deep learning applications. It comes with NVIDIA CUDA 9 as a default configuration which can be switched to other CUDA environments.
AMI with Source Code
It comes pre-configured with deep learning frameworks along with their source codes. It requires a shared Python instance to run and supports NVIDIA CUDA environments.
- AWS also provides containers for deep learning. It comes preconfigured with Docker images called AWS deep learning containers. It helps in reducing model training time and thus increases the performance. Also, it allows the user to quickly deploy a deep learning environment without building and optimizing it from scratch. It can train the configured machine learning environments on a single node or multi-node clusters on CPU/GPUs. It also supports distributed training frameworks like Horovod.
- Amazon SageMaker is one such deep learning API provided by AWS to build and deploy custom Machine Learning Models. It uses Amazon S3 as a storage service that can be used to store and democratize data for deep learning purposes. It supports both the batch and pipe mode for reading data from the above storage service. It comes with built-in machine learning algorithms like support vector machines, K-means, etc. It also supports automatic hyperparameter tuning for different scalable experiments.
- Amazon SageMaker Ground Truth helps to create complex datasets that can be used for training a deep learning model at reduced costs. The label data is automatically generated using machine learning. Thus, the label generation continuously improves based on initial human labeling.
- Amazon SageMaker Neo allows the user to train the deep learning model once. It optimizes the model to run fast and with no loss in accuracy. It compiles the training model into an executable that can be run on a neural network that applies performance optimizations for the target hardware platform.
- Amazon Augmented AI(A2I) can implement human view workflows with a single click and is seamlessly integrated with other AI services provided by AWS to perform the above. It also provides built-in workflows for text extraction and image moderation uses.
- Amazon Textract is another AWS service that is used in preprocessing data for deep learning models. It extracts text and data from scanned documents. It is a fast and efficient process that requires no templates to be maintained and is highly accurate.
- Amazon Rekognition is another AWS service to add image and video analytics for deep learning models. It is highly scalable and requires no machine learning expertise for its usage.
- AWS Lambda is another service that allows the running of deep learning models without provisioning or managing servers. It can be scaled and the model can be re-run from other provided services. Also, it can be called directly from any web or mobile application.
- The above AWS services are integrated in what is called a pipeline which is used to build, train and deploy a model. The pipeline creation workflow is created using a devops framework called Kubernetes. This helps in building scalable workflows on AWS Docker-based containers.
Applications of AWS Deep Learning Framework
Below are the different applications of AWS deep learning framework:
1. Natural Language Processing
AWS provides a service called Amazon Comprehend which is used to find insights and relationships between texts. Its usage requires no expertise in machine learning. It can perform the below tasks:
- Sentimental Analysis
- Part of Speech tagging
- Document Summarization
2. Computer Vision
AWS provides a service called Amazon GluonCV which is used to find detect and classify objects in images and videos. Its usage requires no expertise in machine learning. It provides a preconfigured state of art algorithms to the user. It can perform the below tasks:
- Face Detection and Recognition
- Image Captions
- Object detection
- Generative Autoencoder Networks
- Video Detection
3. Feature Engineering
AWS SageMaker Autopilot is used to generate new features for the deep learning models based on the training data. It uses different combinations of features during the model training and validation to get the best results on the evaluation criterion determined by the user for his/her application.
4. Reinforcement Learning
AWS SageMaker RL is used for role-playing based gaming and finds its application in self-driving cars, policy formulation using Markov Decision Processes. It uses State-Action and Reward mechanism as part of model creation. It is currently supporting the TensorFlow and Apache MXNet environment.
It is continuously evolving and adding new APIs to its ecosystem. It is one of the most popular cloud-based platforms. However, it faces stiff competition in the market from tech giants like Microsoft, Google, etc. which are also providing Deep Learning APIs with their cloud.
This is a guide to AWS Deep Learning. Here we discuss the Introduction, Framework, and applications of AWS deep learning in detail. You may also look at the following articles to learn more –