Supervised Learning vs Deep Learning

Differences Between Supervised Learning vs Deep Learning

In supervised learning, the training data you feed to the algorithm includes the desired solutions, called labels. A typical supervised learning task is classification. The spam filter is a good example of this: it is trained with many example emails along with their class (spam or ham), and it must learn how to classify new emails.

Deep learning is an attempt to mimic the activity in layers of neurons in the neocortex, which is about 80% of the brain where thinking occurs(In a human brain, there are around 100 billion neurons and 100 ~ 1000 trillion synapses). It is called deep because it has more than one hidden layers of neurons which help to have multiple states of nonlinear feature transformation.

Head to Head Comparison of Supervised Learning and Deep Learning (Infographics)

Below is the Top 5 Comparison Between Supervised Learning and Deep Learning:

Key Differences between Supervised Learning and Deep Learning

Both Supervised Learning vs Deep Learning are popular choices in the market; let us discuss some of the major Differences Between Supervised Learning and Deep Learning:

• Major Models

Important supervised models are –

k-Nearest Neighbors: Used for classification and regression

Linear Regression: For Prediction/regression

Logistic Regression: For Classification

Support Vector Machines (SVMs): Used for classification and regression

Decision Trees and Random Forests: Both classification and regression tasks

Most popular Deep neural networks:

• Multilayer Perceptrons ( MLP) – Most basic type. This network is generally the starting phase of building other more sophisticated deep network and can be used for any supervised regression or classification problems
• Autoencoders (AE) – Network has unsupervised learning algorithms for feature learning, dimension reduction, and outlier detection
• Convolution Neural Network (CNN) – particularly suitable for spatial data, object recognition, and image analysis using multidimensional neuron structures. One of the main reason for the popularity of deep learning lately is due to CNN’s.
• Recurrent Neural Network (RNN) – RNNs are used for sequenced data analysis such as time-series, sentiment analysis, NLP, language translation, speech recognition, image captioning. One of the most common types of RNN model is Long Short-Term Memory (LSTM) network.
• Training Data – As mentioned earlier, supervised models need training data with labels. But Deep learning can handle data with or without labels. Some neural network architectures can be unsupervised, such as autoencoders and restricted Boltzmann machines
• Feature Selection – Some Supervised models are capable of analyzing features and a select subset of features to determine the target. But most of the time this has to be handled in the data preparation phase. But in Deep Neural Networks, new features are emerged and unwanted features are discarded as learning progress.
• Data Representation – In classical supervised models, high-level abstraction of input features are not created. Final model trying to predict output by applying mathematical transforms on a subset of input features.
  But in deep neural networks, abstractions of input features are formed internally. For example, while translating text, the neural network first converts input text to internal encoding then transform that abstracted representation to target language.
• Framework –Supervised ML models are supported by a lot of generic ML frameworks across different languages – Apache Mahout, Scikit Learn, Spark ML are some of these. Deep learning frameworks provide a developer-friendly abstraction to create a network easily, take care of distributing computation, and has support for GPUs. Caffe, Caffe2, Theano, Torch, Keras, CNTK, TensorFlow are popular frameworks. Tensorflow from Google is widely used now with active community support.

Supervised Learning and Deep Learning Comparison Table

Below is some key comparison between Supervised Learning and Deep Learning.

The Basis of Comparison between Supervised Learning vs Deep Learning	Supervised Learning	Deep Learning
Model Training	Major tasks in training – Iterate through training instances, mostly as mini-batches, and update weights applied for features. A direction of weight update(increment or decrement) determined by some indicators like a gradient of error function w.r.t weight.  Finally, a target is formulated as a transformation on a weighted sum of subset features.	Major tasks in training – Number of weights to be updated is very large if a number of hidden layers in more. Error in target value calculated first and propagated back to each layer Find the partial derivative of error function w.r.t weights and update weights to reduce error.
Potential of System	Used to solve relatively simple tasks where relation with input features and target is human detectable and feature engineering is direct. Eg : A binomial or multiclass classification like classify customers based on how they interact with a website. Predict the value of real estate property using similar data collected.	Deep learning can do really intelligent tasks like Near-human-level image classification Near-human-level speech recognition Near-human-level handwriting transcription Improved machine translation Digital assistants such as Google Now and Amazon Alexa
Flexibility	Models are more flexible which helps to fine tune ML model easily. There are well-defined methods like grid search with cross-validation to find the right hyperparameters	Less flexible as there are many hyperparameters to tweak like a number of layers, the number of neurons per layer, the type of activation function to use in each layer, the weight initialization logic, and much more.
Feature Representation	Derived or abstract features to be created explicitly. For example polynomial features as input for a Linear regression model	Abstract data representation automatically generated in hidden layers. That is why a trained CNN neural network can detect a cat in an image.
Generative Models	Not possible to generate anything original as automatic abstract data representation is not happening	Once trained, a specific type of deep neural network can generate say new images, songs, or texts. Those are called GNN(Generative neural network) or GAN(Generative Adversarial Networks) Some implementation of this type of network is used to create even new fashion designs

Conclusion

Accuracy and capability of DNN(Deep Neural Network)s has increased a lot in the last few years. That is why now DNNs are an area of active research and, we believe, it has the potential to develop a General Intelligent System. At the same time, it is difficult to reason why a DNN gives a particular output which makes fine-tuning a network really difficult. So if a problem can be solved using simple ML models it is strongly recommended to use that. Due to this fact, a simple linear regression will have relevance even if a general intelligent system is developed using DNNs.

Recommended Articles

This has been a guide to the top differences between Supervised Learning vs Deep Learning. Here we have discussed Supervised Learning vs Deep Learning head-to-head comparison, key difference along with infographics, and comparison table. You may also have a look at the following articles –

1. Supervised Learning vs Reinforcement Learning
2. Supervised Learning vs Unsupervised Learning
3. Neural Networks vs Deep Learning
4. Machine Learning vs Predictive Analytics
5. TensorFlow vs Caffe: What are the Differences
6. What is Supervised Learning?
7. What is Reinforcement Learning?
8. Top 6 Comparisons between CNN vs RNN