EDUCBA

EDUCBA

MENUMENU
  • Blog
  • Free Courses
  • All Courses
  • All in One Bundle
  • Login
Home Data Science Data Science Tutorials Keras Tutorial Keras Regularization

Keras Regularization

Keras Regularization

Introduction to Keras Regularization

Keras regularization allows us to apply the penalties in the parameters of layer activities at the optimization time. Those penalties were summed into the function of loss, and it will optimize the network. It applies on a per-layer basis. The exact API depends on the layer, but multiple layers contain a unified API. The layer will expose arguments of 3 keywords.

Key Takeaways

  • Suppose we need to configure the regularization using multiple arguments, then implement the subclass into the keras regularization.
  • We can also implement the class method and get the config to support the serialization. We can also use regularization parameters.

What is Keras Regularization?

The keras regularization prevents the over-fitting penalizing model from containing large weights. There are two popular parameters available, i.e., L1 and L2. L1 is nothing but the Lasso, and L2 is called Ridge. Both of these parameters are defined at the time of learning the linear regression. When working with tensorflow, we can implement the regularization using an optimizer. We are adding regularization to our code by adding a parameter name as kernel_regularizer. While adding L2 regularization, we need to pass the keras regularizers.l2 () function.

Start Your Free Data Science Course

Hadoop, Data Science, Statistics & others

This function takes one parameter, which contains the strength of regularization. We pass L1 regularizers by replacing the l2 function with the l1 function. Suppose we need to use L2 and l1 regularization this is called the elastic net. The weight regularization provides an approach to reducing the overfitting of neural network models for deep learning. Activity regularization encourages the neural network to learn the sparse features of internal representations for the raw observations. It is common to seek the representation of spark known for autoencoders called sparse encoders.

How to Add Keras Regularization?

It will generally reduce the model overfitting and help the model generalize. The regularization is a penalized model for overfitting, as we know it has two parameters. Below we are using the l1 parameter for adding keras regularization.

Below steps shows how we can add keras regularization as follows:

1. In the first step we are installing the keras and tensorflow module in our system. We are installing those modules by using the import keyword as follows.

Code:

python -m pip install tensorflow
python –m pip install keras

Output:

Keras Regularization - Keyword

Keras Regularization 2

2. After installing the module of keras and tensorflow now we are checking the installation by importing both modules as follows.

Code:

import tensorflow as tf
from keras.layers import Dense

Output:

Keras Regularization - modules

3. After checking the installation now in this step we are importing the required model which was used in it. Basically, we are importing the dense, sequential, l1, and activation modules. We are importing the dense module from the layers library, a sequential module from the library, an l1 module from the regularizers library, and an activation module from the layers library.

Code:

from sklearn.datasets import make_circles
…..
from keras.layers import Activation

Output:

Keras Regularization 4

4. After importing the dataset now in this step we are preparing the dataset for it. We are preparing the dataset by using x and y values. Also, we are defining the value of X_train, y_train, X_test, and y_test as follows.

Code:

X, y = make_circles()
train = 25
X_train, X_test = X[]
y_train, y_test = y[]

Output:

Keras Regularization 5

5. After creating the dataset in this step we are creating the neural network model and adding the regularizer into the input layer as follows. We are adding a sequential model and defining the dense layer as follows.

Code:

mod = Sequential()
mod.add()
mod.add(Activation('relu'))
mod.add(Dense(2, activation = 'relu'))
mod.compile()
mod.summary()

Output:

Keras Regularization - neural network

Keras Regularization Layer

The weight regularization layer of keras is applying penalties to the parameters of layers. The weight regularization layer will expose three keyword arguments as follows:

  • Kernel Regularizer
  • Bias Regularizer
  • Activity Regularizer

The below example shows keras weight regularization layer as follows. This layer is dividing the input batch size.

Code:

from tensorflow.keras import layers
from tensorflow.keras import regularizers
we_lay = layers.Dense(
units = 44,
kernel_regularizer = regularizers.L1L2(),
…
activity_regularizer = regularizers.L2 (1e-5)
)
ten = tf.ones (shape = (7, 7)) * 3.0
out = we_lay(ten)
print(tf.math.reduce_sum (we_lay.losses))

Output:

Input batch size

The L1 and L2 regularizers are available as part of a module of regularizers. The below example shows the L1 class regularizers module.

Code:

from tensorflow.keras import layers
from tensorflow.keras import regularizers
we_lay = layers.Dense (
units = 44,
kernel_regularizer = regularizers.L1L2(),
…
activity_regularizer = regularizers.L2(1e-5)
)
ten = tf.keras.regularizers.L1(l1=0.01 * 3.0)
print (tf.math.reduce_sum (we_lay.losses))

Output:

Keras Regularization module

The below example shows the L1 class regularizers module as follows. We are importing the layers and regularizers model.

Code:

from tensorflow.keras import layers
from tensorflow.keras import regularizers
we_lay = layers.Dense(
units = 44,
kernel_regularizer = regularizers.L1L2(),
…
activity_regularizer = regularizers.L2 (1e-5)
)
ten = tf.keras.regularizers.L2 (l2 = 0.01 * 3.0)
print(tf.math.reduce_sum(we_lay.losses))

Output:

importing the layers

Examples of Keras Regularization

Given below are the examples mentioned:

Example #1

In the below example we are using L2 arguments.

Code:

from sklearn.datasets import make_circles
…..
from keras.layers import Activation
X, y = make_circles()
train = 25
X_train, X_test = X []
y_train, y_test = y []
mod = Sequential()
mod.add()
mod.add(Activation ('relu'))
mod.add(Dense(2, activation = 'relu'))
mod.compile()
mod.summary()

Output:

Using L2 arguments

Example #2

In the below example, we are using L1 arguments.

Code:

from sklearn.datasets import make_circles
…..
from keras.layers import Activation
X, y = make_circles()
train = 35
X_train, X_test = X[]
y_train, y_test = y[]
mod = Sequential()
mod.add()
mod.add(Activation('relu'))
mod.add(Dense(2, activation = 'relu'))
mod.compile()
mod.summary()

Output:

L1 arguments

FAQ

Given below are the FAQs mentioned:

Q1. What is the use of keras regularization?

Answer: It is the technique for preventing the model from large weights. The regularization category is applied to the per-layer basis.

Q2. How many types of weight regularization are in keras?

Answer: Basically there are multiple types of weight regularization like vector norms, L1 and L2. It will require the hyper parameter which is configured.

Q3. Which modules do we need to import at the time of using keras regularization?

Answer: We need to import the keras and tensorflow module at the time of using it. Also, we need to import is a dense layer.

Conclusion

There are two popular keras regularization parameters available i.e. L1 and L2. In that L1 is nothing but the Lasso and L2 is called Ridge. It allows us to apply the penalties to the parameters of layer activities at the time of optimization.

Recommended Articles

This is a guide to Keras Regularization. Here we discuss the introduction, and how to add keras regularization, layer, examples, and FAQ. You may also have a look at the following articles to learn more –

  1. Keras Tuner
  2. What is Keras GPU?
  3. Keras Embedding
  4. Keras predict
EVIEWS Training
32+ Hours of HD Videos
11 Courses
Verifiable Certificate of Completion
Lifetime Access
4.5
MYSQL Certification Course
115+ Hours of HD Videos
18 Courses
3 Mock Tests & Quizzes
Verifiable Certificate of Completion
Lifetime Access
4.5
CLOUD COMPUTING Certification Course
141+ Hours of HD Videos
23 Courses
Verifiable Certificate of Completion
Lifetime Access
4.5
SPLUNK Training Program
70+ Hours of HD Videos
11 Courses
Verifiable Certificate of Completion
Lifetime Access
4.5
Primary Sidebar
Popular Course in this category
KERAS Training
 40+ Hour of HD Videos
10 Courses
Verifiable Certificate of Completion
  Lifetime Access
4.5
Price

View Course
Footer
About Us
  • Blog
  • Who is EDUCBA?
  • Sign Up
  • Live Classes
  • Corporate Training
  • Certificate from Top Institutions
  • Contact Us
  • Verifiable Certificate
  • Reviews
  • Terms and Conditions
  • Privacy Policy
  •  
Apps
  • iPhone & iPad
  • Android
Resources
  • Free Courses
  • Database Management
  • Machine Learning
  • All Tutorials
Certification Courses
  • All Courses
  • Data Science Course - All in One Bundle
  • Machine Learning Course
  • Hadoop Certification Training
  • Cloud Computing Training Course
  • R Programming Course
  • AWS Training Course
  • SAS Training Course

ISO 10004:2018 & ISO 9001:2015 Certified

© 2023 - EDUCBA. ALL RIGHTS RESERVED. THE CERTIFICATION NAMES ARE THE TRADEMARKS OF THEIR RESPECTIVE OWNERS.

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you

EDUCBA
Free Data Science Course

Hadoop, Data Science, Statistics & others

By continuing above step, you agree to our Terms of Use and Privacy Policy.
*Please provide your correct email id. Login details for this Free course will be emailed to you
Let’s Get Started

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA Login

Forgot Password?

By signing up, you agree to our Terms of Use and Privacy Policy.

This website or its third-party tools use cookies, which are necessary to its functioning and required to achieve the purposes illustrated in the cookie policy. By closing this banner, scrolling this page, clicking a link or continuing to browse otherwise, you agree to our Privacy Policy

Loading . . .
Quiz
Question:

Answer:

Quiz Result
Total QuestionsCorrect AnswersWrong AnswersPercentage

Explore 1000+ varieties of Mock tests View more