EDUCBA Logo

EDUCBA

MENUMENU
  • Explore
    • EDUCBA Pro
    • PRO Bundles
    • Featured Skills
    • New & Trending
    • Fresh Entries
    • Finance
    • Data Science
    • Programming and Dev
    • Excel
    • Marketing
    • HR
    • PDP
    • VFX and Design
    • Project Management
    • Exam Prep
    • All Courses
  • Blog
  • Enterprise
  • Free Courses
  • Log in
  • Sign Up
Home Data Science Data Science Tutorials Keras Tutorial Keras Initializers
 

Keras Initializers

Updated March 16, 2023

Keras Initializers

 

 

Definition of Keras Initializers

Keras initializers are the keyword arguments which is used for passing initializers for layers that depend on the layer. Usually, we can say it is a bias initializer and kernel initializer. The keras initialization will define the way for setting initial random weights onto the layers of keras. Training the neural network completely depended on the types of parameters which was used in initializing.

Watch our Demo Courses and Videos

Valuation, Hadoop, Excel, Mobile Apps, Web Development & many more.

Key Takeaways

  • Keras initializers is generating tensors by using values that were constant. The argument value is nothing but a list of values of constant values for the dtype.
  • The resulting tensor is populated by using a type of dtype values, this was specified into a value that was desired in the new tensor.

What are Keras Initializers?

Suppose we have done initialization parameters correctly then the result of the optimization is achieved in a minimal time and suppose we have not initialized, then it creates problems. The performance of the neural network depends on how the parameter is initialized at the time of starting the train. The neural network performance depends on how the parameter is initialized and starts while training the model.

Suppose we are training the neural network by using random weights, then the output will be reproducible. Suppose we train the neural network by using constant weights, then it will consume a lot of time to converge. We will be comparing the accuracy of the keras initializers, and we are looking at visualization. Snippets code is displayed while visualization is done in wandb. By adding the model which was trained by using 20 epochs and the dataset which was chosen in MNIST dataset.

Keras Initializers Constant

If the value is least, then we can say that length is less than or equal to the number of the elements which was implied in the desired shape or tensor. In some cases where a total number of values is less than or greater which is required in tensor shape for initializing or raising the value error.

There are three types of arguments we need to use while working with keras initializers constant as follows:

  • Value
  • Dtype
  • Verify_shape

The below steps show how we can use the keras initializer constant. We are importing the tensorflow module as follows.

1. In the first step we are importing the numpy and tensorflow modules by using the import keyword as follows.

Code:

import numpy as np
import tensorflow as tf

Output:

Keras Initializers - Importing

2. After importing the module now in this step, we are defining the value and declaring the constant initializer as follows.

Code:

val = [10, 21, 32, 43, 54, 65, 76, 87]
con = tf.constant_initializer (val)
print (con)

Output:

Keras Initializers - Value

3. After defining the constant initializer now in this step, we are printing values. The below example shows the standalone usage of keras constant initializer as follows.

Code:

ini = tf.keras.initializers.Constant(3.)
val = ini(shape=(2, 2))
print(val)

Output:

Keras Initializers - Printing values

4. After printing the standalone usage value, we can also print the usage of the keras layer.

Code:

init = tf.keras.initializers.Constant(3.)
lay = tf.keras.layers.Dense(3, kernel_initializer = ini)
print (lay)

Output:

Keras Initializers - Usage

Keras Initializers Layer

Keras initializer layer requires the input shape for understanding the structure of input data. To define the keras initializer layer we need to import the below modules.

Code:

from keras.models import Sequential
….
from keras import constraints

Output:

Keras Initializers - Input shape

Basically, in machine learning, the weight will be assigned to the input data. Keras initializers provide different functions. The below example generates 0 from all the input data.

Code:

tf.keras.initializers.Zeros ()
ini = tf.keras.initializers.Zeros()
val = ini(shape=(2, 2))

Output:

Keras Initializers - Input data

In the below example we are generating one value with all input values by using keras initializers layers.

Code:

tf.keras.initializers.Ones ()
ini = tf.keras.initializers.Ones()
val = ini(shape=(2, 2))

Output:

Keras Initializers - One value

In the below example, we are generating constant values with all input values which were provided by the user using keras initializers layers as follows.

Code:

tf.keras.initializers.Constant(value=0)
ini = tf.keras.initializers.Constant(3)
values = ini (shape=(2, 2))

Output:

Keras Initializers - Constant values

In the below example, we are generating values by using normal distribution in the keras initializer’s layers.

Code:

tf.keras.initializers.RandomNormal (mean=1.0, stddev=0.5, seed=None)
ini = tf.keras.initializers.RandomNormal (mean=0, stddev=1)
val = ini (shape=(2, 2))

Output:

Keras Initializers - normal distribution

In the below example, we are generating values by using uniform distribution in the keras initializer’s layers.

Code:

tf.keras.initializers.RandomUniform(minval=-0.07, maxval=0.09, seed=None)
ini = tf.keras.initializers.RandomUniform(minval=0, maxval=1)
val = ini(shape=(2, 2))

Output:

Keras Initializers 11

In the below example, we are generating values by the truncated normal distribution in the keras initializer’s layers.

Code:

tf.keras.initializers.TruncatedNormal(mean=0.0, stddev=0.07, seed=None)
ini = tf.keras.initializers.TruncatedNormal(mean=0., stddev=1.)
val = ini(shape=(2, 2))

Output:

truncated normal distribution

Keras Initializers Method

All the methods return the object or the specified initializer. Below is the method of tf.keras.initializers.constant._call_. This method returns the tensor object which was specified by initializers.

The below example shows how tf.keras.initializers.constant._call_ method is defined.

Code:

ini = tf.initializers.Constant.__call__(shape,
…
verify_shape = None)

Output:

Keras Initializers - Tensor

The tf.keras.initializers.Constant.from_config method is used to instantiate the initializer from the initialization directory.

Code:

tf.keras.initializers.Constant.from_config (cls,
config)

Output:

Keras Initializers - Instantiate

The below example shows how keras initializer method works. We are defining the random uniform method.

Code:

tf.keras.initializers.RandomUniform(minval=-0.07, maxval=0.09, seed=None)
ini = tf.keras.initializers.RandomUniform(minval=0, maxval=1)
val = ini(shape=(2, 2))

Output:

random uniform method

Examples of Keras Initializers

Given below are the examples mentioned:

Example #1

In the below example, we are defining the he_normal method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.HeNormal (seed=None)
ini = tf.keras.initializers.HeNormal()
val = ini(shape=(2, 2))

Output:

defining he_normal

Example #2

In the below example, we are using the constant method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.Constant(value=0)
ini = tf.keras.initializers.Constant(3)
values = ini (shape=(2, 2))

Output:

constant method

Example #3

In the below example, we are using the random normal method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.RandomNormal(mean=0.0, stddev=0.05, seed=None)
ini = tf.keras.initializers.RandomNormal(mean=0., stddev=1.)
val = ini(shape=(2, 2))

Output:

random normal method

Example #4

In the below example, we are using the random uniform method.

Code:

from keras.models import Sequential
from keras import initializers
from keras.layers import Activation, Dense
tf.keras.initializers.RandomUniform (minval=-0.07, maxval=0.09, seed=None)
ini = tf.keras.initializers.RandomUniform(minval=0, maxval=1)
val = ini(shape=(2, 2))

Output:

random uniform

FAQ

Given below are the FAQs mentioned:

Q1. What is the use of keras initializer?

Answer: Basically keras initializer is used to pass the initializer from the specified layer which we have defined in our code.

Q2. Which dependencies do we need to import while using the keras initializer?

Answer: We need to import the keras, numpy, and tensorflow dependency at the time of working with the keras initializer.

Q3. What is weight initialization in keras neural net?

Answer: By using keras we are initializing weight as per perception models of multi-layer. We need to initialize weight carefully at the time of working with the keras initializer.

Conclusion

Suppose we have done initialization parameters correctly then the result of the optimization is achieved in a minimal time and if we have not initialized, then it will create a problem. Keras initializers are the keyword arguments which was used for passing initializers for layers that depend on the layer.

Recommended Articles

This is a guide to Keras Initializers. Here we discuss the introduction, keras initializers constant, layer and method, and examples. You may also have a look at the following articles to learn more –

  1. Keras Tuner
  2. What is Keras GPU?
  3. Keras Embedding
  4. Keras predict

Primary Sidebar

Footer

Follow us!
  • EDUCBA FacebookEDUCBA TwitterEDUCBA LinkedINEDUCBA Instagram
  • EDUCBA YoutubeEDUCBA CourseraEDUCBA Udemy
APPS
EDUCBA Android AppEDUCBA iOS App
Blog
  • Blog
  • Free Tutorials
  • About us
  • Contact us
  • Log in
Courses
  • Enterprise Solutions
  • Free Courses
  • Explore Programs
  • All Courses
  • All in One Bundles
  • Sign up
Email
  • [email protected]

ISO 10004:2018 & ISO 9001:2015 Certified

© 2025 - EDUCBA. ALL RIGHTS RESERVED. THE CERTIFICATION NAMES ARE THE TRADEMARKS OF THEIR RESPECTIVE OWNERS.

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you
Loading . . .
Quiz
Question:

Answer:

Quiz Result
Total QuestionsCorrect AnswersWrong AnswersPercentage

Explore 1000+ varieties of Mock tests View more

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA
Free Data Science Course

Hadoop, Data Science, Statistics & others

By continuing above step, you agree to our Terms of Use and Privacy Policy.
*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you

EDUCBA Login

Forgot Password?

🚀 Limited Time Offer! - 🎁 ENROLL NOW