EDUCBA

EDUCBA

MENUMENU
  • Free Tutorials
  • Free Courses
  • Certification Courses
  • 600+ Courses All in One Bundle
  • Login
Home Software Development Software Development Tutorials Spring Tutorial Spring Batch Partitioner
Secondary Sidebar
Spring Tutorial
  • Spring
    • What is Spring Framework?
    • Spring Architecture
    • What is Spring Integration?
    • IoC Containers
    • What is AOP?
    • Spring Modules
    • Spring Batch Processing
    • Spring Batch Partitioner
    • Spring Batch Job
    • Spring AOP
    • Spring Expression Language
    • Dependency Injection in Spring
    • Spring Batch Architecture
    • Spring framework Interview Questions
  • Spring Boot
    • What is Spring Boot
    • Spring Boot flyway
    • Spring Boot framework
    • Spring Boot Logback
    • Spring Boot actuator endpoints
    • Spring Boot gRPC
    • Spring Boot jdbctemplate example
    • Spring Boot ehcache
    • Spring Boot Architecture
    • Spring Boot Port
    • Introduction of spring boot
    • Spring Boot ide
    • Spring Boot Netty
    • Spring Boot ORM
    • Spring Boot Versions
    • Spring Boot JUnit
    • Spring Boot Keycloak
    • Spring Boot gradle
    • Spring Boot Lombok
    • Spring Boot autowired
    • Spring Boot bean
    • Spring Boot hibernate
    • Spring Boot integration test
    • Spring Boot jdbc
    • Spring Boot MongoDB
    • Spring Boot postgresql
    • Spring Boot rest
    • Spring Boot swagger
    • Spring Boot thymeleaf
    • Spring Boot Unit Test
    • Spring Boot Webflux
    • Spring Boot webclient
    • Spring Boot kubernetes
    • Spring Boot Properties
    • Spring Boot Validation
    • Spring Boot Feature
    • Spring Boot Application
    • Spring Boot email
    • Spring Boot MVC
    • Spring Boot Exception Handling
    • Spring Boot Starter Parent
    • Spring Boot Docker
    • Spring Boot Logging
    • Spring Boot Query
    • Spring Boot Multiple Data Sources
    • Spring Boot Basic Authentication
    • Spring Boot Test
    • Spring Boot jwt
    • Spring Boot Liquibase
    • Spring Boot Prometheus
    • Spring Boot debug
    • Spring Boot GraalVM
    • Spring Boot Batch
    • Spring Boot controller
    • Spring Boot CLI
    • Spring Boot file upload
    • Spring Boot interceptor
    • Spring Boot Service
    • Spring Boot Configuration
    • Spring Boot Datasource Configuration
    • Spring Boot Annotations
    • Spring Boot Starter We
    • Spring Boot Actuator
    • Spring Boot DevTools
    • Spring Boot Repository
    • Spring Boot Dependencies
    • Spring Boot Path Variable
    • Spring Boot Microservices
    • Spring Boot Run Command
    • Spring Boot application.properties
    • Spring Boot Transaction Management
    • Spring Boot Banner
    • Spring Boot JPA
    • Spring Boot Change Port
    • Spring Boot RestTemplate
    • Spring Boot cors
    • Spring Boot HTTPS
    • Spring Boot OAuth2
    • Spring Boot Profiles
    • Spring Boot Interview Questions
    • Spring Boot filter
    • Spring boot logging level
    • Spring Boot Cache
    • Spring Boot Advantages
    • Spring Boot Scheduler
    • Spring Boot Initializr
    • Spring Boot Maven
    • Spring Boot Admin
    • Spring Boot Tomcat
    • Spring Boot WebSocket
    • Spring Boot Executable Jar
    • Spring Boot CommandLineRunner
    • Spring Boot DataSource
    • Spring Batch Scheduler
    • Spring Batch Example
    • Spring Batch Tasklet
    • Spring Batch Admin
    • Spring Batch
    • Spring Boot Qualifier
    • Spring Boot War
    • Spring Boot Test Configuration
  • Spring Cloud Basics
    • What is Spring Cloud
    • Spring Cloud Contract
    • Spring Cloud Components
    • Spring Cloud Version
    • Spring Cloud Data Flow
    • Spring cloud stream
    • Spring Cloud Dependencies
    • Spring cloud microservices
    • spring cloud gateway
    • Spring Cloud Config
    • Spring Cloud Kubernetes
    • Spring Cloud Sleuth

Related Courses

Spring Boot Certification Course

Spring Framework Course Training

All in One Data Science Course

Spring Batch Partitioner

Spring Batch Partitioner

Introduction to Spring Batch Partitioner

Spring batch partitioner processes a variety of data sets, partitioner uses several threads. A programmatically defined range of data sets can be specified. It depends on the use case and how many threads we want to establish for the partitioner. The amount of threads used is solely determined by the need. We must partition the steps of the batch task in order to make parallel processing possible.

What is a spring batch partitioner?

  • One of the open-source batch processing frameworks is Spring Batch. A single thread is used to execute spring batch scripts.
  • However, there may be instances where the utilization of a single thread consumes a significant amount of time during execution.
  • This can be accomplished by using partitioned step execution, in which each job handles a small chunk of work and all jobs run at the same time.
  • We’ll be delivering data from numerous csv files to the database at the same time. This would result in a decrease in elapsed time and an increase in efficiency.
  • A variety of options are available to implement a job with some parallel processing. There are two types of parallel processing at a higher level:
  • Multi-Process
  • Multi-threaded, single-process
  • When there are millions of data to read from source systems, and we can’t rely on a single thread to process them all, which might be time-consuming, a partitioner comes in handy.
  • We’d like to read and process data in many threads to make the best use of the system’s resources.
  • By employing multiple thread execution, the spring boot batch partitioner will assist enhance the pace of the spring boot batch.
  • The spring boot batch partitioner will divide and run batch processes using several threads in a multi-threaded environment.
  • The partitioner will speed up batch processing. The partitioner will handle each csv file in a different thread.
  • Because batch jobs run in the background and do not require human engagement, they are a little more difficult to scale.
  • As a result, monitoring the time it takes for a user to respond to a request isn’t a useful performance indicator.
  • The time it takes to process a batch job is constrained. Batch applications are typically performed at night and have a set amount of time to finish. Scaling a batch task aims to fulfill the required execution time.

Spring batch partitioner step

  • The partitioner is the central strategy interface for producing ExecutionContext instances as input parameters for a partitioned phase.
  • We’re querying the dataset for the MAX and MIN id values and building partitions amongst all records based on that.
  • We used gridSize = number of threads for the partitioner. We can create our own custom value-based as per our needs.
  • Below are the steps which were used at the time of developing any partitioner application.
  • Create a project template by using a spring initializer.
  • After creating the project, open the project template by using the spring tool suite.
  • Add the spring batch partitioner dependency.
  • Then create a partitioner and define the range of records.
  • After creating the partitioner, create batch and processor.
  • Load the data and run the application.

Spring batch partitioner example

The below example shows the spring batch partitioner are as follows.

  • Create project template of spring batch partitioner by using spring initializer

In the below step, we have providing project group name as com.example, artifact name as SpringBatchPartitioner, project name as SpringBatchPartitioner, and selected java version as 8. Also, we have defined the spring boot version as 2.6.0, defined the project as maven.

All in One Software Development Bundle(600+ Courses, 50+ projects)
Python TutorialC SharpJavaJavaScript
C Plus PlusSoftware TestingSQLKali Linux
Price
View Courses
600+ Online Courses | 50+ projects | 3000+ Hours | Verifiable Certificates | Lifetime Access
4.6 (86,754 ratings)

We have selected spring web, spring batch, and PostgreSQL driver dependency in the below project to implement the spring batch partitioner project.

Start Your Free Software Development Course

Web development, programming languages, Software testing & others

Group – com.example            Artifact name – SpringBatchPartitioner
Name – SpringBatchPartitioner          Spring boot – 2.6.0
Project – Maven                                  Java – 8
Package name - com.example.SpringBatchPartitioner
Project Description - Project for SpringBatchPartitioner
Dependencies – spring web, PostgreSQL driver, spring batch.

Spring Batch Partitioner output 1

  • After generating project extract files and open this project by using spring tool suite –
  • After generating the project using the spring initializer in this step, we extract the jar file and open the project using the spring tool suite.

Spring Batch Partitioner output 2

  • After opening the project using the spring tool suite, check the project and its files –

In this step, we are checking all the project template files. We also need to check maven dependencies and system libraries.

Spring Batch Partitioner output 3

  • Add dependency packages –

In this step, we are adding the partitioner dependency in the spring batch processing project.

Code –

<dependency>   -- Start of dependency tag.
<groupId>org.springframework.batch</groupId>   -- Start and end of groupId tag.
<artifactId>spring-boot-starter-batch</artifactId>  -- Start and end of artifactId tag.
</dependency>    -- End of dependency tag.
<dependency>   -- Start of dependency tag.
<groupId>org.postgresql</groupId>   -- Start and end of groupId tag.
<artifactId>postgresql</artifactId>  -- Start and end of artifactId tag.
</dependency>    -- End of dependency tag.

Spring Batch Partitioner output 4

  • Create database, tables and add records in stud table –

Code –

create database springbatchpartitioner;
create table stud(user_id int);

Spring Batch Partitioner output 5

  • Configure application.properties file –

Code –

spring.datasource.url=jdbc:postgresql://localhost/springbatchpartitioner
spring.datasource.driverClassName=com.postgresql.jdbc.Driver
spring.datasource.username=postgres
spring.datasource.password=postgres
spring.batch.job.enabled=false

output 6

  • Create partitioner –

Code –

public class springbatchpartition implements Partitioner
{
private JdbcOperations JT;
private String t;
private String c;
public void setTable (String tab) {
this.t = tab;
}
public void setColumn (String col) {
this.c = col;
}
}

output 7

  • Configure spring batch job –

Code –

@Configuration
public class configure {
@Autowired
private JobBuilderFactory JBF;
@Autowired
private StepBuilderFactory SBF;
@Autowired
private DataSource DS;
@Bean
public CRP partitioner()
{
CRP CRP = new CRP();
CRP.setCol ("id");
CRP.setCol (DS);
CRP.setTab ("stud");
return CRP;
}
}

output 8

  • Create entity and mapper class –

Code –

@Data
@AllArgsConstructor
@Builder
@NoArgsConstructor
public class Stud
{
private int user_id;
}

output 9

  • Create the main class –

Code –

@SpringBootApplication
@EnableBatchProcessing
public class springpartitioner implements CommandLineRunner
{
@Autowired
private JobLauncher JL;
@Autowired
private Job J;
public static void main(String[] args) {
SpringApplication.run (springpartitioner.class, args);
}

output 10

  • Run the application –

output 11

Conclusion

The partitioner is the central strategy interface for producing ExecutionContext instances as input parameters for a partitioned phase. Partitioner processes a variety of data sets, partitioner uses several threads. We must partition the steps of the batch task in order to make parallel processing possible.

Recommended Articles

This is a guide to Spring Batch Partitioner. Here we discuss What are a spring batch partitioner and the Steps of partitioner along with the examples and codes. You may also have a look at the following articles to learn more –

  1. Spring Boot Docker
  2. Spring Boot Starter Parent
  3. Spring Batch Architecture
  4. Spring Boot HTTPS
Popular Course in this category
Spring Boot Training Program (2 Courses, 3 Project)
  2 Online Courses |  3 Hands-on Projects |  22+ Hours |  Verifiable Certificate of Completion
4.5
Price

View Course

Related Courses

Spring Framework Training (4 Courses, 6 Projects)4.9
All in One Data Science Bundle (360+ Courses, 50+ projects)4.8
0 Shares
Share
Tweet
Share
Primary Sidebar
Footer
About Us
  • Blog
  • Who is EDUCBA?
  • Sign Up
  • Live Classes
  • Corporate Training
  • Certificate from Top Institutions
  • Contact Us
  • Verifiable Certificate
  • Reviews
  • Terms and Conditions
  • Privacy Policy
  •  
Apps
  • iPhone & iPad
  • Android
Resources
  • Free Courses
  • Java Tutorials
  • Python Tutorials
  • All Tutorials
Certification Courses
  • All Courses
  • Software Development Course - All in One Bundle
  • Become a Python Developer
  • Java Course
  • Become a Selenium Automation Tester
  • Become an IoT Developer
  • ASP.NET Course
  • VB.NET Course
  • PHP Course

ISO 10004:2018 & ISO 9001:2015 Certified

© 2022 - EDUCBA. ALL RIGHTS RESERVED. THE CERTIFICATION NAMES ARE THE TRADEMARKS OF THEIR RESPECTIVE OWNERS.

EDUCBA
Free Software Development Course

C# Programming, Conditional Constructs, Loops, Arrays, OOPS Concept

*Please provide your correct email id. Login details for this Free course will be emailed to you

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA Login

Forgot Password?

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA
Free Software Development Course

Web development, programming languages, Software testing & others

*Please provide your correct email id. Login details for this Free course will be emailed to you

By signing up, you agree to our Terms of Use and Privacy Policy.

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you

By signing up, you agree to our Terms of Use and Privacy Policy.

Let’s Get Started

By signing up, you agree to our Terms of Use and Privacy Policy.

This website or its third-party tools use cookies, which are necessary to its functioning and required to achieve the purposes illustrated in the cookie policy. By closing this banner, scrolling this page, clicking a link or continuing to browse otherwise, you agree to our Privacy Policy

Loading . . .
Quiz
Question:

Answer:

Quiz Result
Total QuestionsCorrect AnswersWrong AnswersPercentage

Explore 1000+ varieties of Mock tests View more