Updated March 22, 2023
Introduction to Mainframe Testing
Mainframe server handles a high volume of transactions, manages simultaneous access of thousands of users and large database/heavy bandwidth connections with ease. The servers are rugged in design and known for Reliability, Availability, Serviceability and Security factors. Traditionally, enterprise applications (ERP) for big businesses are hosted in Mainframe computers. ERP typically includes business-critical applications like Finance, Inventory Management, Payroll, Order Processing, HR, etc., Modern digital applications use the Mainframe for its backend operations. Production databases for most mission-critical applications like ATM, eCommerce sites are hosted in Mainframe.
Due to its criticality, Mainframe Applications should be tested thoroughly before the production run.
Let’s look into various aspects of testing of these applications.
The Methodology of Mainframe Testing
Applications in Mainframe are accessed by end-users in a way different from Web applications. The key differences, application testers should be aware of are
| Sl | Mainframe | Web |
| 1 | Accessed thru Terminal Emulator | Accessed thru Browser or UI |
| 2 | Terminal Emulator is the only software to be loaded in a client machine for testing.
Applications reside in the main server. |
Some of the processing is done by Client-side and the apps should be deployed at the right place before testing starts. |
| 3 | User logs into the system directly | Applications are accessed thru two-tier architecture (Client/Server) or three-tier (Presentation/apps/DB storage layers) |
| 4 | The tester should be aware of specific mainframe operations. | Testing can be done directly on the application screen. |
Prerequisite knowledge on Mainframe operations a tester should have
- Time-sharing option (TSO) – Way to access virtual storage and manage datasets through commands.
- Interactive system productivity facility (ISPF) for menu operations
- System display and search facility (SDSF) to control operations and system resources.
- CICS transaction to test for IBM mainframes.
- FTP or another transfer protocol
- Batch job management
- Cross-platform operations where Mainframe is the backend
Steps to follow in Mainframe Testing
The following steps to be followed are given below:
1. Smoke Testing
A hygiene test to confirm the readiness of the software build for a full-fledged test. It covers superficial test scenarios and consumes little effort and time. It saves the effort of testers’ time in testing a defective build.
2. Functionality Testing
Functionalities of various models are tested individually as well with respect to each other, for its conformance to requirements.
3. Batch Testing
Batch jobs are a set of activities that get executed without any user interaction, with the available compute resources. Batch testing validates the results of the output of the batch job with the specification.
4. Online Testing
Front end features of the mainframe applications are tested in this category. It covers navigations within the screen, data input validations, user-friendliness and look & feel aspects.
5. Online-Batch Integration Testing
Integration aspects of the online process with the back end process are tested here. It checks the correctness of the data flow and the interactions, between the screens and the back end system.
6. Database Testing
This test validates the Data stored by the transactions for the conformance with the system specification. Data integrity and other database parameters for optimal performance are also validated here.
7. System Integration Testing
This test is taken up post completion of unit-level tests. Here end to end functionalities of all the modules within the current system (Intra application) as well as interactions with external applications (inter-application) is tested. Here data flow across the modules and applications are tested for correctness. This testing ensures the readiness of the build for deployment.
8. Regression Testing
This testing ensures changes done in a module does not affect the overall function of the parent application as well as the integrated application. This is a negative test to control the impact of any defective code changes in the application in a post-production scenario.
9. Performance Testing
Mainframe applications may have the following performance glitches
- More resources are consumed in the program resulting in a high cost of operation
- Online response time is slow leading to customer dissatisfaction
- Batch jobs and backend process takes more time, cuts into system availability to online users d. Scalability issues
To overcome the above glitches application should be tested for proper
- System and database parameters
- Application and Database design
- System integration parameters
- Coding
- Timing of back end jobs
10. Security Testing
It involves assessing the threats, risks, vulnerabilities and recommend remedial actions for applications as well as network.
Testing should cover use cases in areas such as Identity and access management, Data protection & privacy policy adherence, Risk & Compliance management.
11. Agile Methodologies
Testing methods discussed above follow a sequence of steps with the assumption that requirements remain constant throughout the project. But modern digital applications undergo changes frequently and evolve continuously. Agile methodology facilitates the incremental development of software and responds to changes quickly. Incremental Testing is used in an agile development scenario.
How to do Mainframe Testing?
Mainframe testing normally is done manually and it can be automated using tools like REXX, QTP, IBM application performance analyzer, CA application fine tuner, etc.. Irrespective of mode, the testing activities should be managed by the following shift left approach.
1. Planning
A testing team in co-ordination with the Project management and development team should prepare Test scenarios and test cases well in advance. System requirement document, Business requirement document, other project documents and inputs from the development team will be useful for preparing test plans. The resources with prerequisite mainframe knowledge should be part of the team.
2. Scheduling
A testing schedule should be drawn in line with the project delivery schedule and it should be realistic.
3. Deliverables
Deliverables should be well defined without any ambiguity and it should be within the scope of test objectives.
4. Execution
Execution should be as per plan and deliverables. There should be periodical reviews with the development team to check the progress and make a course correction.
5. Reporting
Test results should be shared with the development team on a periodical basis. In case of urgent situations, the testing team can get in touch with the development team for immediate correction to maintain continuity.
6. Advantages
The robust testing process avoids unnecessary rework, utilizes resources optimally, reduces production downtime, enhances the user experience, improves customer retention and reduces the cost of IT operation overall.
Conclusion
Mainframe computers still play a vital role in hosting legacy ERP applications for large enterprises as well as hosting back end for Digital applications. Mainframes are here to stay for a longer time. It is paramountly important that Application hosted in mainframe should be tested thoroughly before deployment.
Recommended Articles
This is a guide to Mainframe Testing. Here we discuss the Introduction to Mainframe Testing, Steps to follow in Mainframe Testing, How to do Mainframe Testing along with Methodology. You can also go through our other suggested articles to learn more–
