What is Data Migration Testing: Strategy, Types and Challenges

In data migration testing, we verify if the migration of the legacy system (old system/application) to the new system happens with minimal disruption/downtime, with complete data integrity, and without any loss of data, while making sure that all the specified functional and non-functional aspects of the application are met post-migration.

When an application is moved to a different server and the technology is updated to the next version, the testing team ensures that the application is tested thoroughly end to end alongside the successful migration from the existing system to the new system.

System testing would be performed in this case with all the old data, (data used in the old application) and the new data as well. And the existing functionality is to be verified along with the new/modified functionality. Along with new/upgraded applications, it is mandatory to continue testing legacy applications until the new/upgraded ones become stable and consistent.

The different types of migration testing that take place are:

1. Database Migration where all the data in the application’s database is migrated to another database.
2. Application Migration is where an entire application is migrated from one platform/environment to another. .
3. OS Migration is where an application is migrated from one operating system (OS) to another.
4. Server Migration where the server data, along with configuration, is moved from one server to another server

The different reasons for performing migration testing

Data migration to a new system could be for different reasons such as system consolidation, obsolete technology, optimization, or any other.

During the process where the system in use is migrated to a new system, it is essential to make sure that

• Disruption or inconvenience caused to the user, such as downtime or data loss because of migration, needs to be avoided/minimized. 
• The user is able to continue to use all the features of the software with minimal or no damage during migration. For e.g., changes in the functionality or the removal of a particular functionality
• All possible glitches/hindrances that might occur during the actual migration of the live system are anticipated and ruled out/resolved.

Thus, migration testing is undertaken in a lap in order to ensure a smooth migration of the live system by eliminating those defects.

When it comes to data, migration testing assumes great significance. Technically, migration testing is required for the following:  

• To check for compatibility of the new/upgraded application with all hardware and software that the legacy application supports. And, for new software and new hardware platforms, new compatibility should be tested too. 
• To make sure that all the existing functionalities work as they did in the legacy application, without any change in the way the new application works.
• With the possibility of a large number of defects due to migration being high and with many of these being related to data, these defects need to be identified and fixed during testing.
• To ensure that the system response time of the new application is the same or less than what it takes while running the legacy/old application.
• To make certain that the connection between servers, hardware, software, etc., are all intact and do not break while testing. It is important that data flow between different components does not break at all.

Migration testing has to be performed both before and after migration.

Migration testing occurs in different phases: 

1. Pre-Migration Testing
2. Migration Testing
3. Post Migration Testing

As part of the Migration activity, the following tests are also executed in addition to the above:

1. Backward Compatibility Verification
2. Rollback Testing

Before migration testing, it is important for the tester

• To be aware of the changes happening as a part of the new system (server, data flow, front end, schema, DB, functionality, etc.,)
• And, to comprehend the migration strategy proposed by the team: the actual process of migration, the step-by-step changes that happen in the backend of the system, and the corresponding scripts.

A thorough analysis of the old and the new systems is warranted so that test cases can be planned and designed accordingly.

Strategy: Data migration testing 

Designing the test strategy for migration is critical as it helps minimize the errors and risks that occur on account of migration and ensures effective migration testing.

• Customized or specialized team: The testing team should comprise experienced and knowledgeable members and they need to be trained with respect to the system that is being migrated.
• Business risk analysis and possible errors analysis: Business Risk Analysis involves stakeholders (Business Analyst, Architects, Product Owners, Test Manager, Business Owner etc.,) and identify the risks and the implementable mitigations. The testing should thus include all those scenarios to uncover those identified risks and check if mitigations are in place. ‘Possible Error Analysis’ using ‘Error Guessing Approaches’ is conducted and then tests need to be designed around these errors to resolve them during testing.
• Analysis of the scope of migration: When and what needs to be tested is decided in the scope analysis.
• Identification of the fitting tool for Migration testing: It is important to identify the tool – automated or manual – to be used even while defining the strategy of this testing such as automated tool to compare source and destination data.
• Appropriate test environment for Migration: It is necessary to identify separate environments for Pre and Post Migration to carry out verifications if any as part of the testing process. The technical aspects of both the legacy and new systems of migration need to be documented to set up the test environment accordingly.
• Migration test specification document: The specification document is prepared, describing the test approach, areas and methods of testing, (automated, manual), the number of cycles of testing, methodology of testing (black box, white box testing), testing schedule, usage of live data (masking sensitive data), specification of the test environment. 
• Production launch of the migrated system: The to-do list for production needs to be published well in advance.

The different phases of Migration Testing

Phase 1: Pre-migration testing

Before data migration begins, a set of testing activities are carried out as part of the Pre-Migration test phase. While this step is ignored while migrating simpler applications, it is a must for complex applications.

During this phases, the following activities need to be performed.

• Set a clear scope of the data 
• Perform higher and lower levels of data mapping between the legacy and the new application.
• Study the new application’s data schema – such as field names, types, length, minimum and maximum values, etc.
• The number of tables in the legacy system are to be noted down and if any tables are dropped or added post-migration, they need to be verified.
• The number of records in each table are to be noted in the legacy application.
• The interfaces and their connections in the new application need to be studied carefully: data flowing should be secure and unbroken.
• Test cases, use cases and test scenarios for new conditions in new applications need to be prepared.
• In prior, a set of test cases and test scenarios need to be executed, with a set of users and the results/logs need to be stored and verified after migration to ensure that data of the legacy system and the functionality are intact.
• The data count should be kept so that they can be verified after migration so that there is no loss of data.

Phase 2: Migration testing

Migration is carried out following the ‘Migration Guide’ prepared by the Migration team. It commences with a backup of data on the tape so that at any point in time, the legacy system can be restored.

Verifying the documentation takes place to see if the document is clear and easy to follow and if all the scripts and steps are documented correctly without any ambiguity. Errors in documentation and mismatches need to be reported and fixed.

The time taken to migrate the system is to be recorded in the final rest report delivered as part of the migration test results because it is relevant during the production launch. The downtime thus recorded is extrapolated to arrive at the approximate downtime in the live system. The migration activity typically undertaken on the legacy system will include:

• The application’s actual migration
• Modification of firewalls, hosts, port, hardware, and software configurations as per the new system 
• Performance of security checks
• Checking of the connectivity between all the components of the application

Upon completion of the migration activity as specified in the guide, all the servers are brought up, and basic tests involving the verification of successful migration would be done, ensuring that all the end to end systems are correctly connected and there is smooth interaction among all the components. If the software supports multiple different platforms, migration needs to be verified on every one of those platforms separately. 

Migration testing usually will be a combination of both white box and Black box testing. Once the migration activity is completed and all related verification is done and the corresponding tests are passed, the post-migration testing phase begins.

Phase 3: Post-migration testing

Here, end to end system testing is performed in the testing environment. Testers execute identified test cases, use cases with legacy and new set of data, and test scenarios. In addition to these, backward compatibility and rollback are to be verified in the migrated environments. These are all documented as a test case and included in the ‘Test Specification’ document.

As the scope of post migration test is very big, important tests need to be prioritized so as to know if the migration is successful. 

Automation of end-to-end functional test cases and other possible test cases is advised in order to reduce testing time and for ensuring quick availability of the results.

• Backward compatibility testing: Testers also need to verify Backward Compatibility, that is, if the new system is compatible with the old system ( 2 versions at least) to ensure that the functionality supported in the earlier 2 versions are supported by the new system and that system migration can be successfully performed from the earlier 2 versions without any hitch.
•  Rollback testing: In the event of issues or failure during the migration, a roll back to the legacy system should be made possible in order to resume its function quickly and smoothly without any impact on the user. For this, migration failure test scenarios ought to be designed as part of negative testing; also, the rollback mechanism needs to be tested. 

Following the rollback, the main functionality and automated regression testing need to be run to validate that migration has not impacted anything and that the rollback is successful in bringing back the legacy system in place.

Data Migration test summary

The test summary report is to be produced after completing the testing and should cover a summary of the various tests/scenarios, with the result status (pass/fail) and the test logs. It should contain the total time taken for migration, downtime of the applications, time spent on migrating 10000 records, and the time spent on rollback.

Issues and challenges in data migration testing

• The quality of data:  Data used in the legacy application could be of poor quality in the new/upgraded application. In such cases, the quality of data has to be improved to meet business expectations/standards. If we examine as to what leads poor data quality, the factors turn out to be assumptions, invalid data entered in the legacy application, data conversion after migration. This leads to high operational costs and increased data integration risks. 
• Mismatch of data: There could be a data mismatch in the data of new system with the data migrated from the legacy; this could be on account of change in data type, data storage format, redefinition of the purpose for which the data is used. A great deal of effort to either correct the mismatched data or tweak it to the purpose at hand.
• Loss of data: Data loss is likely while migrating from legacy systems to new ones. If data loss occurs in non-mandatory fields, then the record would remain valid and can be updated again; but in the case of mandatory fields, the record itself becomes void and cannot be retracted. This would translate to huge data loss which would have to be retrieved either from the backup database or audit logs.
• Volume of data: There are situations when a large amount of data would need to be migrated within the downtime window of migration for example, users on an intelligent network platform, scratch, etc. where the challenge is that by the time the legacy data gets cleared, a large amount of new data would be created which needs to be migrated again. 
• Simulating a real-time environment (with the actual data): Yet another challenge is the simulating real-time data and real system which throw up issues that are not otherwise encountered while testing. So, sampling of real data, replication of real-time environment, and identification of volume of data during the migration activity are all critical while performing migration testing.
• Simulation of the volume of data: Simulation of real time data can be challenging: teams would have to study the data in the live system carefully and produce a typical analysis and sampling of the data. If data cannot be obtained from life, data creation needs to be undertaken in the testing environment. They need to use automated tools to create a large volume of data and use extrapolation if the volume cannot be simulated.

The sheer complexity of data migration testing warrants a thorough and careful study of the system before and after migration. It also means roping in skilled and well-trained testers, planning and designing effective migration strategies, and using robust tools. Smooth migration is a challenging process and it reflects the quality of the application. The verification of functionality, performance, usability, security, reliability, compatibility, etc., ensures a successful ‘migration testing’.

8 Tips to Smoothen the Data Migration Testing Process

1. Have a detailed plan in place: Before starting the migration, make sure to have a comprehensive plan that outlines the steps involved, the timelines, and the resources required.
2. Test the migration: Perform thorough testing of the migration process on a small scale before rolling it out on the entire data set. This will help identify and fix any potential issues early on.
3. Back up your data: Ensure that you have a reliable backup of your data in case something goes wrong during the migration process.
4. Communicate with stakeholders: Keep all stakeholders informed of the migration process and any potential risks. This will help them plan accordingly and minimize disruptions to their work.
5. Monitor the migration process: Continuously monitor the migration process to ensure that it is proceeding as planned and to quickly identify and address any issues that arise.
6. Have rollback plan: Have a rollback plan in place in case of failure. This will minimize the impact of any issues that occur during the migration process.
7. Have a post-migration plan: Have a plan for after the migration is complete, including testing and validation, as well as a process for addressing any issues that may arise after the migration is complete.
8. Have a support team: Have a dedicated support team in place to handle any issues that may arise during and after the migration.

Conclusion

Are you looking to compare the new data with old data to find any discrepancies in transferring data from databases that are not in use to a different destination? At Hurix, our team of engineers test carries out precise data migration tasks to migrate the existing software to the brand new one without disruption or downtime. We ensure to retain the integrity of data and prevent any loss of data.

To know more about Data Migration Testing solutions from HurixDigital, please write to us at marketing@hurix.com

Devika Deshpande