Why preparation is the most important ingredient of a successful data migration project

There are all kinds of reasons to get started with a new system such as SAP S4/HANA or any other system. In recent years I was involved in several SAP data migration projects. Sometimes it involved companies migrating to a completely new environment. In other cases, the data had to be moved to the systems of the new parent company after an acquisition.

In both situations the steps to be taken are quite similar. And in both cases two things are very clear.

1. No data, no go-live.
2. Ninety percent of your data migration project is preparation.

The perception prevails that data migration is a matter of pressing a button. Export data… Import data… Done. In practice, however, it is much more unruly. The reality is that the information needed in the new system is mostly not stored in a single place. The data may reside in multiple systems, spread across multiple countries. Maybe even in obscure IT solutions or in Excel sheets.

With the data in all those systems, anything can be wrong. Perhaps there is information that has been stored in multiple systems, but in slightly different ways. In one system the color black is called ‘black’. In another system the same color may be called Pantone 19-0303 TCX. Some data might no longer be needed. Old products, for example, or customers to whom you have not been supplying anything for the last several years. Or perhaps your European branch uses centimeters to indicate the size of a product while your new American colleagues work with inches. And maybe some information that should be used in your new system is completely missing.

You can therefore safely consider migrating data to a new system as an indispensable part of your IT project that is seamlessly intertwined with the implementation of the software and the functional processes. In recent years I was closely involved in various data migration projects. At one company we moved 5,000 products, at another we migrated 100,000s of materials. (How many lines you migrate does not matter, the type of work remains the same. The only difference is that it takes longer.)

My colleagues from Expertum and I have combined our experiences from those projects in a migration approach that we have successfully applied multiple times. Our approach guarantees a careful, controlled migration in which data quality is at the center.

In my opinion, a successful data migration project consists for ninety percent of preparation. It is therefore important to get a good overview of the data that is required in the new situation. You can approach this per stream. (HR, Finance, Sales, Production, etc.) But don't forget that there are also all kinds of data that are used cross-company. Think of something as simple as address and contact information. This information is not the responsibility of a specific stream but can be indispensable in all kinds of processes.

After you have determined the scope of your project, can start working on the approach. The ETL methodology is the most suitable approach for data migration in an SAP setting.

Step 1 - Extract

First you need to define and create a mechanism to extract the data from the existing systems. That means gaining insight into all systems and data sources that are used in the current day to day processes – From SAP to, for example, Excel – and making sure you know exactly how to extract the data from those systems, either manually or automated.

Step 2 - Transform

Then you start working on the different objects. An object is a cluster of related data in SAP. Think for example of the object Products, which contains every product you sell, including specifications like color, dimensions, material, and price. The same goes for other objects such as Customers, Suppliers, etc. For each object, you accurately determine which field of which source system contains the correct information and in which field of your target system that information should then land. You also determine field by field how the information from the source system must be enriched or cleaned up before that data can be transferred to the new system.

Step 3 - Load

After the data has been extracted and transformed, it can be safely loaded into the new environment. This means uploading the data to your new SAP S/4HANA environment, object per object. Ideally, you use the standard SAP load tools for uploading the cleansed and enriched data to the target system. Depending on the needs, customized load tools can be built.
Keep in mind that dependencies can exist and that some objects are prerequisites for others. Profit centers, for example, are a prerequisite for products. So, carefully determining the right order of your uploads is of the essence.

Of course, you don't do all this work in the production environment. As a rule, we work with a development, a quality, and a production and thus live environment. In the development environment you check per object whether your tools work. Does the extraction tool work? Does the transformation tool work? Does the load tool do what it's supposed to do? In this phase you you work with a limited dataset. Ten, twenty lines are enough to check if your mechanism works as it’s supposed to.
In the quality environment, we import the 'real' data per object and tweak the transformation rules until the information from each individual field ends up in the target system in the right way. Here, the focus is on the quality of the data.

Then you can go live. Of all the steps you need to take, this seems to be the biggest. Yet it is the least exciting. You have already prepared everything meticulously. You have determined what information you need. You've checked if your extraction tools are OK. The transformation process is right. The dates are OK. Your load tools work like a charm. That means you can now press the go-button with confidence.

Are you curious about how you make S/4HANA data migration a success? Then participate in our Expert Session on October 28, 2021. During that session we will share SAP S/4HANA insights and lessons learned in moving to S/4HANA. My colleagues and I share insights and experiences from projects. We provide ideas on how to get started. And you can connect with your peers and people that have been implementing S/4HANA for different customers. Discover more and join this session.