Schneider Electric - Data migration challenge mastered

How we mastered the challenge of data migration for Schneider Electric.

In this blog post, we report on a major migration project for Schneider Electric.

It was nothing less than the transfer of the entire technical documentation of all business units from one creation and management system to another.

We are talking here about millions of information modules that have been created and managed for many years as so-called topics in an editorial system.

Who is Schneider Electric and what was this project about?

Schneider Electric is a global company leading the digital transformation of energy management and automation. The Group was named one of the World's Most Admired Companies by Fortune in 2021 for the fourth consecutive year.

Since 2016, c-rex.net has been involved in various projects for Schneider Electric.

The project we report on here was commissioned by Schneider Electric Automation GmbH in Germany, where everything revolves around high-performance automation solutions for mechanical engineering.

Since 1999, i.e. for more than 20 years, the technical documentation of the products has been created using an XML-based content management system, TC Toolbox from Ovidius.

Adobe FrameMaker was used to create and publish PDF files, HTML5, and help files.

Since the manufacturer discontinued the system, a changeover to MOTIF, which is set as the company standard in the Schneider Electric group, was to take place.

Over 1,600 manuals were migrated in the source language. One of the biggest challenges was to also migrate the existing translations and establish the relationships between the topics and graphics of the source and target languages. We migrated a total of 6,434 manuals," summarizes Silke Köhler, Process Manager for Technical Publication and Translation at Schneider Electric and project manager of the migration project.The migration covered a total of 3,570,652 objects (Topics, graphics).

The old and the new system for creating, managing and publishing technical documentation

The picture shows a notepad with two sheets. The sheet with the text "Old" is about to be turned over, revealing a sheet with the text "New".

The initial system, the TC Toolbox (TCT)

In this XML-based editorial system, the documentation was created on a module basis.
The document structure resulted from a custom Document Type Definition (DTD) called Mapdoc.
The database model was based on a proprietary structure specified by the editorial system.
Reuse played a major role in this system constellation.

The target system: MOTIF

MOTIF is the name of the system environment that Schneider Electric uses throughout the group to create and publish technical documentation.
MOTIF is based on DITA(Darwin Information Typing Architecture).
MOTIF includes several content creation and management tools from PTC, including Arbortext Editor and Windchill.

So far, so good!

Task, facts and figures

The picture shows a pad and post-its on a desk. Labeling of the block: Facts and Figures.

The task seems trivial at first glance: All content must be extracted from TCT, converted to DITA and prepared for import into MOTIF.

But when you consider the volume and complexity of the data we were dealing with, you understand why we call this project a challenge.

Essentially, the following were the issues that challenged us.

Challenge 1: Huge data pool
It is over three million information objects created and grown over years.
The data to be migrated is the result of more than twenty years of development, during which a wide variety of system updates have led to constant improvement and refinement of the content.
Over 1,600 manuals in the source language were transferred from one system to another. Including up to 13 foreign languages, we are talking about a total of 6,434 manuals!
Challenge 2: Minimize downtime
The migration was to have as little impact as possible on productive operations.
The products were further developed in parallel. At each point in the migration, new product descriptions and other content were also created and published.
Therefore, a concept and migration plan was needed to keep downtime to a minimum. The dataset could therefore not be migrated all at once.
And so each business unit has developed a plan for what data should be migrated and when. Ongoing translations also had to be completed in order to be able to migrate them as well. This allowed the migration timing to be aligned with and take into account release cycles and publication dates of the individual business units.
Challenge 3: Retain reuse
In the old system, reuse played a big role. This should be preserved as much as possible, for the source language and all target languages, especially since one of the strengths of the file format in MOTIF, namely DITA, is the concept of reuse.
The problem that had to be solved was the individual migration dates for each manual and the reuse of information objects in manuals with different migration dates.
The solution was to combine manuals with common fragments into migration batches. This allowed the reuse of a large part of the information objects to be transferred to the new system.
For information used in different versions in the manuals being migrated, reuse was unfortunately lost and had to be restored in the new system.
Challenge 4: Maintain output channels
The MOTIF target system does not support the established HTML5 and CHM output paths as the old system does. The context sensitivity in the published help files would have been lost. This could not be accepted under any circumstances.
Therefore, for MOTIF, the publication sections had to be made available again according to HTML5 and CHM in such a way that the basic requirement regarding context-sensitive help could be fulfilled.
Challenge 5: Target system supports only DITA in standard without specialization
This meant that the Custom XML structure had to be restored in such a way that the basic features from the old system were still available without being able to specialize DITA for it. The content distinguished in TCT via semantic structures had to be converted into structures that could adequately represent this in the DITA environment.
Challenge 7: The time frame with 5 months development time and 6 months migration
In addition, there was the given time frame. With the project starting in February 2020, we had five months to plan and develop a stable migration environment. Start of the actual migration, which we accompanied until completion in December 2020, was set for July 2020.

Step by step into the new system

The picture shows a white paneled wall with three doors in yellow, red and blue.

The migration project was divided into three areas:

Migration of documents from the old to the new system
- Migration of Schneider Electric DTDMapdoc) to DITA
- Development of the implementation concept
Migration of SQL metadata from the old to the new system
What is source language, what is target language and how can the relationships be mapped and migrated?
Export of additional data that is not transferred to the target system
- Export of Translation Memories managed in TCT for import into the Translation Memory System.
- Export and naming of graphics to be able to identify the original graphics in the file repository from within MOTIF.
  Graphics have been and will continue to be created in Microsoft Visio and Adobe Illustrator. For the publication not these original formats are used, but other formats.
  MOTIF manages the formats required for publication, but not other format variants.
  When users revise graphics, they need to find the graphics in the creation format.
  And for this we have included a function that establishes a relationship from the MOTIF internal ID to the graphic stored in the file system in the source format.

And this is what the conversion process looked like.

pictorial representation of the process to migrate from TCT to MOTIF — The conversion process for Schneider Electric at a glance

The basic technology for processing the subtasks for the transfer is our conversion environment c-rex.net DTS.

We configured a Conversion Pipeline as shown in the image and thus achieved the required goals step by step.

XML technologies, DITA and other standards formed the basis for this.

Migration batch by migration batch, we thus converted the documents, restored the relationships between the individual elements, provided external resources for further work in the new environment.

For publication according to HTML5 and CHM with support for context-sensitive access to content, Adobe FrameMaker 2020 is used in production operation, which we have adapted, extended and automated for Schneider Electric 's specific requirements.

Conclusion

A task that at first seemed somewhat trivial, namely the transfer of data from a system based on the XML standard to another system based on XML, turned out to be a great challenge in detail.

Given the enormous amount of data created over more than 20 years of processing, careful planning and deciding what to migrate and to what depth is important.

We managed to complete the project both on schedule and within the available budget, and on time for the end of 2020.

The image shows a calendar with three pins on three date fields.

Another decisive factor for the success of the project was the cooperative and partnership-based collaboration between c-rex.net and Schneider Electric.

The technology contained in our c-rex.net DTS software, as well as our experience with process automation and migration, provided a good basis for managing a project of these dimensions and for designing and configuring a stable environment within a few months, in which the transfer of data to the new MOTIF system could succeed as planned.

c-rex.net made the implementation of this very important Schneider Electric project possible. All deadlines and agreed costs were met, which unfortunately cannot be taken for granted in a project of this size. c-rex.net was a very cooperative partner and always reacted flexibly to requests for changes at short notice," summarizes customer project manager Silke Köhler.

This good cooperation will certainly find a continuation in further projects. We look forward to it and will keep you, dear readers, up to date.