CN115048439A

CN115048439A - Data conversion method and device, electronic equipment and storage medium

Info

Publication number: CN115048439A
Application number: CN202210443651.2A
Authority: CN
Inventors: 李亚南; 陆旖婧; 高嵩淼
Original assignee: Bank of China Financial Technology Co Ltd
Current assignee: Bank of China Financial Technology Co Ltd
Priority date: 2022-04-25
Filing date: 2022-04-25
Publication date: 2022-09-13

Abstract

The invention provides a data conversion method, a data conversion device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring a first data conversion request initiated by a first system; responding to a conversion request, and searching a second system associated with a first system, second data associated with the first data in the second system and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model; merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy; and calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction. The invention realizes that the whole data conversion operation is automatically completed on line, thereby not only greatly reducing the working intensity of manually converting the data one by depending on manpower and improving the conversion efficiency, but also quickly and accurately obtaining the conversion result.

Description

Data conversion method and device, electronic equipment and storage medium

Technical Field

The present invention relates to the field of data processing technologies, and in particular, to a data conversion method and apparatus, an electronic device, and a storage medium.

Background

During the integration and upgrade of the information system, the system usually affects the data of multiple systems and performs multiple data conversions when operating the functional module. Each data conversion can only ensure the accuracy of business data storage in a certain data flow direction, and partial data conversion with internal connection needs to be synchronously executed, so that the combined data conversion operation is formed.

Taking customer management as an example, if an enterprise needs to perform technical innovation, a new comprehensive customer management system capable of accommodating customers of different systems as a whole needs to be created. In the transitional period of the enterprise system upgrading, because different clients are distributed in different systems, the data conversion of the systems needs to be accurately and parallelly executed by the multiple systems to ensure the data consistency of the systems.

In a conventional data processing mode, data managers of different systems are responsible for compiling data conversion logic, performing data conversion and verifying consistency of data conversion results of the systems. When the magnitude of data conversion related to one functional scene is increased, the processed data is converted one by one only by manual operation, so that the conversion efficiency is low and errors are easy to occur.

Disclosure of Invention

The invention provides a data conversion method, a data conversion device, electronic equipment and a storage medium, which are used for overcoming the defects that the conversion efficiency is low and errors are easy to occur when the data are manually converted one by one in the prior art and realizing the rapid and accurate conversion of the data.

The invention provides a data conversion method, which comprises the following steps:

acquiring a first data conversion request initiated by a first system; the conversion request comprises a conversion strategy corresponding to the first data;

responding to the conversion request, and searching a second system associated with the first system, second data associated with the first data in the second system and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model;

merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy;

and calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction.

According to the data conversion method provided by the invention, the conversion policy corresponding to the second data comprises a lexical conversion policy of the second data, an execution sequence of a data table to which the second data belongs, and an execution sequence of a second system to which the second data belongs.

According to the data conversion method provided by the invention, the mapping relations in the data conversion model comprise a first mapping relation among systems, a second mapping relation among data tables, a third mapping relation among data and a fourth mapping relation among data and a lexical conversion strategy;

the searching for a second system associated with the first system, second data associated with the first data in the second system, and a conversion policy corresponding to the second data according to a mapping relationship in a data conversion model includes:

searching the second system and the execution sequence of the second system according to the attribute information of the first system and the first mapping relation;

according to the attribute information of a first data table to which the first data belongs and the second mapping relation, searching a second data table associated with the first data table in the second system and the execution sequence of the second data table;

searching second data associated with the first data in the second data table according to the attribute information of the first data and the third mapping relation;

and searching a lexical conversion strategy of the second data according to the attribute information of the second data and the fourth mapping relation.

According to a data conversion method provided by the present invention, the searching for a second system associated with the first system, second data associated with the first data in the second system, and a conversion policy corresponding to the second data according to a mapping relationship in a data conversion model includes:

acquiring a system range influenced by the conversion operation of the first data;

and searching the second system, the second data and a conversion strategy corresponding to the second data in the system range according to the data conversion model by taking the most critical core system in the system range as a reference.

According to a data conversion method provided by the present invention, after the invoking of the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction, the method further includes:

recording conversion results corresponding to the combined data conversion instructions returned by the first system and the second system in a conversion state recorder;

and under the condition that the conversion result returned by any one of the first system and the second system is in a conversion failure state, calling the first system and the second system again to execute the conversion action corresponding to the combined data conversion instruction until the conversion results returned by all the systems in the first system and the second system are in a conversion success state.

According to a data conversion method provided by the present invention, the recalling that the first system and the second system execute the conversion action corresponding to the combined data conversion instruction comprises:

counting the total times of repeatedly calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction before the moment corresponding to the current call;

under the condition that the total times are greater than a calling time threshold value, adjusting a combined data conversion instruction corresponding to the first data and the second data;

and calling the first system and the second system again to execute the conversion action corresponding to the adjusted combined data conversion instruction.

According to the data conversion method provided by the invention, the conversion result comprises one or more combinations of the identifier of the conversion action, the attribute information of the first system, the attribute information of the second system, the attribute information of the data table to which the first data belongs, the attribute information of the data table to which the second data belongs, the attribute information of the first data, the attribute information of the second data, and the combined data conversion instruction.

The present invention also provides a data conversion apparatus, comprising:

a request acquisition module to: acquiring a first data conversion request initiated by a first system; the conversion request comprises a conversion strategy corresponding to the first data;

a lookup module to: responding to the conversion request, and searching a second system associated with the first system, second data associated with the first data in the second system and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model;

a data conversion generation module to: merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy;

a data conversion execution module to: and calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction.

The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the data conversion method.

The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a data conversion method as described in any of the above.

The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a data conversion method as described in any one of the above.

According to the data conversion method, the data conversion device, the electronic equipment and the storage medium, the second system associated with the first system, the second data associated with the first data in the second system and the conversion strategy corresponding to the second data can be quickly and accurately obtained according to the mapping relation in the data conversion model, the conversion strategies corresponding to the first data and the second data are combined, the combined data conversion instruction is automatically generated, the first system and the second system are automatically called to execute the conversion action corresponding to the combined data conversion instruction, the whole data conversion operation is automatically completed on line, the work intensity of manually converting the data one by means of manpower is greatly reduced, the conversion efficiency is improved, and the conversion result can be quickly and accurately obtained.

Drawings

In order to more clearly illustrate the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a schematic flow chart of a data conversion method provided by the present invention;

FIG. 2 is a second schematic flow chart of a data conversion method provided by the present invention;

FIG. 3 is a schematic structural diagram of data distribution inside a combined data conversion instruction in the data conversion method provided by the present invention;

FIG. 4 is a schematic structural diagram of mapping relationship distribution inside a data conversion model in the data conversion method provided by the present invention;

FIG. 5 is a third schematic flow chart of a data conversion method provided by the present invention;

FIG. 6 is a schematic structural diagram of data distribution inside a conversion result in the data conversion method provided by the present invention;

FIG. 7 is a schematic structural diagram of a data conversion device according to the present invention;

FIG. 8 is a second schematic structural diagram of a data conversion device according to the present invention;

fig. 9 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the descriptions in this application as referring to "first", "second", etc. are for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.

In the existing data conversion, the writing of data conversion logic of each system, the execution of data conversion and the consistency verification of data conversion results are required manually, so that a large amount of communication cost and time cost are required, and the error rate is high.

In order to solve the above problem, this embodiment provides a data conversion method, which supports automatic generation of combined data conversion, supports automatic execution of combined data conversion operation, supports self-recovery characteristics of an abnormal scene, increases automation execution strength, and reduces a risk of manual operation error.

The data conversion method of the present invention is described below with reference to fig. 1, and includes the steps of:

step 101, acquiring a first data conversion request initiated by a first system; the conversion request comprises a conversion strategy corresponding to the first data;

the first system is an initial system that initiates a data conversion request, and may be referred to as an initiating system or a current system hereinafter.

The first data is data that needs to be subjected to data conversion in the first system, and may also be referred to as initiating data hereinafter.

The conversion request may be generated after the service personnel operates the function button, and the first system stores the conversion request of the first data into the database, and formats the first data into the initial data to be converted synchronously.

The conversion request carries information such as attribute information of the first system, attribute information of the first data, operation information, and a conversion policy, which is not specifically limited in this embodiment.

The attribute information of the first system includes but is not limited to a name, an identification and a type identification of the first system; the attribute information of the first data includes, but is not limited to, the name and identification of the first data, and the identification and name of the affiliated data table; the operation information includes, but is not limited to, an operation type name and identification.

Optionally, the first system is monitored in real time to obtain a conversion request of the first data initiated by the first system in real time on line.

Step 102, responding to the conversion request, and searching a second system associated with the first system, second data associated with the first data in the second system, and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model;

the data conversion model stores a plurality of mapping relations in advance according to preset mapping rules, including but not limited to a system-level mapping relation between systems, a direct and/or indirect field-level mapping relation between data, a direct and/or indirect conversion policy-level mapping relation between data and conversion policies, and a data table-level mapping relation between data tables. The conversion strategy comprises a field-level lexical conversion strategy, an execution sequence and the like.

It should be noted that the system-level mapping relationship, the data table-level mapping, the data mapping, and the field-level lexical conversion rule among the systems in the data conversion model can support flexible configuration, that is, the preset mapping rule can be set according to actual requirements, and can also be added, deleted, and modified according to actual requirements.

The second system is a system which needs to synchronously change and store data and is affected when the first system performs the first data conversion, and is also called an affected system in the following.

The number of the second systems may be one or more, which is not specifically limited in this embodiment.

The second data is data that needs to be synchronously converted according to the first data in the first system in the second system under the condition that the data of the first system and the second system are consistent, that is, data affected by the first data conversion, and may also be referred to as affected data hereinafter.

Optionally, in the case of acquiring a conversion request of first data initiated by a first system, a second system associated with the first system, second data associated with the first data in the second system, and a conversion policy corresponding to the second data may be directly searched according to attribute information of the first system, attribute information and operation information of the first data, and a mapping relationship in a data conversion model, which are carried in the conversion request.

The embodiment adds a mapping process from the initiating system to the influencing system for the combined data conversion. Data management personnel no longer need to be concerned about how to generate and perform different data transformations for different business systems. Data management personnel only need to define a data conversion model according to the business system difference criticality degree, the business system difference, the data table analysis, the data field analysis and the like, and labor cost and workload can be effectively reduced.

103, merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy;

as shown in fig. 2, after the conversion policy corresponding to the first data and the conversion policy corresponding to the second data are obtained, since the first system and the second system have a parallel multi-directional relationship, that is, when the first system performs data conversion, the second data in the second system may be simultaneously affected, and in order to ensure data consistency between the first system and the second system, a combination operation of processing the first data and the second data needs to be synchronously executed.

Optionally, the conversion policy corresponding to the first data and the conversion policy corresponding to the second data are merged according to the search result, and the merged conversion policy is converted into a logic statement to generate a combined data conversion instruction, so as to synchronously perform an operation of performing combined data conversion on the first data and the second data.

As shown in fig. 3, each combined data conversion instruction includes, but is not limited to, a combined data conversion unique identification surrogate key, a first system identifier, a first system type identifier, a data table identifier to which the first data belongs, a second system identifier, a second system type identifier, a second data table identifier, a data operation type, a combined data conversion statement, and other information, which is not specifically limited in this embodiment.

For example, the unique identification surrogate key of the combined data conversion in the combined data conversion instruction corresponding to the second data is SDFG5686213, the first system is identified as sys1, the first system type is oracle, the data table to which the first data belongs is identified as customer, the second system is identified as sys2, the second system type is gbase, the data table to which the second data belongs is identified as person, the data operation type is insert, and the combined data conversion statement includes group _ contact (targfield).

And 104, calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction.

In which data is converted into necessary data to be downloaded to an external system due to integration and upgrade of an information system, and a necessary data processing process is required to be performed on the data due to system differences. The conversion actions include, but are not limited to, the following actions: data combination, data splitting, data mapping, and data configuration, which is not specifically limited by this embodiment.

The combined data conversion instruction comprises a data conversion instruction corresponding to the first data and a data conversion instruction corresponding to the second data;

optionally, for the data conversion instruction corresponding to the second data, in the case that the combined data conversion instruction is obtained, the second system may be invoked to execute the conversion action corresponding to the data conversion instruction corresponding to the second data according to the data conversion instruction corresponding to the second data in the combined data conversion instruction.

And simultaneously storing the conversion action corresponding to the data conversion instruction corresponding to the second data into the data conversion state recorder, and updating the conversion result into the data conversion state recorder after receiving the conversion result corresponding to the conversion action returned by the second system.

Similarly, for the data conversion instruction corresponding to the first data, the first system may also be invoked to execute the conversion action corresponding to the data conversion instruction corresponding to the first data according to the invocation step corresponding to the data conversion instruction corresponding to the second data, and receive the conversion result returned by the first system in real time.

According to the embodiment, the second system associated with the first system, the second data associated with the first data in the second system and the conversion strategy corresponding to the second data can be quickly and accurately acquired according to the mapping relation in the data conversion model, the conversion strategies corresponding to the first data and the second data are combined, the combined data conversion instruction is automatically generated, the first system and the second system are automatically called to execute the conversion action corresponding to the combined data conversion instruction, the whole data conversion operation is automatically completed on line, the working intensity of manually converting the data one by manpower is greatly reduced, the conversion efficiency is improved, and the conversion result can be quickly and accurately acquired.

On the basis of the foregoing embodiment, in this embodiment, the conversion policy corresponding to the second data includes a lexical conversion policy of the second data, an execution order of a data table to which the second data belongs, and an execution order of a second system to which the second data belongs.

The lexical conversion strategy is that data with specific meanings input by a certain system is converted into data with using meanings in an external system according to a predefined keyword recognition conversion rule.

The second system comprises a plurality of data tables, and the execution sequence of each data table is used for representing the service execution sequence of the data table in the second system;

the execution sequence of each data table may be predetermined according to the importance and the dependency relationship of each data table in the second system, for example, in the data conversion process, the service corresponding to the data table with the higher importance among the interdependencies is executed first.

The execution order of each system can also be predetermined according to the importance degree and the dependency relationship of each system.

According to the lexical conversion strategy of the second data and the execution sequence of the data table to which the second data belongs, and the execution sequence of the second system to which the second data belongs, the second system can be called to sequentially execute the conversion operation of each second data according to the corresponding execution sequence and the lexical conversion strategy, so that the data can be quickly and accurately converted.

On the basis of the foregoing embodiment, the mapping relationships in the data conversion model in this embodiment include a first mapping relationship between systems, a second mapping relationship between data tables, a third mapping relationship between data, and a fourth mapping relationship between data and a lexical conversion policy; the searching for a second system associated with the first system, second data associated with the first data in the second system, and a conversion policy corresponding to the second data according to a mapping relationship in a data conversion model includes: searching the second system and the execution sequence of the second system according to the attribute information of the first system and the first mapping relation; according to the attribute information of a first data table to which the first data belongs and the second mapping relation, searching a second data table associated with the first data table in the second system and the execution sequence of the second data table; searching second data associated with the first data in the second data table according to the attribute information of the first data and the third mapping relation; and searching a lexical conversion strategy of the second data according to the attribute information of the second data and the fourth mapping relation.

The first mapping relation is used for representing a system-level mapping relation, the second mapping relation is used for representing a data table-level mapping relation, the third mapping relation is used for representing a field-level mapping relation, and the fourth mapping relation is used for representing a lexical conversion strategy-level mapping relation.

As shown in fig. 4, the initiating system in the figure is the first system, and the influencing system is the second system.

The system-level mapping relationship is pre-established with a mapping relationship between the attribute information of the first system and the attribute information of the second system.

The data table level mapping relationship is pre-established with the attribute information of the first system and the mapping relationship between the attribute information of the first data table in the first system and the attribute information of the second data table in the second system.

The field-level mapping relationship is pre-established with the mapping relationship among the attribute information of the first system, the attribute information of the first data table in the first system, the field attribute information in the first data table, the attribute information of the second system, the attribute information of the second data table in the second system and the field attribute information in the second data table.

The lexical conversion strategy level mapping relation is stored with the attribute information of the first system, the lexical conversion strategy of the data in the first system, and the mapping relation between the attribute information of the second system and the lexical conversion strategy of the data in the second system in advance.

Optionally, before step 102 is executed, a data storage manner of the data conversion model may be designed and implemented according to requirements of a multi-system combined data conversion operation, that is, each system is stored according to a first mapping relationship, each data table is stored according to a second mapping relationship, each data is stored according to a third mapping relationship, each data and a lexical conversion policy are stored according to a fourth mapping relationship, for example, appropriate lexical conversion rules which are easy to expand are designed and implemented for fields of each data, and are stored in a database in an appropriate data format.

Optionally, the searching process in step 102 specifically includes the following steps:

firstly, the mapping relation in the data conversion model is identified, and the first mapping relation, the second mapping relation, the third mapping relation and the fourth mapping relation are obtained.

Then, the attribute information of the first system and the first mapping relation are combined to find the system range influenced by the first system, namely, the second system set associated with the first system and the execution sequence of the second system.

Then, combining the attribute information of the first data table to which the first data belongs and the second mapping relation, and searching a second data table associated with the first data table and an execution sequence of the second data table in a data table range of a second system;

then, combining the attribute information of the first data with a third mapping relation, and searching second data associated with the first data in a data range of a second data table, namely a field attribute set of each data table needing data conversion in each second system;

and then, according to the attribute information of the second data and the fourth mapping relation, searching a data conversion rule of fields in a data table of each second system, namely a lexical conversion strategy of the second data.

And finally, integrating the search results of the stages, automatically generating a corresponding combined data conversion instruction, and calling the first system and the second system to execute the combined data conversion instruction.

In addition, when a new system needs to be added in a scene of data conversion between existing systems of multiple systems, only the corresponding mapping rule configuration of the new system and other systems needs to be added in the data conversion model, and by matching with the exception handling mechanism in the implementation, a data administrator can complete the data consistency requirement of the newly added system without repeatedly developing data conversion and paying more attention to data exception handling and with very little manual handling.

In the embodiment, through the mapping relationship, the lexical conversion strategies of each second system, each second data table, each second data and each second data, which are affected by the first data conversion of the first system, are searched step by step, so that the combined data conversion instruction is generated quickly and accurately, and the efficiency and the accuracy of data conversion are improved.

On the basis of the foregoing embodiment, in this embodiment, the searching for the second system associated with the first system, the second data associated with the first data in the second system, and the conversion policy corresponding to the second data according to the mapping relationship in the data conversion model includes: acquiring a system range influenced by the conversion operation of the first data; and searching the second system, the second data and a conversion strategy corresponding to the second data in the system range according to the data conversion model by taking the most critical core system in the system range as a reference.

Optionally, the searching step in step 102 may further include the steps of:

first, a system range affected by the conversion operation of the first data is identified, and specific identification manners include, but are not limited to, a system range related to the conversion operation of taking a business system cluster where the first business system is located as the first data, or a system range related to the conversion operation of obtaining the first data is identified by using a clustering algorithm, and the like, which is not specifically limited in this embodiment.

And then, acquiring the most critical core system in the system range, and taking the core system as a reference, taking the data conversion operation of the core system as a basic universal data conversion operation, and constructing a data conversion model. And with the core system as a reference, in a system range, searching a second system and second data influenced by the conversion operation of the first data and a conversion strategy corresponding to the second data according to the data conversion model.

It should be noted that, before the search step of step 102 is performed, it is necessary to identify the basic generic data transformation in advance, and define the multilevel mapping relationship and the lexical transformation policy in the data transformation model to obtain the data transformation model.

Even if the data is recorded in one piece, various inter-system combined data conversion scenarios exist according to the use requirements of the data. For example, for the customer management function, for the newly added financial statement of the customer, the data record needs to be newly added in the core system, and other associated systems perform corresponding data conversion to newly store the data record to the respective systems, so as to ensure the consistency of the customer data among the systems.

In the embodiment, the system range related to the conversion operation of the first data is firstly obtained, in the system range, the second system and the second data affected by the conversion operation of the first data are searched according to the data conversion model, and the conversion strategy corresponding to the second data is searched, so that the search range can be accurately locked, the search efficiency is improved, the search accuracy can be improved, and the accuracy and the efficiency of data conversion are further improved.

On the basis of the foregoing embodiments, in this embodiment, after the invoking the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction, the method further includes: recording conversion results corresponding to the combined data conversion instructions returned by the first system and the second system in a conversion state recorder; and under the condition that the conversion result returned by any one of the first system and the second system is in a conversion failure state, calling the first system and the second system again to execute the conversion action corresponding to the combined data conversion instruction until the conversion results returned by all the systems in the first system and the second system are in a conversion success state.

The conversion state recorder is used for executing actions aiming at data conversion corresponding to the combined data conversion instruction and recording conversion results returned by each system after the actions are executed. And when the conversion state recorder records that the conversion result returned by any system is in a conversion failure state, the first system and the second system are periodically called again to execute the combined data conversion instruction again.

As shown in fig. 5, after the first system and the second system are called to execute the conversion action corresponding to the combined data conversion instruction, the conversion result corresponding to the combined data conversion instruction returned by the first system and the second system may be received in real time, and the conversion result may be recorded in the conversion state recorder in real time.

And under the condition that the conversion failure state exists in the conversion result recorded in the conversion state recorder, acquiring a combined data conversion instruction corresponding to the conversion result, and calling the first system and the second system again to execute the conversion action corresponding to the combined data conversion instruction until the conversion results returned by all the systems in the first system and the second system are in the conversion success state.

The timing of the recall includes, but is not limited to, starting the recall operation when a conversion failure state exists in the conversion result recorded by the conversion state recorder; or periodically initiate a recall operation, etc., which is not specifically limited by the present embodiment.

In the prior art, data management personnel need to know the data requirement and the abnormal record processing mechanism of each step of data conversion by processing data conversion and abnormal conversion records, so that the data management personnel need to spend more time for analyzing data characteristics and communicating and confirming with a data manager influencing a system, and have no time to consider important matters such as data performance management and the like.

When data conversion of a certain initiating system fails due to error reporting of certain factors, data management personnel of the initiating system and data management personnel influencing the system need to perform synchronous communication, investigation analysis and manual intervention together and confirm whether a final data conversion processing result meets expectations or not; when a certain system environment is abnormal, data managers need to perform manual data conversion supplementary execution according to other data records which affect a certain time period and are manually sent by the system data managers, a large amount of communication cost and time cost are often needed, and the error rate is high.

When the abnormality of multi-data conversion between systems occurs, if each system manager does not communicate in time and intervene manually, inconsistent data can be stored in the system, certain interference is brought to the use of subsequent services, and the evaluation of the services on the use of the system is influenced.

The data conversion method provided in this embodiment not only supports automatic generation of the combined data conversion instruction, but also supports automatic invocation of the corresponding system to execute the conversion operation corresponding to the combined data conversion instruction, and supports the self-recovery characteristic of the abnormal scene, thereby effectively increasing the automation execution strength, reducing the time cost for manually synchronizing the data conversion exception records of each system, and reducing the risk of manual operation errors and the manual cost.

On the basis of the foregoing embodiment, in this embodiment, the recalling that the first system and the second system execute the conversion action corresponding to the combined data conversion instruction includes: counting the total times of calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction before the moment corresponding to the current call; under the condition that the total times are larger than a calling time threshold value, adjusting a combined data conversion instruction corresponding to the first data and the second data; and calling the first system and the second system again to execute the conversion action corresponding to the adjusted combined data conversion instruction.

The threshold value of the calling times can be set according to actual requirements.

Optionally, in order to enable each system to successfully complete data conversion, for combined data for which execution of combined data conversion fails, before re-executing a combined data conversion instruction each time, the total number of times of repeatedly invoking the first system and the second system to execute a conversion action corresponding to the same combined data conversion instruction before a time corresponding to a current invocation, that is, the total number of times of performing combined data conversion failure, needs to be counted.

And after the total times of the combined data conversion execution failures exceed the calling time threshold, adjusting the combined data conversion instructions corresponding to the first data and the second data until the combined data conversion execution is successful.

The method for adjusting the combined data conversion instruction comprises the following steps: and triggering enhancement of the lexical conversion strategy in the fourth mapping relation in the data conversion model until the combined data conversion exception record is successfully converted after the enhancement of the lexical conversion strategy.

It should be noted that, if the combined data conversion fails due to some data factors, the abnormal data conversion records in the conversion status recorder may be summarized and fed back to specific service personnel for subsequent data optimization.

In this embodiment, a transition state recorder is added to perform self-execution on the exception record of which the execution result in the transition state recorder is the execution failure state, so as to implement the self-recovery characteristic in the exception scene. By combining the data conversion modeling and the abnormal scene self-recovery characteristic, the error rate of data conversion and the data recovery cost are reduced, so that a data administrator does not need to process the complex scenes of data conversion of all systems, only needs to pay attention to the most universal basic data conversion and establish a data conversion model, and the workload of the data administrator for developing data conversion logic for many times and manually calling the systems to perform data conversion is reduced. In addition, once data conversion is initiated by a certain system, a combined data conversion instruction with a unique identification surrogate key is generated according to a mapping relation model, and the data conversion instruction and the abnormal automatic repair are automatically executed, so that the data consistency among systems is ensured with little manual intervention.

On the basis of the above embodiment, in this embodiment, the conversion result includes one or more combinations of an identifier of the conversion action, attribute information of the first system, attribute information of the second system, attribute information of the data table to which the first data belongs, attribute information of the data table to which the second data belongs, attribute information of the first data, attribute information of the second data, and an identifier of the conversion result and the conversion instruction of the combined data.

The attribute information of any system of the first system and the second system comprises but is not limited to attribute information such as system identification, system type identification and system name;

the attribute information of any data table of the data table to which the first data belongs and the data table to which the second data belongs includes but is not limited to attribute information such as data table identification and data table name;

the identification of the conversion action is a unique identification surrogate key of the combined data conversion operation;

the conversion result identifies identification information for marking a conversion success or failure state.

As shown in fig. 6, the only identification surrogate key of the combined data transformation operation in the transformation result is SDFG5686213, the first system is identified as sys1, the data table to which the first data belongs is identified as customer, the second system is identified as sys2, the data table to which the second data belongs is identified as person, the transformation result is identified as success or failure, or the transformation result is identified as suc or fail.

Whether each combined data conversion is successful or not can be determined according to the unique identification surrogate key and the conversion result identification of the combined data conversion operation, combined data conversion records which are successfully completed are filtered, combined data conversion instructions which fail in data conversion are summarized, and the first system and the second system are regularly called to execute conversion actions corresponding to the failed combined data conversion instructions again, so that manual communication summary is reduced, and the efficiency and the accuracy of data conversion are effectively improved.

In the embodiment, a data conversion state recorder is added to record the abnormal record, and the abnormal record is self-executed. Data management personnel do not need to care about data consistency after data conversion of each system, and the data conversion state recorder automatically collects the unique identification agent key and the conversion result identification of the combined data conversion operation so as to automatically call the corresponding system to execute the combined data conversion instruction again, thereby realizing the abnormal self-repairing performance. And data management personnel can obtain the result of data consistency between systems in real time only by checking the final execution result.

In summary, the data conversion method provided by the embodiment has the following beneficial effects:

(1) the data conversion method provided by the application supports the custom data conversion strategy, supports the mapping relation among custom combined data conversion, and can realize the automatic execution, abnormal recording, rule reinforcement and automatic repair functions of combined data conversion operation.

(2) The data conversion of a plurality of data flow directions involved in each data record is used as a combined data conversion operation, the combined data conversion operation is formed by data conversion of a plurality of data flow directions, and a mapping relation exists between the data conversion to some extent. The data of the data conversion mapping relation can be stored according to a predefined data conversion model, the configurability is strong, the application range is wide, and the data consistency among multiple systems and the accuracy of data conversion can be effectively ensured.

(3) According to the mapping relation in the data conversion model of the combined data conversion mapping, when data conversion of a certain data flow direction is executed, data conversion of an influence system is automatically generated and executed according to the mapping rule of the data conversion of the current initiating system and the data conversion of the influence system, and the execution state related to the combined data conversion operation is synchronously recorded. And analyzing the abnormal reason of the combined data conversion operation record, and strengthening the combined data conversion mapping model. According to the abnormal record list, the system automatically re-executes a certain combined data conversion operation until the data conversion of each system has no abnormal occurrence, the automatic execution, abnormal record, rule reinforcement and automatic repair functions of the combined data conversion operation can be realized, the automation degree is high, and the correctness, reliability and consistency of the data conversion among the systems can be ensured.

The data conversion apparatus provided by the present invention is described below, and the data conversion apparatus described below and the data conversion method described above may be referred to in correspondence with each other.

As shown in fig. 7, this embodiment further provides a data conversion apparatus, which includes a request obtaining module 701, a searching module 702, a data conversion generating module 703 and a data conversion executing module 704, where:

the request obtaining module 701 is configured to: acquiring a first data conversion request initiated by a first system; the conversion request comprises a conversion strategy corresponding to the first data;

The lookup module 702 is configured to: responding to the conversion request, and searching a second system associated with the first system, second data associated with the first data in the second system and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model;

optionally, in the case of acquiring a conversion request of first data initiated by a first system, a second system associated with the first system, second data associated with the first data in the second system, and a conversion policy corresponding to the second data may be searched according to attribute information of the first system, attribute information and operation information of the first data, and a mapping relationship in a data conversion model, which are carried in the conversion request.

The data conversion generation module 703 is configured to: merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy;

optionally, after the conversion policy corresponding to the first data and the conversion policy corresponding to the second data are obtained, because the first system and the second system have a parallel multi-directional relationship, that is, when the first system performs data conversion, the second data in the second system may be affected synchronously, and in order to ensure data consistency between the first system and the second system, a combination operation of processing the first data and the second data needs to be executed synchronously.

Optionally, the conversion policy corresponding to the first data and the conversion policy corresponding to the second data are merged according to the search result, and the merged conversion policy is converted into a logic statement to generate a combined data conversion instruction, so as to perform a combined data conversion operation on the first data and the second data.

The data conversion execution module 704 is configured to: and calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction.

And simultaneously storing the conversion action corresponding to the data conversion instruction corresponding to the second data into the data conversion state recorder, and updating the conversion result into the data conversion state recorder after receiving the conversion result returned by the second system.

Similarly, for the data conversion instruction corresponding to the first data, the second system may also be invoked to execute the conversion action corresponding to the data conversion instruction corresponding to the first data according to the invocation step corresponding to the data conversion instruction corresponding to the second data, and receive the conversion result returned by the first system in real time.

According to the embodiment, the second system associated with the first system, the second data associated with the first data in the second system and the conversion strategy corresponding to the second data can be quickly and accurately acquired according to the mapping relation in the data conversion model, the conversion strategies corresponding to the first data and the second data are combined to generate the combined data conversion instruction, the first system and the second system are automatically called to execute the conversion action corresponding to the combined data conversion instruction, the whole data conversion operation is automatically completed on line, the working intensity of one-to-one conversion of the data by manpower is greatly reduced, the conversion efficiency is improved, and the conversion result is quickly and accurately acquired.

As shown in fig. 8, the data conversion apparatus further includes a conversion state recorder 801, a service system cluster 802;

the service system cluster 802 is in communication with the data conversion execution module 704, and is in communication with the data conversion generation module 703; the data conversion execution module 704 communicates with the conversion status recorder 801 and communicates with the data conversion generation module 703; the transition state recorder 801 communicates with the data transition generation module 703.

The business system cluster 802 is used to enter first data and store second data affecting the first data operation association to a local business data database.

The data conversion generating module 703 is configured to generate, according to the data conversion model, data conversion that affects other systems, that is, second data conversion in the second system and a combined data conversion instruction of the first data and the second data, for a conversion request of the first data initiated by the first system; and transmits the combined data conversion instruction to the data conversion execution module 704. And the lexical conversion rule in the data conversion model is enhanced according to the abnormal record in the conversion state recorder.

The data conversion executing module 704 is configured to call the first system and the second system to perform corresponding data conversion and record data conversion execution result information returned by the first system and the second system based on the received combined data conversion instruction of the corresponding service system.

The conversion state recorder 801 is configured to record, for the data conversion execution action of the data conversion execution module 704, an execution result corresponding to the execution action returned by each system. When the execution result recorded in the conversion status recorder 801 is the execution failure status, that is, the data conversion status is abnormal, the data conversion execution module 704 periodically calls up again to execute the failed combined data conversion instruction again.

On the basis of the foregoing embodiment, the mapping relationships in the data conversion model in this embodiment include a first mapping relationship between systems, a second mapping relationship between data tables, a third mapping relationship between data, and a fourth mapping relationship between data and a lexical conversion policy;

a lookup module specifically configured to: searching the second system and the execution sequence of the second system according to the attribute information of the first system and the first mapping relation; according to the attribute information of a first data table to which the first data belongs and the second mapping relation, searching a second data table associated with the first data table in the second system and the execution sequence of the second data table; searching second data associated with the first data in the second data table according to the attribute information of the first data and the third mapping relation; and searching a lexical conversion strategy of the second data according to the attribute information of the second data and the fourth mapping relation.

On the basis of the foregoing embodiments, the search module in this embodiment is further configured to: acquiring a system range influenced by the conversion operation of the first data; and searching the second system, the second data and a conversion strategy corresponding to the second data in the system range according to the data conversion model by taking the most critical core system in the system range as a reference.

On the basis of the foregoing embodiments, in this embodiment, the data conversion execution module is further configured to: recording conversion results corresponding to the combined data conversion instructions returned by the first system and the second system in a conversion state recorder; and under the condition that the conversion result returned by any one of the first system and the second system is in a conversion failure state, calling the first system and the second system again to execute the conversion action corresponding to the combined data conversion instruction until the conversion results returned by all the systems in the first system and the second system are in a conversion success state.

On the basis of the foregoing embodiments, in this embodiment, the data conversion execution module is further configured to: counting the total times of repeatedly calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction before the moment corresponding to the current call; under the condition that the total times are greater than a calling time threshold value, adjusting a combined data conversion instruction corresponding to the first data and the second data; and calling the first system and the second system again to execute the conversion action corresponding to the adjusted combined data conversion instruction.

In addition to the above embodiments, in this embodiment, the conversion result includes one or more combinations of an identifier of the conversion action, attribute information of the first system, attribute information of the second system, attribute information of the data table to which the first data belongs, attribute information of the data table to which the second data belongs, attribute information of the first data, attribute information of the second data, and the combined data conversion instruction.

Fig. 9 illustrates a physical structure diagram of an electronic device, and as shown in fig. 9, the electronic device may include: a processor (processor)901, a communication Interface (Communications Interface)902, a memory (memory)903 and a communication bus 904, wherein the processor 901, the communication Interface 902 and the memory 903 are communicated with each other through the communication bus 904. The processor 901 may call logic instructions in the memory 903 to perform a data translation method comprising: acquiring a first data conversion request initiated by a first system; the conversion request comprises a conversion strategy corresponding to the first data; responding to the conversion request, and searching a second system associated with the first system, second data associated with the first data in the second system and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model; merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy; and calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction.

In addition, the logic instructions in the memory 903 may be implemented in a software functional unit and stored in a computer readable storage medium when the logic instructions are sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer-readable storage medium, the computer program being capable of executing, when executed by a processor, a data conversion method provided by the above methods, the method including: acquiring a first data conversion request initiated by a first system; the conversion request comprises a conversion strategy corresponding to the first data; responding to the conversion request, and searching a second system associated with the first system, second data associated with the first data in the second system and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model; merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy; and calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction.

In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which when executed by a processor, implements a data conversion method provided by the above methods, the method comprising: acquiring a first data conversion request initiated by a first system; the conversion request comprises a conversion strategy corresponding to the first data; responding to the conversion request, and searching a second system associated with the first system, second data associated with the first data in the second system and a conversion strategy corresponding to the second data according to a mapping relation in a data conversion model; merging the conversion strategy corresponding to the first data and the conversion strategy corresponding to the second data, and generating a combined data conversion instruction according to the merged conversion strategy; and calling the first system and the second system to execute the conversion action corresponding to the combined data conversion instruction according to the combined data conversion instruction.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A method of data conversion, comprising:

2. The data conversion method according to claim 1, wherein the conversion policy corresponding to the second data includes a lexical conversion policy of the second data, an execution order of a data table to which the second data belongs, and an execution order of a second system to which the second data belongs.

3. The data conversion method according to claim 2, wherein the mapping relationships in the data conversion model include a first mapping relationship between systems, a second mapping relationship between data tables, a third mapping relationship between data, and a fourth mapping relationship between data and lexical conversion policy;

the searching a second system associated with the first system, second data associated with the first data in the second system, and a conversion policy corresponding to the second data according to a mapping relationship in a data conversion model includes:

searching the second data associated with the first data in the second data table according to the attribute information of the first data and the third mapping relation;

4. The data conversion method according to any one of claims 1 to 3, wherein the searching for the second system associated with the first system, the second data associated with the first data in the second system, and the conversion policy corresponding to the second data according to the mapping relationship in the data conversion model comprises:

5. The data conversion method according to any one of claims 1 to 3, further comprising, after the invoking of the conversion action corresponding to the combined data conversion instruction by the first system and the second system, the performing of the conversion action corresponding to the combined data conversion instruction:

6. The data conversion method according to claim 5, wherein the recalling that the first system and the second system execute the conversion action corresponding to the combined data conversion instruction comprises:

7. The data conversion method according to claim 5, wherein the conversion result includes one or more combinations of an identifier of a conversion action, attribute information of the first system, attribute information of the second system, attribute information of a data table to which the first data belongs, attribute information of a data table to which the second data belongs, attribute information of the first data, attribute information of the second data, and the combined data conversion instruction.

8. A data conversion apparatus, comprising:

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the data conversion method according to any one of claims 1 to 7 when executing the program.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data conversion method according to any one of claims 1 to 7.