CN114356889A - Data conversion method, migration method, conversion device and migration device - Google Patents
Data conversion method, migration method, conversion device and migration device Download PDFInfo
- Publication number
- CN114356889A CN114356889A CN202111679983.2A CN202111679983A CN114356889A CN 114356889 A CN114356889 A CN 114356889A CN 202111679983 A CN202111679983 A CN 202111679983A CN 114356889 A CN114356889 A CN 114356889A
- Authority
- CN
- China
- Prior art keywords
- data
- migration
- determining
- target
- mapping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data conversion method, a migration method, a conversion device and a migration device. Wherein, the method comprises the following steps: determining a data mapping relation between a first system and a second system, wherein the first system is a source system of data migration, the second system is a target system of the data migration, and the first system and the second system form the data mapping relation in a target service scene; establishing a data conversion rule of the first system and the second system according to the data mapping relation; and determining the code value mapping relation between the first system and the second system according to the data conversion rule. The invention solves the technical problem of low applicability of data conversion in the prior art.
Description
Technical Field
The present invention relates to the field of data conversion, and in particular, to a data conversion method, a migration method, a conversion apparatus, and a migration apparatus.
Background
As the number of bank clients and the types of services are continuously increased, a new generation of personal service core system is established according to the requirements of service development, and in order to match the construction of a new service core system, data migration work needs to be implemented, namely, service data borne by an old core service system is stably migrated to the new service core system, and the service meanings of all data before and after migration are ensured not to change.
The prior art provides a method for data conversion, specifically, a first field mapping table and a second field mapping table are established, the first field mapping table is a fuzzy matching field, and the second field mapping table is a screening confirmation field. However, the data migration of the new generation of personal business core system has large data scale, wide data range and large difference of data models of new and old systems, and the prior art scheme is difficult to apply.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a data conversion method, a migration method, a conversion device and a migration device, which are used for at least solving the technical problem of low applicability of data conversion in the prior art.
According to an aspect of an embodiment of the present invention, there is provided a data conversion method, including: determining a data mapping relation between a first system and a second system, wherein the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relation in a target service scene; establishing a data conversion rule of the first system and the second system according to the data mapping relation; and determining the code value mapping relation between the first system and the second system according to the data conversion rule.
Optionally, the determining the data mapping relationship between the first system and the second system includes: establishing the entity mapping relation between the first system and the second system according to the entity of the first system and the entity of the second system; and establishing the attribute mapping relation between the first system and the second system based on the entity mapping relation and according to the attributes in the entity of the first system and the attributes in the entity of the second system.
Optionally, the data mapping relationship includes an attribute mapping relationship, and establishing a data conversion rule between the first system and the second system according to the data mapping relationship includes: determining a data processing mode of the first system according to the attribute mapping relation and the business meaning and the compiling rule of the attributes in the second system; generating a conversion rule according to the data processing mode; and coding the conversion rule to obtain the data conversion rule.
Optionally, determining a data processing manner of the first system according to the business meaning and the formulation rule of the attribute in the second system includes: when new data is introduced into the second system, the data processing mode of the second system is supplemented by adopting the coding specification of the first system; when the first system and the second system have data with the same service function, determining that a data processing mode of the first system adopts a direct mapping assignment mode; when the first system and the second system have data with the same service function but different data types, generating a general data processing mode as the data processing mode of the first system; when the first system and the second system have data with different business functions, determining an acquisition mode and a processing method of the data, and generating a special data processing mode as the data processing mode of the first system.
Optionally, after determining the code value mapping relationship between the first system and the second system according to the data conversion rule, the method further includes: and performing data cleaning on invalid code values in the first system.
According to another aspect of the embodiments of the present invention, there is also provided a data migration method, including: and obtaining target data from source data of a first system by adopting any data conversion method, and importing the target data into a second system, wherein the first system is a source system for data migration, and the second system is a target system for data migration.
Optionally, before importing the target data into the second system, the method further includes: and performing quality check and/or consistency check on the target data.
According to another aspect of the embodiments of the present invention, there is also provided a data conversion apparatus, including: the system comprises a first determining module, a second determining module and a data mapping module, wherein the first determining module is used for determining a data mapping relation between a first system and a second system, the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relation in a target service scene; the establishing module is used for establishing a data conversion rule of the first system and the second system according to the data mapping relation; and the second determining module is used for determining the code value mapping relation between the first system and the second system according to the data conversion rule.
According to another aspect of the embodiments of the present invention, there is also provided a data migration apparatus, including: and the migration module is used for acquiring target data from source data of a first system by adopting any data conversion method and importing the target data into a second system, wherein the first system is a source system for data migration, and the second system is a target system for data migration.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, where the computer-readable storage medium includes a stored program, where when the program runs, the apparatus where the computer-readable storage medium is located is controlled to execute the data conversion method described in any one of the above, or execute the data migration method described in any one of the above.
In the embodiment of the present invention, in the data transfer method, first, a data mapping relationship between a first system and a second system is determined, where the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relationship in a target service scenario, then, a data conversion rule between the first system and the second system is established according to the data mapping relationship, and finally, a code value mapping relationship between the first system and the second system is determined according to the data conversion rule. Compared with the data conversion method provided under the condition that the difference of the data models of the new service system and the old service system is small in the prior art, in the scheme, establishing a data conversion rule between the first system and the second system according to the determined data mapping relation between the first system and the second system, determining a code value mapping relation between the first system and the second system according to the established data conversion rule, namely, the scheme can realize the data transfer under the condition that the data models of the new service system and the old service system are greatly different, the method and the device ensure high applicability of data transfer, and the code value mapping relation in the scheme not only ensures consistent meanings of loaded services before and after data transfer, but also ensures high data transfer efficiency and high correctness of new core service system data, thereby solving the technical problem of low applicability of data conversion in the prior art.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a flow diagram of a data conversion method according to an embodiment of the invention;
FIG. 2 is a schematic diagram of analyzing data mappings according to an embodiment of the invention;
FIG. 3 is a diagram illustrating a determination of a data mapping relationship according to an embodiment of the invention;
FIG. 4 is a schematic diagram of analyzing data transformation relationships, according to an embodiment of the invention;
FIG. 5 is a schematic diagram of analyzing a code value mapping relationship according to an embodiment of the invention;
fig. 6 is a flowchart of a data conversion apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For convenience of description, some terms or expressions referred to in the embodiments of the present application are explained below:
old core service system (old core for short): as a whole bank personal deposit remittance core service system, the system realizes the centralized processing of the whole bank personal deposit remittance service, better supports the innovation of personal deposit remittance products and provides better customer service;
a new generation of personal service core system (new core service system for short): the business process is combed and the business requirements are compiled in a business modeling mode, the modular and micro-service IT architecture and IT service design are guided, and the quick deployment and realization of innovative business and scenes thereof are supported;
the data migration system: in order to cooperate with the construction of a new generation of personal service core system, the service and technical data borne by data in an old core service system are stably migrated to the new core service system, and the service meanings of all data before and after migration are ensured not to be changed;
data mapping relation: the new and old core data models are different in nature and different in data standard, and the new and old core data models are combed by combining service scenes through deep understanding of the new and old cores and the contrast relationship of the attributes under the new and old core data models;
converting rules: the attributes of the new and old core data models are different (data length, data type, code value and the like), and data are converted according to some specific rules during data migration and then stored in the new core, wherein the specific rules are conversion rules obtained by further refining the data mapping relation.
In accordance with an embodiment of the present invention, there is provided a data conversion method embodiment, it should be noted that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a data conversion method according to an embodiment of the present invention, as shown in fig. 1, the method including the steps of:
step S102, determining a data mapping relationship between a first system and a second system, wherein the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relationship in a target service scene;
step S104, establishing a data conversion rule of the first system and the second system according to the data mapping relation;
and step S106, determining the code value mapping relation between the first system and the second system according to the data conversion rule.
In the data transfer method, first, a data mapping relationship between a first system and a second system is determined, where the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relationship in a target service scenario, then, a data conversion rule between the first system and the second system is established according to the data mapping relationship, and finally, a code value mapping relationship between the first system and the second system is determined according to the data conversion rule. Compared with the data conversion method provided under the condition that the difference of the data models of the new service system and the old service system is small in the prior art, in the scheme, establishing a data conversion rule between the first system and the second system according to the determined data mapping relation between the first system and the second system, determining a code value mapping relation between the first system and the second system according to the established data conversion rule, namely, the scheme can realize the data transfer under the condition that the data models of the new service system and the old service system are greatly different, the method and the device ensure high applicability of data transfer, and the code value mapping relation in the scheme not only ensures consistent meanings of loaded services before and after data transfer, but also ensures high data transfer efficiency and high correctness of new core service system data, thereby solving the technical problem of low applicability of data conversion in the prior art.
In order to ensure that the established entity mapping relationship and attribute mapping relationship are accurate, in an embodiment of the present application, the determining the data mapping relationship between the first system and the second system includes: establishing the entity mapping relationship between the first system and the second system according to the entity of the first system and the entity of the second system; and establishing the attribute mapping relation between the first system and the second system based on the entity mapping relation and according to the attributes in the entity of the first system and the attributes in the entity of the second system.
In an actual application process, as shown in fig. 2, because the new and old core service systems are built at different periods, the new core service system takes a customer as a center and adopts a unitized architecture to segment data, while the old core service system divides data in service dimensions such as card and discount, and the data models of the two are different in nature; secondly, there are many-to-many, many-to-one, one-to-many mapping relations in the data table and field conversion, so the data mapping relations need to be formed for the data in the new and old core service systems according to the service scenes, and the specific analysis method is as follows: according to the new core data model design specification, the purpose and the business meaning of the data model design are determined; analyzing a service base table under a scene corresponding to the old core system according to a service scene used by the new core data model to form a preliminary range of a base table mapping relation of the new core and the old core; analyzing the service use scene corresponding to the current core base table according to the description document of the old core system base table, and determining whether the service ranges of the new core base table and the old core base table are connected; analyzing the field meaning of the new and old core data model to form the mapping relation of the new and old core attribute level; finding new core added fields with mapping relation in the old core system library table, analyzing to form a primary migration method, and submitting a new core service system for discussion and confirmation; the old core business system base table has business meaning but is not mapped to the base table and the field in the new data model, a migration suggestion is formed through analysis, and the new core is submitted for revision.
In another embodiment of the present application, the data mapping relationship includes an attribute mapping relationship, and the establishing a data transformation rule between the first system and the second system according to the data mapping relationship includes: determining the data processing mode of the first system according to the attribute mapping relation and the business meaning and the compiling rule of the attribute in the second system; generating a conversion rule according to the data processing mode; and coding the conversion rule to obtain the data conversion rule. In the embodiment, the determined data processing mode is used for generating the conversion rule, and the obtained conversion rule is coded to obtain the data conversion rule, so that the service meanings of data bearing before and after migration are ensured not to be changed, and the data conversion rule can be conveniently used by a subsequent data migration function developer.
In a specific embodiment of the present application, as shown in fig. 3, according to an old core library table in an old core service system and a new core entity table in a new core service system, entity mapping relationship analysis is performed to form a corresponding relationship (i.e., an entity mapping relationship) between the old core entity and the new core entity; and finally, according to the formed new and old core entity corresponding relation and the new and old core attribute corresponding relation (namely the data mapping relation), carrying out attribute conversion relation analysis to form an attribute conversion rule (namely the data conversion rule).
In order to ensure that the determined data processing method of the first system is accurate, further ensure that the data meanings before and after data migration are unchanged, and further ensure that the data migration rate is high, in another embodiment of the present application, the determining the data processing method of the first system according to the business meaning and the formulation rule of the attribute in the second system includes: when the second system introduces new data, the coding specification of the first system is adopted to supplement the data processing mode of the second system; when the first system and the second system have data with the same service function, determining that the data processing mode of the first system adopts a direct mapping assignment mode; generating a general data processing mode as the data processing mode of the first system when the first system and the second system have data with the same service function and different data types; and when the first system and the second system have data with different business functions, determining the data acquisition mode and the data processing method, and generating a special data processing mode as the data processing mode of the first system.
Specifically, as shown in fig. 4, the difference of data models between the new and old core service systems is large, and the mapping relationship between data fields is complex; the new core business system is subjected to standard penetration according to the enterprise-level data standard, and the old core business system does not implement a uniform data standard due to historical reasons, so that the data standards of the new core business system and the old core business system are inconsistent; meanwhile, the new core is added with public mechanisms such as products, contracts, quotations and the like, and data needs to be supplemented through complex rules in the conversion process. In order to ensure that the business meanings of data bearing before and after migration do not change, the new and old core data relationships and the business meanings are analyzed to form a data conversion rule, so that a developer of a subsequent data migration function can use the data conversion rule conveniently, and the specific analysis method comprises the following steps: carrying out conversion rule analysis on the basis of the data mapping relation; analyzing a processing mode of data related to a source model (a first system) according to the business meaning and the compiling rule of the attribute of the target model (namely in the second system); new concept data is introduced aiming at the new core, and a new core coding standard is adopted for supplement; aiming at data with the same functions of new and old services, a direct mapping assignment mode is adopted; aiming at data with the same functions of new and old services but different data types, a universal processing mode is formed; aiming at the data of the difference of new and old service functions, a data acquisition mode and a processing method need to be analyzed one by one to form a special data processing mode; classifying and summarizing the data processing modes obtained by analysis, and abstracting a method formation rule which can be used commonly; generating rule conversion according to dimensions such as data types and complexity; and coding the conversion rule to obtain the data conversion rule.
In order to reduce the occupation of invalid code values on resources in the first system and further ensure that the subsequent data transfer efficiency is higher, in an embodiment of the present application, after determining the code value mapping relationship between the first system and the second system according to the data conversion rule, the method further includes: and performing data cleaning on the invalid code values in the first system.
Specifically, as shown in fig. 5, because the standards of the new and old core data are not consistent and the compiling rules of the code values are different, a service scene needs to be used for analyzing according to the new and old core code values, and the mapping relationship between the new and old core code values is arranged to ensure that the service meanings of the data bearers are consistent, and the specific analysis method includes: analyzing the specific service application scene of the target model entity, the attribute and the attribute code value and different dimensionalities divided by the code value, and forming a mapping conversion relation with the current service function; mapping new and old core attribute code values, analyzing a conversion method, and extracting a method forming rule which can be shared; for complex specific code values, independently forming a conversion rule; analyzing the mapping result of the new and old core code values, and determining and supplementing the new core if the new core code value can cover the code value scene of the old core, if the new core code value cannot cover the old core code value scene; data scrubbing needs to be considered for redundant invalid code values in old core data.
In another exemplary embodiment of the present application, there is also provided a data migration method, including: and obtaining target data by adopting any one of the data conversion methods for the source data of the first system, and importing the target data into a second system, wherein the first system is a source system for data migration, and the second system is a target system for data migration.
In the data migration method, firstly, by any of the above data conversion methods, source data in a first system is converted to obtain target data, and then the obtained target data is imported into a second system, where the first system is a source system for data migration and the second system is a target system for data migration, that is, the scheme realizes conversion of the source data in the first system to obtain the target data, and then the target data is imported into the second system, that is, data transfer can be realized even under the condition of huge difference of data models of new and old service systems, and the method ensures high applicability, consistent service meanings of data bearers before and after data migration, high data migration efficiency, and high correctness of data of a new core service system.
In order to further ensure that the accuracy of subsequently importing the target data into the second system is high, in an embodiment of the present application, before importing the target data into the second system, the method further includes: and performing quality check and/or consistency check on the target data.
In the actual application process, the data verification is divided into two verification methods of quality verification and consistency verification, wherein the quality verification mainly verifies the value range of a code value, a date format, a field length and the like; the consistency check comprises field value check, association check, record number check and summary check, wherein the field value check refers to converted data field check, the association check refers to association relation check between the converted data, the record number check refers to converted data record number check, the summary check refers to summary check of data values before and after conversion aiming at certain fields (such as money amount and the like), namely, the scheme adopts a strict data check method to check before data import, and thus the accuracy of data migration is improved.
The embodiment of the present application further provides a data conversion device, and it should be noted that the data conversion device according to the embodiment of the present application may be used to execute the method for data conversion provided by the embodiment of the present application. The following describes a data conversion apparatus provided in an embodiment of the present application.
Fig. 6 is a schematic diagram of a data conversion apparatus according to an embodiment of the present application, as shown in fig. 6, the apparatus including:
a first determining module 40, configured to determine a data mapping relationship between a first system and a second system, where the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relationship in a target service scenario;
an establishing module 42, connected to the first determining module 40, for establishing a data transformation rule between the first system and the second system according to the data mapping relationship;
a second determining module 44, connected to the establishing module 42, for determining a code value mapping relationship between the first system and the second system according to the data conversion rule.
In the data conversion apparatus, a first determining module is configured to determine a data mapping relationship between a first system and a second system, where the first system is a source system for data migration, the second system is a target system for data migration, and the first system and the second system form the data mapping relationship in a target service scenario; the establishing module is used for establishing a data conversion rule of the first system and the second system according to the data mapping relation; and the second determining module is used for determining the code value mapping relation between the first system and the second system according to the data conversion rule. Compared with the data conversion method provided under the condition that the difference of the data models of the new service system and the old service system is small in the prior art, in the scheme, establishing a data conversion rule between the first system and the second system according to the determined data mapping relation between the first system and the second system, determining a code value mapping relation between the first system and the second system according to the established data conversion rule, namely, the scheme can realize the data transfer under the condition that the data models of the new service system and the old service system are greatly different, the method and the device ensure high applicability of data transfer, and the code value mapping relation in the scheme not only ensures consistent meanings of loaded services before and after data transfer, but also ensures high data transfer efficiency and high correctness of new core service system data, thereby solving the technical problem of low applicability of data conversion in the prior art.
In order to ensure that the established entity mapping relationship and the established attribute mapping relationship are accurate, in an embodiment of the present application, the data mapping relationship includes an entity mapping relationship and an attribute mapping relationship, and the first determining module includes a first establishing submodule and a second establishing submodule, where the first establishing submodule is configured to establish the entity mapping relationship between the first system and the second system according to an entity of the first system and an entity of the second system; the second establishing submodule is configured to establish the attribute mapping relationship between the first system and the second system based on the entity mapping relationship and according to the attribute in the entity of the first system and the attribute in the entity of the second system.
In an actual application process, as shown in fig. 2, because the new and old core service systems are built at different periods, the new core service system takes a customer as a center and adopts a unitized architecture to segment data, while the old core service system divides data in service dimensions such as card and discount, and the data models of the two are different in nature; secondly, there are many-to-many, many-to-one, one-to-many mapping relations in the data table and field conversion, so the data mapping relations need to be formed for the data in the new and old core service systems according to the service scenes, and the specific analysis method is as follows: according to the new core data model design specification, the purpose and the business meaning of the data model design are determined; analyzing a service base table under a scene corresponding to the old core system according to a service scene used by the new core data model to form a preliminary range of a base table mapping relation of the new core and the old core; analyzing the service use scene corresponding to the current core base table according to the description document of the old core system base table, and determining whether the service ranges of the new core base table and the old core base table are connected; analyzing the field meaning of the new and old core data model to form the mapping relation of the new and old core attribute level; finding new core added fields with mapping relation in the old core system library table, analyzing to form a primary migration method, and submitting a new core service system for discussion and confirmation; the old core business system base table has business meaning but is not mapped to the base table and the field in the new data model, a migration suggestion is formed through analysis, and the new core is submitted for revision.
In another embodiment of the present application, the data mapping relationship includes an attribute mapping relationship, and the establishing module includes a first determining submodule, a converting submodule, and a processing submodule, where the first determining submodule is configured to determine a data processing manner of the first system according to the attribute mapping relationship and according to a business meaning and a compiling rule of an attribute in the second system; the conversion submodule is used for generating a conversion rule according to the data processing mode; the processing submodule is used for coding the conversion rule to obtain the data conversion rule. In the embodiment, the determined data processing mode is used for generating the conversion rule, and the obtained conversion rule is coded to obtain the data conversion rule, so that the service meanings of data bearing before and after migration are ensured not to be changed, and the data conversion rule can be conveniently used by a subsequent data migration function developer.
In a specific embodiment of the present application, as shown in fig. 3, according to an old core library table in an old core service system and a new core entity table in a new core service system, entity mapping relationship analysis is performed to form a corresponding relationship (i.e., an entity mapping relationship) between the old core entity and the new core entity; and finally, according to the formed new and old core entity corresponding relation and the new and old core attribute corresponding relation (namely the data mapping relation), carrying out attribute conversion relation analysis to form an attribute conversion rule (namely the data conversion rule).
In order to ensure that the determined data processing mode of the first system is accurate, further ensure that the data meanings before and after data migration are unchanged, and further ensure that the data migration rate is high, in another embodiment of the present application, the first determining submodule includes a supplementing submodule, a second determining submodule, a first generating submodule, and a second generating submodule, where the supplementing submodule is configured to supplement the data processing mode of the second system with the encoding specification of the first system when new data is introduced into the second system; the second determining submodule is used for determining that the data processing mode of the first system adopts a direct mapping assignment mode when the first system and the second system have data with the same service function; the first generation submodule is configured to generate a general data processing method as a data processing method of the first system when the first system and the second system have data with the same service function but different data types; the second generation submodule is configured to, when there is data with different business functions between the first system and the second system, determine an acquisition method and a processing method of the data, and generate a dedicated data processing method as a data processing method of the first system.
Specifically, as shown in fig. 4, the difference of data models between the new and old core service systems is large, and the mapping relationship between data fields is complex; the new core business system is subjected to standard penetration according to the enterprise-level data standard, and the old core business system does not implement a uniform data standard due to historical reasons, so that the data standards of the new core business system and the old core business system are inconsistent; meanwhile, the new core is added with public mechanisms such as products, contracts, quotations and the like, and data needs to be supplemented through complex rules in the conversion process. In order to ensure that the business meanings of data bearing before and after migration do not change, the new and old core data relationships and the business meanings are analyzed to form a data conversion rule, so that a developer of a subsequent data migration function can use the data conversion rule conveniently, and the specific analysis method comprises the following steps: carrying out conversion rule analysis on the basis of the data mapping relation; analyzing a processing mode of data related to a source model (a first system) according to the business meaning and the compiling rule of the attribute of the target model (namely in the second system); new concept data is introduced aiming at the new core, and a new core coding standard is adopted for supplement; aiming at data with the same functions of new and old services, a direct mapping assignment mode is adopted; aiming at data with the same functions of new and old services but different data types, a universal processing mode is formed; aiming at the data of the difference of new and old service functions, a data acquisition mode and a processing method need to be analyzed one by one to form a special data processing mode; classifying and summarizing the data processing modes obtained by analysis, and abstracting a method formation rule which can be used commonly; generating rule conversion according to dimensions such as data types and complexity; and coding the conversion rule to obtain the data conversion rule.
In yet another embodiment of the present application, the apparatus further includes a determining module and a supplementing module, where the determining module is configured to determine whether a code value of the second system covers a code value scenario of the first system after determining a code value mapping relationship between the first system and the second system according to the data conversion rule; the complementing module is configured to complement the code value of the second system when the code value of the second system does not cover the code value scenario of the first system. In this embodiment, by judging whether the code value of the second system covers the code value scenario of the first system, and supplementing the code value of the second system under the condition that it is determined that the code value scenario of the first system is not covered, it is ensured that the obtained mapping relationship between the code values of the first system and the second system is relatively comprehensive, further it is ensured that the service meanings of the data bearers before and after migration do not change, and further the correctness and stability of data migration are ensured.
In order to reduce the occupation of invalid code values on resources in the first system and further ensure that the subsequent data transfer efficiency is higher, in an embodiment of the present application, the apparatus further includes a cleaning module, configured to perform data cleaning on the invalid code values in the first system after determining the code value mapping relationship between the first system and the second system according to the data conversion rule.
Specifically, as shown in fig. 5, because the standards of the new and old core data are not consistent and the compiling rules of the code values are different, a service scene needs to be used for analyzing according to the new and old core code values, and the mapping relationship between the new and old core code values is arranged to ensure that the service meanings of the data bearers are consistent, and the specific analysis method includes: analyzing the specific service application scene of the target model entity, the attribute and the attribute code value and different dimensionalities divided by the code value, and forming a mapping conversion relation with the current service function; mapping new and old core attribute code values, analyzing a conversion method, and extracting a method forming rule which can be shared; for complex specific code values, independently forming a conversion rule; analyzing the mapping result of the new and old core code values, and determining and supplementing the new core if the new core code value can cover the code value scene of the old core, if the new core code value cannot cover the old core code value scene; data scrubbing needs to be considered for redundant invalid code values in old core data.
In another exemplary embodiment of the present application, there is also provided a data migration apparatus, including: and the migration module is used for acquiring target data from source data of a first system by adopting any data conversion method and importing the target data into a second system, wherein the first system is a source system for data migration, and the second system is a target system for data migration.
In the data migration device, the device includes a migration module configured to obtain target data from source data of a first system by using any one of the data conversion methods, and import the target data into a second system, where the first system is a source system for data migration, and the second system is a target system for data migration, that is, this scheme realizes conversion of source data in the first system to obtain target data, and then imports the target data into the second system, that is, data transfer can be realized even when data models of new and old service systems are very different, and it is ensured that applicability is high, service meanings of data bearers before and after data migration are consistent, it is also ensured that data migration efficiency is high, and it is ensured that correctness of new core service system data is high.
In order to further ensure that the accuracy of subsequently importing the target data into the second system is high, in an embodiment of the present application, before importing the target data into the second system, the apparatus further includes a verification module, configured to perform quality verification and/or consistency verification on the target data.
In the actual application process, the data verification is divided into two verification methods of quality verification and consistency verification, wherein the quality verification mainly verifies the value range of a code value, a date format, a field length and the like; the consistency check comprises field value check, association check, record number check and summary check, wherein the field value check refers to converted data field check, the association check refers to association relation check between the converted data, the record number check refers to converted data record number check, the summary check refers to summary check of data values before and after conversion aiming at certain fields (such as money amount and the like), namely, the scheme adopts a strict data check method to check before data import, and thus the accuracy of data migration is improved.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, which includes a stored program, where the program, when executed, controls an apparatus in which the computer-readable storage medium is located to execute the data conversion method of any one of the above items, or execute the data migration method of any one of the above items.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A method of data conversion, comprising:
determining a data mapping relation between a first system and a second system, wherein the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relation in a target service scene;
establishing a data conversion rule of the first system and the second system according to the data mapping relation;
and determining the code value mapping relation between the first system and the second system according to the data conversion rule.
2. The method of claim 1, wherein the data mapping comprises an entity mapping and an attribute mapping, and wherein determining the data mapping between the first system and the second system comprises:
establishing the entity mapping relation between the first system and the second system according to the entity of the first system and the entity of the second system;
and establishing the attribute mapping relation between the first system and the second system based on the entity mapping relation and according to the attributes in the entity of the first system and the attributes in the entity of the second system.
3. The method of claim 1, wherein the data mapping relationship comprises an attribute mapping relationship, and wherein establishing the data transformation rule between the first system and the second system according to the data mapping relationship comprises:
determining a data processing mode of the first system according to the attribute mapping relation and the business meaning and the compiling rule of the attributes in the second system;
generating a conversion rule according to the data processing mode;
and coding the conversion rule to obtain the data conversion rule.
4. The method of claim 3, wherein determining the data processing mode of the first system according to the business meaning and formulation rule of the attribute in the second system comprises:
when new data is introduced into the second system, the data processing mode of the second system is supplemented by adopting the coding specification of the first system;
when the first system and the second system have data with the same service function, determining that a data processing mode of the first system adopts a direct mapping assignment mode;
when the first system and the second system have data with the same service function but different data types, generating a general data processing mode as the data processing mode of the first system;
when the first system and the second system have data with different business functions, determining an acquisition mode and a processing method of the data, and generating a special data processing mode as the data processing mode of the first system.
5. The method of claim 1, wherein after determining the code value mapping relationship between the first system and the second system according to the data conversion rule, the method further comprises:
and performing data cleaning on invalid code values in the first system.
6. A method of data migration, comprising:
the method comprises the steps of obtaining target data from source data of a first system by adopting the data conversion method of any one of claims 1 to 5, and importing the target data into a second system, wherein the first system is a source system of data migration, and the second system is a target system of data migration.
7. The method of claim 6, wherein prior to importing the target data into the second system, the method further comprises:
and performing quality check and/or consistency check on the target data.
8. A data conversion apparatus, comprising:
the system comprises a first determining module, a second determining module and a data mapping module, wherein the first determining module is used for determining a data mapping relation between a first system and a second system, the first system is a source system of data migration, the second system is a target system of data migration, and the first system and the second system form the data mapping relation in a target service scene;
the establishing module is used for establishing a data conversion rule of the first system and the second system according to the data mapping relation;
and the second determining module is used for determining the code value mapping relation between the first system and the second system according to the data conversion rule.
9. A data migration apparatus, comprising:
the migration module is configured to obtain target data from source data of a first system by using the data conversion method according to any one of claims 1 to 5, and import the target data into a second system, where the first system is a source system of data migration, and the second system is a target system of data migration.
10. A computer-readable storage medium, comprising a stored program, wherein when the program runs, the program controls a device in which the computer-readable storage medium is located to execute the data conversion method according to any one of claims 1 to 5, or execute the data migration method according to any one of claims 6 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111679983.2A CN114356889A (en) | 2021-12-31 | 2021-12-31 | Data conversion method, migration method, conversion device and migration device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111679983.2A CN114356889A (en) | 2021-12-31 | 2021-12-31 | Data conversion method, migration method, conversion device and migration device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114356889A true CN114356889A (en) | 2022-04-15 |
Family
ID=81104727
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111679983.2A Pending CN114356889A (en) | 2021-12-31 | 2021-12-31 | Data conversion method, migration method, conversion device and migration device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114356889A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114936199A (en) * | 2022-07-21 | 2022-08-23 | 平安银行股份有限公司 | Data processing method for system reconstruction, computer equipment and storage medium |
-
2021
- 2021-12-31 CN CN202111679983.2A patent/CN114356889A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114936199A (en) * | 2022-07-21 | 2022-08-23 | 平安银行股份有限公司 | Data processing method for system reconstruction, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103927615B (en) | Entity is associated with classification | |
| CN110428555A (en) | Shared charging equipment gives back method and device | |
| CN101923563A (en) | Messaging device, information processing method and program | |
| CN105721629A (en) | User identifier matching method and device | |
| CN110263145A (en) | A kind of electronic contract signature method and electronic contract sign equipment | |
| CN110008397A (en) | A kind of recommended models training method and device | |
| CN107798579A (en) | The generation method and its terminal of a kind of document of agreement | |
| CN109615504A (en) | Products Show method, apparatus, electronic equipment and computer readable storage medium | |
| CN109062957A (en) | The acquisition of case source and recommended method and system | |
| CN110852559A (en) | Resource allocation method and device, storage medium and electronic device | |
| CN112434501A (en) | Work order intelligent generation method and device, electronic equipment and medium | |
| CN110796458A (en) | Information management system | |
| CN114356889A (en) | Data conversion method, migration method, conversion device and migration device | |
| CN104965846B (en) | Visual human's method for building up in MapReduce platform | |
| CN110555023A (en) | Data storage method and device based on credit contract | |
| CN113656699A (en) | User feature vector determination method, related device and medium | |
| CN110544143B (en) | Contract establishing method and device for grouping credit project | |
| KR102104162B1 (en) | Generating network resource | |
| CN113535774A (en) | Service data processing method and device | |
| CN106951225B (en) | Data processing method and device based on service configuration template | |
| CN107491460A (en) | The data mapping method and device of adaption system | |
| CN114997109A (en) | Receipt conversion method and device, computer equipment and storage medium | |
| CN113448876B (en) | Service testing method, device, computer equipment and storage medium | |
| CN111144990B (en) | Recommendation method and system | |
| CN114218914A (en) | Service matching method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |