CN114416847A

CN114416847A - Data conversion method, device, server and storage medium

Info

Publication number: CN114416847A
Application number: CN202210080326.4A
Authority: CN
Inventors: 欧阳伟
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2022-01-24
Filing date: 2022-01-24
Publication date: 2022-04-29

Abstract

The application is applicable to the technical field of data processing, and provides a data conversion method, a data conversion device, a server and a storage medium, wherein the method comprises the following steps: receiving a data migration request; acquiring original code data of the target document from an original database based on the document identification, performing semantic analysis on the original code data, and marking function keywords contained in the original code data; searching a mapping relation between the original database and the target database, and respectively determining mapping keywords corresponding to the function keywords based on the mapping relation; sequentially converting the original code segments corresponding to the function keywords into migration code segments corresponding to the mapping keywords; migration code data is generated for the target database based on all of the migration code segments. By adopting the method, the efficiency of data migration is improved, and meanwhile, the labor cost is reduced.

Description

Data conversion method, device, server and storage medium

Technical Field

The present application belongs to the technical field of data processing, and in particular, to a method, an apparatus, a server, and a storage medium for data conversion.

Background

With the continuous progress of the electronic process, more and more documents can be stored in a data form, and in order to facilitate the management and storage of a large number of electronic documents, the electronic documents can be stored in a database so as to improve the efficiency of document searching and management.

In the development of a business, an original database may not be suitable for a new business scenario, and at this time, an original electronic document needs to be migrated from one type of database to another type of database to implement data migration of the electronic document. However, in the existing data management technology, because the function syntax structures of different databases have great differences, developers are often required to recreate corresponding data tables in a new database and to re-import each item into the created data tables one by one, however, as the service expansion speed is increased, the frequency of database migration is higher and higher, the efficiency of data migration is greatly affected by manually configuring the data tables and re-importing data, and a large amount of labor cost is increased.

Disclosure of Invention

The embodiment of the application provides a data conversion method, a data conversion device, a data conversion server and a storage medium, which can solve the problem that in the existing data management technology, developers are often required to recreate corresponding data tables in a new database and re-import each item into the created data tables one by one, however, as the service expansion speed is increased, the frequency of database migration is higher and higher, the efficiency of data migration is greatly influenced by manually configuring the data tables and re-importing data, and a large amount of labor cost is increased.

In a first aspect, an embodiment of the present application provides a data conversion method, including:

receiving a data migration request; the data migration request comprises a document identifier of a target document to be migrated and a database identifier of the target document to be expected to be migrated;

acquiring original code data of the target document from an original database based on the document identification, performing semantic analysis on the original code data, and marking function keywords contained in the original code data;

searching a mapping relation between the original database and the target database, and respectively determining mapping keywords corresponding to the function keywords based on the mapping relation;

sequentially converting the original code segments corresponding to the function keywords into migration code segments corresponding to the mapping keywords;

migration code data is generated for the target database based on all of the migration code segments.

In a possible implementation manner of the first aspect, before the searching for the mapping relationship between the original database and the target database, and respectively determining the mapping keywords corresponding to the function keywords based on the mapping relationship, the method further includes:

acquiring all first functions in the original database, and determining the function key words corresponding to the first functions;

acquiring all second functions in the target database, and determining the mapping keywords corresponding to the second functions;

determining mapping keywords matched with function keywords in all the mapping keywords, and establishing a corresponding relation between a first function corresponding to the function keywords and a second function corresponding to the matched mapping keywords;

and generating a mapping relation between the original database and the target database according to all the corresponding relations.

In a possible implementation manner of the first aspect, the determining, among all the mapping keywords, a mapping keyword matched with a function keyword, and establishing a correspondence between a first function corresponding to the function keyword and a second function corresponding to the matched mapping keyword includes:

respectively calculating the correlation confidence between the function key words and each mapping key word; the associated confidence is:

wherein, SimiarLv is the correlation confidence coefficient between the function key words and the mapping key words; qst_xaFor the keyword attribute corresponding to the a-th function keyword, Qst_ybThe key word attribute corresponding to the b-th mapping key word; n is the original numberTotal number of keywords in the database; m is the total number of the keywords of the target database; IDF is an inverse text probability calculation function; semta is a semantic similarity calculation function; alpha is a preset coefficient;

and if the correlation confidence between any mapping keyword and the function keyword is greater than a preset matching threshold, selecting the mapping keyword with the maximum correlation confidence as the mapping keyword matched with the function keyword.

In a possible implementation manner of the first aspect, after the respectively calculating the association confidence degrees between the function keywords and the mapping keywords, the method further includes:

if the associated confidence degrees between all the mapping keywords and the function keywords are less than or equal to the matching threshold, identifying the function keywords as keywords to be matched;

identifying a second function corresponding to a mapping keyword which is not related to any function keyword in the target database as a function to be mapped, and combining all the functions to be mapped to generate at least one function group to be mapped; the set of functions to be mapped comprises more than two functions to be mapped;

processing a preset training example through a first function of the keyword to be matched to obtain a first processing result, and processing the training example through the function group to be mapped to obtain a second processing result;

determining the processing similarity between the first function of the keyword to be matched and the function group to be mapped according to the first processing result and the second processing result;

and if the processing similarity between any function group to be mapped and the first function of the keyword to be matched is greater than a preset similarity threshold, establishing the corresponding relation between the first function of the keyword to be matched and the function group to be mapped.

if the mapping relation is not inquired, acquiring a mapping establishment record between the original database and the target database;

if the mapping establishing record does not exist, acquiring a first function library of the original database and a second function library of the target database, and generating the mapping relation according to the first function library and the second function library;

if the mapping establishing record exists, analyzing the mapping establishing record, and determining an association function pair with an established association relation;

and generating mapping configuration information according to all the association function pairs so that a user can establish an association relation for the functions to be configured to obtain the mapping relation between the original database and the target database.

In a possible implementation manner of the first aspect, after the generating migration code data about the target database based on all the migration code segments, the method further includes:

acquiring a plurality of verification data from the original database, and determining a corresponding reference result of each verification data after the verification data is processed through original code data of the target document;

respectively processing each verification data through migrating code data to obtain an actual result;

calculating a deviation value corresponding to the data migration process according to all the reference results and the actual results;

and if the deviation value is larger than a preset deviation threshold value, generating migration failure information.

In a possible implementation manner of the first aspect, the obtaining, based on the document identifier, original code data of the target document from an original database, performing semantic parsing on the original code data, and marking a function keyword included in the original code data includes:

acquiring a first function library of the original database; the first function library comprises a plurality of preset function templates;

respectively determining the function key words corresponding to the function templates to generate a function key dictionary;

marking each function keyword in the original code data and an original code segment where the function keyword appears through the function keyword dictionary.

In a second aspect, an embodiment of the present application provides an apparatus for data conversion, including:

a migration request receiving unit configured to receive a data migration request; the data migration request comprises a document identifier of a target document to be migrated and a database identifier of the target document to be expected to be migrated;

a function keyword marking unit, configured to obtain original code data of the target document from an original database based on the document identifier, perform semantic analysis on the original code data, and mark a function keyword included in the original code data;

the mapping key word determining unit is used for searching the mapping relation between the original database and the target database and respectively determining the mapping key words corresponding to the function key words based on the mapping relation;

a migration code segment generation unit, configured to sequentially convert the original code segments corresponding to the function keywords into migration code segments corresponding to the mapping keywords;

a data migration unit for generating migration code data about the target database based on all the migration code sections.

In a third aspect, an embodiment of the present application provides a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor, when executing the computer program, implements the method according to any one of the above first aspects.

In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the method according to any one of the above first aspects.

In a fifth aspect, embodiments of the present application provide a computer program product, which when run on a server, causes the server to perform the method of any one of the first aspect.

Compared with the prior art, the embodiment of the application has the advantages that: when data migration is needed, acquiring original code data of a target document to be migrated from an original database, and determining a function keyword contained in the original code data; and respectively determining mapping keywords associated with each function keyword through the mapping relation between the original database and a target database to be migrated, automatically translating original code segments containing the function keywords into migration code segments corresponding to the mapping keywords, realizing format conversion of the code segments among the same functions, obtaining corresponding migration code segments for all code segments in a target document in the manner, packaging all the migration code segments to obtain migration code data conforming to the target database, and realizing the purpose of migrating the target document from one database to another database. Compared with the existing data management technology, the data management method and the data management system have the advantages that the data table is not re-established in the target database to be migrated by the user, the data items are led in one by one, the mapping relation among different databases can be established, the corresponding code segments are subjected to format conversion by identifying the function keywords with the mapping relation, the data migration efficiency is greatly improved, and meanwhile, the labor cost is reduced.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a data conversion system according to an embodiment of the present application;

FIG. 2 is a flowchart illustrating an implementation of a method for data transformation according to an embodiment of the present application;

FIG. 3 is a schematic diagram illustrating an implementation manner of a method for data conversion according to an embodiment of the present application;

FIG. 4 is a schematic diagram illustrating an implementation manner of a method for data conversion according to an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating an implementation manner of a method for data conversion according to an embodiment of the present application;

fig. 6 is a schematic diagram illustrating an implementation manner of S202 of a method for data conversion according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an apparatus for data conversion according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.

The data conversion method provided by the embodiment of the application can be applied to electronic devices such as smart phones, servers, tablet computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks and the like. The embodiment of the present application does not set any limit to the specific type of the electronic device. Particularly, the electronic device is specifically an intermediate server, and the intermediate server can acquire corresponding code data from different database servers and convert the code data in one database server into code data suitable for another database server when code data migration is required, so that the purposes of data conversion and data migration are achieved.

Exemplarily, fig. 1 shows a schematic structural diagram of a data migration system provided in an embodiment of the present application. Referring to fig. 1, the data migration system includes three servers, namely, an original database server 11 storing original code data of a target text, a target database server 12 to be converted, and an intermediate server 13 performing data format conversion, where the intermediate server 13 may obtain the original code data of the target document from the original database server 11, convert the original code data into migration code data suitable for the target database server 12, and store the converted migration code data in the target database server 12, thereby completing data migration on the target document.

It should be emphasized that the execution subject of the above-mentioned data conversion method may be the original database server, or the target database server, that is, the original database server and/or the target database server may be installed with the middleware provided in the embodiment of the present application, and format conversion is performed on the original code data through the middleware to implement data migration.

For example, a user may initiate a data migration request within the primary database server, in which case middleware within the primary database server may convert the primary code data into migration code data that matches the target database server and send to the target database server.

For another example, the user may initiate a data migration request within the target database server, in which case the target database server may obtain the original code data from the original database server via the middleware, convert the original code data into migration code data that matches the local database, and store the migration code data in the local target database server.

Referring to fig. 2, fig. 2 is a schematic diagram illustrating an implementation of a method for data conversion according to an embodiment of the present application, where the method includes the following steps:

in S201, a data migration request is received; the data migration request comprises a document identification of a target document to be migrated and a database identification of the expected migration of the target document.

In this embodiment, a large number of data tables are stored in the original database, and each data table can be regarded as a target document, such as a data table for statistics of policy and a data table for statistics of employee personnel data. When the target document is stored in the original database, the target document is specifically in the form of code data matched with the format of the original database, and the original code data corresponding to the target document is formed. In this case, the user may generate a data migration request when it is necessary to migrate one or some data tables to another database. The data migration request may carry a document identifier of a target document to be migrated and a database identifier of an expected migration of the target document.

In this embodiment, the document identifier is specifically used to uniquely determine the document to be migrated, and the document identifier may be a document number or a document name. Certainly, when the user needs to perform batch migration, one or more target documents may be selected by means of checking or frame selection, and based on the document identifiers of the selected target documents, the selected target documents may be encapsulated in one data migration request, that is, one data migration request carries the document identifiers of the target documents.

In this embodiment, the database identifier is used to uniquely determine the type of the database that needs to be migrated, such as an Oracle database, a PostgreSQL database, a MySql database, and the like. Of course, the database identifier may uniquely determine the type of the database to be migrated, and may also determine a corresponding server, such as a communication address, a network address, or a domain name of the server. In this case, the electronic device may send a type obtaining request to the server of the target database according to the communication address and the network address, so that the server of the target database feeds back the database type of the electronic device.

In a possible implementation manner, if the electronic device is a target database server, in this case, the database identifier may specifically be a database identifier corresponding to an original database server.

In S202, based on the document identifier, the original code data of the target document is acquired from the original database, and the original code data is subjected to semantic parsing, so as to mark the function keywords included in the original code data.

In this embodiment, the electronic device may analyze the data migration request, obtain a document identifier carried in the data migration request, and determine corresponding code data, that is, the original code data, according to the fact that the document identifier is missing from the original database. The original code data is code data generated in the data format of the original database.

In this embodiment, after obtaining the original code data of the target document, the electronic device may perform semantic analysis on the original code data, determine whether the original code data includes a function keyword associated with a function library of the original database, if so, mark the function keyword in the original code data, and use a code segment where the function keyword appears as an original code segment corresponding to the function keyword.

Illustratively, the original database is an Oracle database, where a certain code segment appears as "true (date, 'MM')", if a function true corresponding to the Oracle database is included in the code segment, the function key is marked in the original code data, and "true (date, 'MM')" is identified as the original code segment corresponding to the function key of the true.

In S203, a mapping relationship between the original database and the target database is searched, and mapping keywords corresponding to the function keywords are respectively determined based on the mapping relationship.

In this embodiment, after determining each function keyword included in the original code data, the electronic device may map a mapping keyword corresponding to the function keyword in the target database according to a preset mapping relationship. The function with the same function often exists in different databases, and the function expression formats adopted by different functions are different. Therefore, when data conversion is carried out, two functions with incidence relation can be determined firstly, and therefore data conversion can be achieved according to format differences among the functions with the same functions in different databases.

In this embodiment, the electronic device may pre-establish a mapping relationship between the original database and the target database, where the mapping relationship records mapping keywords having the same function in the target database for each function keyword in the original database, that is, corresponding relationships between the function keywords and the mapping keywords, and all the corresponding relationships constitute the mapping relationship between the original data and the target database. Based on this, the electronic device may determine a mapping keyword corresponding to the function keyword from the mapping relationship, such as a TRUNC function in an Oracle database, where the corresponding function keyword is "TRUNC", and in a PG database, the corresponding function is a DATE _ TRUNC function, and the corresponding function keyword is "DATE _ TRUNC", and then the mapping relationship may include "Oracle: TRUNC-PG: and determining the mapping key words of the function key words in the original database in the target database according to the corresponding relation of the DATE _ TRUNC'.

In S204, the original code segments corresponding to the function keywords are sequentially converted into migration code segments corresponding to the mapping keywords.

In this embodiment, after determining the mapping key corresponding to each function key in the target database, the electronic device may determine an original function format corresponding to the function key in the original database and determine a mapping function format of the mapping key in the target database. The electronic equipment can determine each function parameter in the original code segment corresponding to the function key word and the parameter position of each function parameter according to the original function format, perform mutual comparison according to the original function format and the mapping function format, determine the target position corresponding to each function parameter in the mapping function format, and add the corresponding function parameter into the associated target position in the mapping function format, thereby obtaining the migration code segment of the function format of the composite target database.

For example, the original function format of the TRUNC function in the Oracle database is: TRUNC (data 'month'), and in the PG database, the mapping function format corresponding to the function as DATE _ TRUNC function is: DATE _ trunk ('month', data). If a certain code segment in the original code data is TRUNC (zhangsan125, '12'), the value corresponding to the data parameter is "zhangsan 125," and the value corresponding to the month parameter is "12," and in the mapping function format of the target database, the first parameter is the month parameter, and the second parameter is the data parameter, so the mapping code data after conversion is: DATE _ true ('12', zhangsan 125).

In S205, migration code data about the target database is generated based on all the migration code segments.

In this embodiment, after converting each original code segment in the original code data into a corresponding migration code segment, the electronic device may encapsulate all migration code segments, thereby generating migration code data for the target database. The function format of the migration code segment is generated according to the function format corresponding to the mapping keyword of the target database, namely the data format of the migration code segment is suitable for the target database, so that the purpose of migrating the target document from the original database to the target database can be realized.

As can be seen from the above, in the data conversion method provided in the embodiment of the present application, when data migration is required, original code data of a target document to be migrated is obtained from an original database, and a function keyword included in the original code data is determined; and respectively determining mapping keywords associated with each function keyword through the mapping relation between the original database and a target database to be migrated, automatically translating original code segments containing the function keywords into migration code segments corresponding to the mapping keywords, realizing format conversion of the code segments among the same functions, obtaining corresponding migration code segments for all code segments in a target document in the manner, packaging all the migration code segments to obtain migration code data conforming to the target database, and realizing the purpose of migrating the target document from one database to another database. Compared with the existing data management technology, the data management method and the data management system have the advantages that the data table is not re-established in the target database to be migrated by the user, the data items are led in one by one, the mapping relation among different databases can be established, the corresponding code segments are subjected to format conversion by identifying the function keywords with the mapping relation, the data migration efficiency is greatly improved, and meanwhile, the labor cost is reduced.

Fig. 3 is a flowchart illustrating a specific implementation of a method for data conversion according to a second embodiment of the present invention. Referring to fig. 3, in the embodiment, with respect to fig. 2, before the searching for the mapping relationship between the original database and the target database and determining the mapping keywords corresponding to the function keywords respectively based on the mapping relationship, the method for data conversion according to the embodiment further includes: s301 to S304 are detailed as follows:

further, before the searching for the mapping relationship between the original database and the target database and respectively determining the mapping keywords corresponding to the function keywords based on the mapping relationship, the method further includes:

in S301, all first functions in the original database are obtained, and the function keyword corresponding to each first function is determined.

In this embodiment, the electronic device may automatically establish a mapping relationship between different databases. In order to determine that functions with the same function exist in different databases, all first functions in original data need to be acquired, and function keywords corresponding to the first functions are determined. The obtained first function may be a function template of an original database, for example, "true (data 'month')", does not carry any actual function parameter, but defines a data type corresponding to different positions, that is, a function template, which may also be referred to as a format corresponding to the function; the first function may also be an example of a function corresponding to the function, such as "DATE _ true (zhangsan123, '12') - - -zhangsan: data, 12: month ", i.e. contains the example function arguments.

In S302, all the second functions in the target database are obtained, and the mapping keywords corresponding to each of the second functions are determined.

In this embodiment, similar to the way of determining the function keywords, the electronic device may obtain all the second functions corresponding to the target database, perform keyword extraction, and determine the mapping keywords corresponding to each second function. The second function may specifically be a function template or a function example of the target database.

In S303, a mapping keyword matched with a function keyword is determined among all the mapping keywords, and a correspondence between a first function corresponding to the function keyword and a second function corresponding to the matched mapping keyword is established.

In this embodiment, after determining the function keywords and the mapping keywords, the electronic device may establish a corresponding relationship between different keywords, and since each function keyword is associated with a function corresponding to one database, if any two keywords have an associated relationship, it indicates that an associated relationship exists between the functions corresponding to the keywords, so that a code segment based on a certain function can be conveniently converted into a code segment in a function format conforming to the same function of another database.

In one possible implementation, functions with the same function are often represented using the same keywords. Based on this, the electronic device may establish a correspondence between function keywords and mapping keywords in which the same keywords exist. For example, the function key corresponding to the TRUNC function in the Oracle database is "TRUNC", and the function key corresponding to the TRUNC function in the PG database is "DATE _ TRUNC", and both include the same key word "TRUNC", so that it can be recognized that the DATE _ TRUNC function is a function in which the same function exists in the target database in the TRUNC function, and establish the correspondence between the DATE _ TRUNC and the TRUNC.

In a possible implementation manner, functions with the same function may use different keywords, but the different function keywords have the same or similar semantics, for example, the different keywords are expressed by using different languages or similar words, in which case, the electronic device may determine whether there is a translation or an alias relationship between the two function keywords through a preset language conversion algorithm and a semantic analysis algorithm; if yes, identifying that the two keywords have an association relationship, and taking the mapping keyword with the translation name or the alias relationship as the mapping keyword associated with the function keyword.

Further, as another embodiment of the present application, the determining, among all the mapping keywords, a mapping keyword matched with a function keyword, and establishing a correspondence between a first function corresponding to the function keyword and a second function corresponding to the matched mapping keyword includes:

in S303.1, calculating the associated confidence between the function keyword and each mapping keyword respectively; the associated confidence is:

wherein, SimiarLv is the correlation confidence coefficient between the function key words and the mapping key words; qst_xaFor the keyword attribute corresponding to the a-th function keyword, Qst_ybThe key word attribute corresponding to the b-th mapping key word; n is the total number of the keywords of the original database; m is the total number of the keywords of the target database;IDF is an inverse text probability calculation function; semta is a semantic similarity calculation function; alpha is a preset coefficient.

In this embodiment, after obtaining the function keywords corresponding to the first function and the mapping keywords corresponding to the second function, the electronic device may respectively match the mapping keywords corresponding to each second function with any function keyword, and calculate the association confidence between the two keywords. The two correlation confidence degrees are related to the semantic similarity between the two functions, and if the semantic similarity between the two correlation confidence degrees is higher, the probability that the two correlation confidence degrees have the same function is higher; conversely, if the semantic similarity between the two is smaller, the probability that the two have the same function is smaller. In addition to the semantic similarity correlation between the two, the probability correlation of the function key words with the inverse texts corresponding to all the appeared function key words in the original database is also included, namely the probability correlation of the function key words

If the probability of the reverse text is larger, the difference degree between the function key word and other function key words in the original database is smaller, and therefore the corresponding confidence coefficient is higher; on the contrary, if the difference degree between the function keyword and other function keywords in the original data is larger, the corresponding confidence coefficient is lower; similarly, the mapping keyword can also be determined in the above manner, so that the confidence of association between the two can be calculated according to the three types of parameters.

In this embodiment, if it is detected that the association confidence between any one of the mapping keywords and the function keyword in the original database is greater than the preset matching threshold, the electronic device identifies a first function corresponding to the function keyword, and a second function with the same function exists in the target database, and performs the operation of S303.2; on the contrary, if the correlation confidence between the mapping keyword and the function keyword in the original database does not exist in the target database, the similarity function is further searched for identification, and the operation of S303.3 is executed.

In S303.2, if the correlation confidence between any of the mapping keywords and the function keyword is greater than a preset matching threshold, the mapping keyword with the maximum correlation confidence is selected as the mapping keyword matched with the function keyword.

In this embodiment, if it is detected that the associated confidence between any one of the mapping keywords and the function keyword is greater than the matching threshold in the target database, the electronic device may use the second function corresponding to the mapping keyword with the highest associated confidence as the function having the same function as the first function corresponding to the function keyword, and thus may recognize the second function as the mapping keyword matched with the function keyword.

In the embodiment of the application, the accuracy of the association identification between the function keywords and the mapping keywords can be improved by calculating the association confidence coefficient between the function keywords and the mapping keywords and performing the index parameters during association calculation through semantics, inverse text probability and the like, so that the accuracy of the subsequent data conversion is improved.

In S303.3, if the associated confidence degrees between all the mapping keywords and the function keywords are less than or equal to the matching threshold, the function keywords are identified as the keywords to be matched.

In this embodiment, if it is detected by the electronic device that the association confidence between all the mapping keywords and the function keywords is too low, that is, the association confidence is less than or equal to the preset matching threshold, it may be that the function of the first function corresponding to the function keyword is implemented only after two or more second functions in the target database are combined. For example, the function key corresponding to the first function in the original database is "while", and the function of the function key is equivalent to the combination of the "if" function and the "while" function in the target database, that is, the function key needs to correspond to two mapping keys at the same time. To determine a similar situation to the above type, the electronic device may identify a function keyword for which there is no association as the keyword to be matched.

In S303.4, identifying a second function corresponding to a mapping keyword that is not associated with any function keyword in the target database as a function to be mapped, and combining all the functions to be mapped to generate at least one function group to be mapped; the set of functions to be mapped comprises more than two functions to be mapped.

In this embodiment, after the electronic device identifies the association confidence of all mapping keywords, it may determine all unassociated function keywords to be matched in the original database and mapping keywords of the unassociated function keywords in the target database, at this time, the electronic device may combine functions to be mapped corresponding to the mapping keywords of the unassociated function keywords, and generate one or more function groups to be mapped, where each function group to be mapped includes two or more functions to be mapped of the first function in the original database to be associated.

In S303.5, a preset training example is processed through the first function of the keyword to be matched to obtain a first processing result, and the training example is processed through the set of functions to be mapped to obtain a second processing result.

In this embodiment, the electronic device may store training examples corresponding to different functions, and process the training examples through a first function corresponding to the keyword to be matched, so as to obtain a corresponding first processing result.

It should be noted that the number of the training examples may be multiple, and the electronic device may respectively process each training example through the first function and the set of functions to be mapped, so as to obtain multiple first processing results and multiple second processing results.

In S303.6, according to the first processing result and the second processing result, a processing similarity between the first function of the keyword to be matched and the set of functions to be mapped is determined.

In this embodiment, after the electronic device processes the same training example through the first function and the to-be-processed function group, it may be determined whether the two processing results are the same, and the number of the first processing result is the same as that of the second processing result, and the processing similarity between the first function of the to-be-matched keyword and the to-be-mapped function group is calculated according to the same number and the total number of the training examples.

In S303.7, if the processing similarity between any one of the function groups to be mapped and the first function of the keyword to be matched is greater than a preset similarity threshold, establishing a corresponding relationship between the first function of the keyword to be matched and the function group to be mapped.

In this embodiment, if the processing similarity between any one of the function groups to be mapped and the first function of the keyword to be matched is greater than the similarity threshold, it indicates that the first function and the function group to be mapped have the same processing logic, that is, the combination of the plurality of second functions can implement the same function as the first function.

In the embodiment of the application, when the mapping keywords with the correlation confidence degrees larger than the correlation threshold do not exist, the functions with the same function are identified from the processing logic by combining the second functions of the plurality of mapping keywords and determining the processing similarity between the different first functions and the function groups to be matched, so that the accuracy of corresponding relation identification is improved.

In S304, a mapping relationship between the original database and the target database is generated according to all the corresponding relationships.

In this embodiment, after determining the corresponding relationship between the first function and the second function, the electronic device may encapsulate all the corresponding relationships, that is, the corresponding relationships are used as mapping relationships between the target database and the original database, and format conversion between code segments with the same function is realized through the mapping relationships.

In the embodiment of the application, the mapping relation between the two databases is generated based on all the corresponding relations by acquiring the first function of the original database and the second function of the target database to establish the corresponding relation between the same functions, so that the purpose of automatically establishing the mapping relation is realized.

Fig. 4 is a flowchart illustrating a specific implementation of a data conversion method according to a third embodiment of the present invention. Referring to fig. 4, with respect to the embodiment shown in fig. 2, before the searching for the mapping relationship between the original database and the target database and determining the mapping keywords corresponding to the function keywords respectively based on the mapping relationship, the method for data conversion according to this embodiment further includes: s401 to S404 are specifically detailed as follows:

in S401, if the mapping relationship is not found, a mapping establishment record between the original database and the target database is obtained.

In this embodiment, if the electronic device detects that the mapping relationship between the target database and the original database is not stored locally, there may be two cases, which are respectively:

in case 1, the mapping relationship of part of the keywords is constructed, but the mapping relationship of part of the keywords cannot be established, that is, the mapping is incomplete. In this case, the electronic device may be configured with a corresponding mapping establishment record, and the mapping establishment record includes the function pair whose existence is identified, and then the operation of S403 is performed.

And 2, the mapping relation between the two databases is not established. In this case, the electronic device does not store a mapping record between the two databases, and then performs the operation of S402.

In S402, if the mapping establishment record does not exist, a first function library of the original database and a second function library of the target database are obtained, and the mapping relationship is generated according to the first function library and the second function library.

In this embodiment, if the electronic device does not establish the mapping relationship between the two databases, there is no mapping establishment record related to the mapping between the two databases, and at this time, the electronic device may establish the mapping relationship between the two databases according to the function libraries corresponding to the two types of databases.

In S403, if the mapping establishment record exists, the mapping establishment record is analyzed, and an association function pair having an association relationship established is determined.

In this embodiment, if the electronic device has already tried to establish the mapping relationship between the two databases, but the electronic device still does not store the mapping relationship, it indicates that there is a function group for which the corresponding relationship cannot be established, and manual configuration is required.

In S404, mapping configuration information is generated according to all the association function pairs, so that a user establishes an association relationship between functions to be configured, so as to obtain a mapping relationship between the original database and the target database.

In this embodiment, the electronic device may add all the function pairs identified to obtain the association relationship to the mapping configuration information, and output the mapping configuration information, and the user may determine the function pairs to be associated and the associated function pairs in the two databases according to the mapping configuration information, and the user only needs to configure the function pairs to be configured.

In the embodiment of the application, the function required to be configured by the user is determined by generating the corresponding mapping configuration information, so that unnecessary work of the user is reduced, and the efficiency of data conversion can be improved.

Fig. 5 is a flowchart illustrating a specific implementation of a method for data conversion according to a fourth embodiment of the present invention. Referring to fig. 5, with respect to any one of the embodiments in fig. 2 to 4, after the generating migration code data about the target database based on all the migration code segments, the method for data conversion provided by this embodiment further includes: s501 to S503 are specifically detailed as follows:

further, after the generating migration code data about the target database based on all the migration code segments, the method further includes:

in S501, a plurality of verification data are obtained from the original database, and a reference result corresponding to each verification data processed by the original code data of the target document is determined.

In S502, each of the verification data is processed by migrating code data, respectively, to obtain an actual result.

In S503, an offset value corresponding to the data migration process is calculated according to all the reference results and the actual results.

In S504, if the deviation value is greater than a preset deviation threshold, migration failure information is generated.

In this embodiment, after the conversion is completed, the electronic device may perform conversion verification in order to determine whether the format of the converted data table is consistent with the original data format. The electronic equipment can extract a plurality of verification data from the original database, respectively determine a reference result corresponding to each verification data, locate data corresponding to the verification data in the target database, also calculate a transferred actual result, obtain corresponding converted power based on a deviation value between the actual result corresponding to the plurality of verification data and the reference result, identify that the conversion is failed if the converted power is lower than a preset deviation threshold value, calibrate the mapping relation, and convert again based on the calibrated mapping relation.

Optionally, if the target document includes a plurality of data tables, and the headers corresponding to different data tables are different, in this case, the conversion powers corresponding to different data tables may be calculated, for example, the target document includes a data table a and a data table B, the verification data is extracted from the data table a, the verification data is extracted from the data table B, the conversion power of the data table a and the conversion power of the data table B are calculated, and then only the abnormal mapping relation in a certain data table needs to be adjusted, and all mapping relations do not need to be adjusted, so that the adjustment efficiency is improved.

In the embodiment of the application, the corresponding conversion power is obtained after the conversion is finished, so that the accuracy of data format conversion can be improved.

Fig. 6 shows a flowchart of a specific implementation of the method S202 for data conversion according to the fifth embodiment of the present invention. Referring to fig. 6, with respect to any one of the embodiments shown in fig. 2 to 4, in the method for data conversion provided by this embodiment, S202 includes: s2021 to S2023 are specifically described as follows:

further, the obtaining, based on the document identifier, original code data of the target document from an original database, performing semantic parsing on the original code data, and marking a function keyword included in the original code data includes:

in S2021, a first function library of the raw database is obtained; the first function library comprises a plurality of preset function templates.

In S2022, the function keywords corresponding to the function templates are determined, and a function keyword dictionary is generated.

In S2023, each of the function keywords and the original code segment where the function keyword appears are marked within the original code data by the function keyword dictionary.

In this embodiment, when performing semantic analysis on original code data, the electronic device needs to construct a function key dictionary corresponding to the original database, where function keywords included in the function key dictionary are determined by a function included in a first function library corresponding to the original database. The first function library comprises a plurality of first functions, each first function corresponds to one function template, the electronic equipment can analyze the function template and determine the function key words contained in the function template, so that a function key dictionary corresponding to the threshold value is formed, and the original code data is identified through the dictionary to mark the function key words contained in the function template.

Fig. 7 is a block diagram illustrating a method and an apparatus for data conversion according to an embodiment of the present invention, where the server includes units for performing the steps implemented by the intermediate server in the embodiment corresponding to fig. 2. Please refer to fig. 2 and fig. 2 for the corresponding description of the embodiment. For convenience of explanation, only the portions related to the present embodiment are shown.

Referring to fig. 7, the method and apparatus for data conversion includes:

a migration request receiving unit 71, configured to receive a data migration request; the data migration request comprises a document identifier of a target document to be migrated and a database identifier of the target document to be expected to be migrated;

a function keyword labeling unit 72, configured to obtain original code data of the target document from an original database based on the document identifier, perform semantic analysis on the original code data, and label a function keyword included in the original code data;

a mapping keyword determining unit 73, configured to search a mapping relationship between the original database and the target database, and determine, based on the mapping relationship, mapping keywords corresponding to the function keywords respectively;

a migration code segment generating unit 74, configured to sequentially convert the original code segments corresponding to the function keywords into migration code segments corresponding to the mapping keywords;

a data migration unit 75 for generating migration code data regarding the target database based on all the migration code segments.

Optionally, the data conversion apparatus further includes:

a first function obtaining unit, configured to obtain all first functions in the original database, and determine the function key corresponding to each first function;

a second function obtaining unit, configured to obtain all second functions in the target database, and determine the mapping keyword corresponding to each second function;

the mapping keyword matching unit is used for matching the function keyword with the function keyword according to the mapping keyword;

and the mapping relation establishing unit is used for generating the mapping relation between the original database and the target database according to all the corresponding relations.

Optionally, the correspondence relationship establishing unit includes:

an association confidence calculation unit for calculating an association confidence between the function keyword and each of the mapping keywords, respectively; the associated confidence is:

wherein, SimiarLv is the correlation confidence coefficient between the function key words and the mapping key words; qst_xaFor the keyword attribute corresponding to the a-th function keyword, Qst_ybThe key word attribute corresponding to the b-th mapping key word; n is the total number of the keywords of the original database; m is the total number of the keywords of the target database; IDF is an inverse text probability calculation function; semta is a semantic similarity calculation function; alpha is a preset coefficient;

and the association identification unit is used for selecting the mapping keyword with the maximum association confidence as the mapping keyword matched with the function keyword if the association confidence between any mapping keyword and the function keyword is greater than a preset matching threshold.

Optionally, the data conversion apparatus further includes:

a to-be-associated response unit, configured to identify the function keyword as a to-be-matched keyword if the association confidence degrees between all the mapping keywords and the function keyword are less than or equal to the matching threshold;

a function group to be mapped determining unit, configured to identify a second function corresponding to a mapping keyword that is not associated with any function keyword in the target database as a function to be mapped, and combine all the functions to be mapped to generate at least one function group to be mapped; the set of functions to be mapped comprises more than two functions to be mapped;

the processing result generating unit is used for processing a preset training example through a first function of the keyword to be matched to obtain a first processing result, and processing the training example through the function group to be mapped to obtain a second processing result;

the processing similarity calculation unit is used for determining the processing similarity between the first function of the keyword to be matched and the function group to be mapped according to the first processing result and the second processing result;

and the function group association unit is used for establishing the corresponding relation between the first function of the keyword to be matched and the function group to be mapped if the processing similarity between any function group to be mapped and the first function of the keyword to be matched is greater than a preset similarity threshold.

Optionally, the data conversion apparatus further includes:

a mapping establishing record obtaining unit, configured to obtain a mapping establishing record between the original database and the target database if the mapping relationship is not queried;

a mapping establishment triggering unit, configured to, if the mapping establishment record does not exist, obtain a first function library of the original database and a second function library of the target database, and generate the mapping relationship according to the first function library and the second function library;

the association function pair determining unit is used for analyzing the mapping establishment record and determining an association function pair with an established association relationship if the mapping establishment record exists;

and the mapping configuration information generating unit is used for generating mapping configuration information according to all the association function pairs so that a user can establish an association relation between the functions to be configured to obtain the mapping relation between the original database and the target database.

Optionally, the data conversion apparatus further includes:

the verification data acquisition unit is used for acquiring a plurality of verification data from the original database and determining a corresponding reference result of each verification data after the verification data is processed by the original code data of the target document;

the actual result output unit is used for respectively processing each verification data through migrating code data to obtain an actual result;

the deviation value calculating unit is used for calculating a deviation value corresponding to the data migration process according to all the reference results and the actual results;

and the migration failure information prompting unit is used for generating migration failure information if the deviation value is greater than a preset deviation threshold value.

Optionally, the function keyword labeling unit 72 includes:

a first function library acquisition unit configured to acquire a first function library of the original database; the first function library comprises a plurality of preset function templates;

a key dictionary generating unit, configured to determine the function keywords corresponding to the function templates, respectively, and generate a function key dictionary;

and the keyword recognition unit is used for marking each function keyword in the original code data and the original code segment with the function keyword through the function keyword dictionary.

Therefore, the method and the device for data conversion provided by the embodiment of the invention can also obtain the original code data of the target document to be migrated from the original database when data migration is needed, and determine the function key words contained in the original code data; and respectively determining mapping keywords associated with each function keyword through the mapping relation between the original database and a target database to be migrated, automatically translating original code segments containing the function keywords into migration code segments corresponding to the mapping keywords, realizing format conversion of the code segments among the same functions, obtaining corresponding migration code segments for all code segments in a target document in the manner, packaging all the migration code segments to obtain migration code data conforming to the target database, and realizing the purpose of migrating the target document from one database to another database. Compared with the existing data management technology, the data management method and the data management system have the advantages that the data table is not re-established in the target database to be migrated by the user, the data items are led in one by one, the mapping relation among different databases can be established, the corresponding code segments are subjected to format conversion by identifying the function keywords with the mapping relation, the data migration efficiency is greatly improved, and meanwhile, the labor cost is reduced.

It should be understood that, in the structural block diagram of the method and apparatus for data conversion shown in fig. 7, each module is used to execute each step in the embodiment corresponding to fig. 2 to 6, and each step in the embodiment corresponding to fig. 2 to 6 has been explained in detail in the above embodiment, and specific reference is made to the relevant description in the embodiment corresponding to fig. 2 to 6 and fig. 2 to 6, which is not repeated herein.

Fig. 8 is a block diagram of a server according to another embodiment of the present application. As shown in fig. 8, the server 800 of this embodiment includes: a processor 810, a memory 820 and a computer program 830, e.g. a program of a method of data conversion, stored in the memory 820 and executable on the processor 810. The processor 810, when executing the computer program 830, implements the steps in the various embodiments of the method for data transformation described above, such as S201 to S205 shown in fig. 2. Alternatively, the processor 810, when executing the computer program 830, implements the functions of the modules in the embodiment corresponding to fig. 8, for example, the functions of the units 71 to 75 shown in fig. 7, and refer to the related description in the embodiment corresponding to fig. 7 specifically.

Illustratively, the computer program 830 may be partitioned into one or more modules, which are stored in the memory 820 and executed by the processor 810 to accomplish the present application. One or more of the modules may be a series of computer program instruction segments capable of performing certain functions that are used to describe the execution of computer program 830 in server 800. For example, the computer program 830 may be divided into unit modules, each of which functions as described above.

The server 800 may include, but is not limited to, a processor 810, a memory 820. Those skilled in the art will appreciate that fig. 8 is merely an example of a server 800, and does not constitute a limitation of server 800, and may include more or fewer components than shown, or some components in combination, or different components, e.g., the server may also include input-output devices, network access devices, buses, etc.

The processor 810 may be a central processing unit, but may also be other general purpose processors, digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or any conventional processor or the like.

The storage 820 may be an internal storage unit of the server 800, such as a hard disk or a memory of the server 800. The memory 820 may also be an external storage device of the server 800, such as a plug-in hard disk, a smart card, a flash memory card, etc. provided on the server 800. Further, the memory 820 may also include both internal storage units of the server 800 and external storage devices.

The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A method of data conversion, comprising:

receiving a data migration request; the data migration request comprises a document identifier of a target document to be migrated and a database identifier of a target database to which the target document is expected to be migrated;

2. The method according to claim 1, before the searching for the mapping relationship between the original database and the target database, and respectively determining the mapping keywords corresponding to the function keywords based on the mapping relationship, further comprising:

3. The method according to claim 2, wherein the determining, among all the mapping keywords, a mapping keyword matching with a function keyword, and establishing a correspondence between a first function corresponding to the function keyword and a second function corresponding to the matching mapping keyword comprises:

4. The method according to claim 3, further comprising, after said respectively calculating the associated confidence between the function keyword and each of the mapping keywords:

5. The method according to claim 1, before the searching for the mapping relationship between the original database and the target database, and respectively determining the mapping keywords corresponding to the function keywords based on the mapping relationship, further comprising:

6. The method according to any of claims 1-5, further comprising, after said generating migration code data about the target database based on all migration code segments:

7. The method according to any one of claims 1-5, wherein the obtaining of the original code data of the target document from the original database based on the document identification, performing semantic parsing on the original code data, and marking the function keywords contained in the original code data comprises:

8. An apparatus for data conversion, comprising:

9. A server comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.