CN116204587A - Data synchronization task generation method, device and computer readable storage medium - Google Patents
Data synchronization task generation method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN116204587A CN116204587A CN202310155859.9A CN202310155859A CN116204587A CN 116204587 A CN116204587 A CN 116204587A CN 202310155859 A CN202310155859 A CN 202310155859A CN 116204587 A CN116204587 A CN 116204587A
- Authority
- CN
- China
- Prior art keywords
- data
- synchronized
- task
- database
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000001360 synchronised effect Effects 0.000 claims abstract description 178
- 238000004590 computer program Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 210000001072 colon Anatomy 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application provides a data synchronization task generation method, a data synchronization task generation device and a computer readable storage medium, wherein the method comprises the following steps: acquiring data information of a source database to be synchronized currently and data information of a target database to be synchronized currently; acquiring a task creation template; the task creation template comprises step execution information of each step required for creating the data synchronization task; and automatically generating a data synchronization target task according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creation template, thereby reducing the labor investment and improving the efficiency of creating the data synchronization task.
Description
Technical Field
The present invention relates to the field of data synchronization technologies, and in particular, to a method and an apparatus for generating a data synchronization task, and a computer readable storage medium.
Background
With the rapid increment of the application of informatization high-speed development of big data, the Internet of things, cloud computing and the like, the volume of data is increased explosively. The digital construction in the enterprise breaks the information island, and the data synchronization is converged to the data center for wide application, so that the data synchronization task is used in a large amount.
The existing data synchronization tasks are all created manually by a user, and the creation of the data synchronization tasks comprises a plurality of steps and configurations, so that the manual creation takes a lot of time, and the creation efficiency of the data synchronization tasks is low.
Disclosure of Invention
An object of the embodiments of the present application is to provide a method, an apparatus, and a computer readable storage medium for generating a data synchronization task, so as to solve the problem of low creation efficiency existing in the current data synchronization task created manually by a person.
In a first aspect, the present invention provides a method for generating a data synchronization task, the method comprising: acquiring data information of a source database to be synchronized currently and data information of a target database to be synchronized currently; acquiring a task creation template; the task creation template comprises step execution information of each step required for creating the data synchronization task; and generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template.
According to the data synchronization task generation method, the task creation template is created in advance and comprises step execution information of all steps required for creating the data synchronization task, then when the data synchronization task is generated, the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creation template are obtained, so that the data synchronization target task is automatically generated according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creation template, and the labor investment is reduced, and the efficiency of creating the data synchronization task is improved.
In an optional implementation manner of the first aspect, the step execution information includes source database data and target database data required for execution; generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template, wherein the data synchronization target task comprises the following steps: configuring source database data required by step execution in a task creation template as data information of a source database to be synchronized currently, and configuring target database data required by step execution in the task creation template as data information of a target database to be synchronized currently, so as to obtain a synchronous execution file; and generating a data synchronization target task according to the synchronization execution file.
In an optional implementation manner of the first aspect, the source database data and the target database data required for execution in the step execution information are preset parameter variables; configuring source database data required by step execution in a task creation template as data information of the source database to be synchronized currently, and configuring target database data required by step execution in the task creation template as data information of the target database to be synchronized currently, wherein the method comprises the following steps: replacing preset parameter variables of source database data required by step execution by utilizing the data information of the source database to be synchronized currently; and replacing the preset parameter variables of the target database data required by the step execution by utilizing the data information of the target database to be synchronized at present to obtain a synchronous execution file.
In an optional implementation manner of the first aspect, the data information of the source database to be synchronized currently and the data information of the target database to be synchronized currently each include a plurality of data; replacing preset parameter variables of source database data required by step execution by utilizing the data information of the source database to be synchronized currently; and replacing the preset parameter variables of the target database data required by the step execution by utilizing the data information of the target database to be synchronized currently, wherein the method comprises the following steps: according to each data of the source database to be synchronized currently and each data of the target database to be synchronized currently, searching corresponding preset parameter variables in the step execution information, wherein the preset parameter variables corresponding to different data are different; and replacing the searched corresponding preset parameter variable by each data to obtain the synchronous execution file.
In an optional implementation manner of the first aspect, before configuring the source database data required for performing the step in the task creation template as the data information of the source database to be currently synchronized, the method further includes: judging whether the task creation template is a template file in a preset format; if the task creation template is judged to be a template file in a preset format, judging whether step execution information of each step in the task creation template is complete or not; and if the step execution information of each step in the task creation template is complete, executing the step of configuring the source database data required by step execution in the task creation template as the data information of the source database to be synchronized currently.
In an optional implementation manner of the first aspect, before acquiring the data information of the source database to be currently synchronized and the data information of the target database to be currently synchronized, the method further includes: creating a task creation template and storing the created task creation template.
In an alternative implementation of the first aspect, creating the task creation template includes: acquiring step execution information of each step required for creating a data synchronization task; integrating each step required for creating the data synchronization task and the step execution information of each step into one file, and setting corresponding preset parameter variables for the source database data and the target database data required in the step execution information of each step to create the task creation template.
In an optional implementation manner of the first aspect, the source databases to be synchronized include a plurality of source databases to be synchronized, and each source database to be synchronized corresponds to one target database to be synchronized; acquiring data information of a source database to be synchronized currently and data information of a target database to be synchronized currently, including: acquiring data information of each current source database to be synchronized and data information of a target database to be synchronized corresponding to each current source database to be synchronized; generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template, wherein the data synchronization target task comprises the following steps: and generating a corresponding data synchronization target task according to the data information of each current source database to be synchronized, the data information of the target database corresponding to the current source database to be synchronized and the task creation template.
In a second aspect, the present invention provides a data synchronization task generating device, the device comprising: the device comprises an acquisition module and a generation module; the acquisition module is used for acquiring the data information of the source database to be synchronized currently and the data information of the target database to be synchronized currently; acquiring a task creation template; the task creation template comprises step execution information of each step required for creating the data synchronization task; the generation module is used for generating a data synchronization target task according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creation template.
According to the data synchronization task generating device, the task creating template is pre-created and comprises step execution information of all steps required for creating the data synchronization task, then when the data synchronization task is generated, the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creating template are obtained, so that the data synchronization target task is automatically generated according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creating template, and the labor investment is reduced and the efficiency of creating the data synchronization task is improved.
In an optional implementation manner of the second aspect, the step execution information includes source database data and target database data required for execution; the generation module is specifically used for configuring source database data required by step execution in a task creation template as data information of a source database to be synchronized currently, and configuring target database data required by step execution in the task creation template as data information of a target database to be synchronized currently, so as to obtain a synchronous execution file; and generating a data synchronization target task according to the synchronization execution file.
In an optional implementation manner of the second aspect, source database data and target database data required for execution in the step execution information are preset parameter variables; the generation module is also specifically configured to replace a preset parameter variable of source database data required for executing the step by using data information of the source database to be synchronized currently; and replacing the preset parameter variables of the target database data required by the step execution by utilizing the data information of the target database to be synchronized at present to obtain a synchronous execution file.
In an optional implementation manner of the second aspect, the data information of the current to-be-synchronized source database and the data information of the current to-be-synchronized target database each include a plurality of data, and the generating module is further specifically configured to execute corresponding preset parameter variables in the information according to each data of the current to-be-synchronized source database and each data searching step of the current to-be-synchronized target database, where preset parameter variables corresponding to different data are different; and replacing the searched corresponding preset parameter variable by each data to obtain the synchronous execution file.
In an optional implementation manner of the second aspect, the apparatus further includes a creation storage module configured to create a task creation template, and store the created task creation template.
In an optional implementation manner of the second aspect, the creation storage module is specifically configured to obtain each step required for creating the data synchronization task and step execution information of each step; integrating each step required for creating the data synchronization task and the step execution information of each step into one file, and setting corresponding preset parameter variables for the source database data and the target database data required in the step execution information of each step to create the task creation template.
In an optional implementation manner of the second aspect, the source databases to be synchronized include a plurality of source databases to be synchronized, and each source database to be synchronized corresponds to one target database to be synchronized; the acquisition module is specifically configured to acquire data information of each current source database to be synchronized and data information of a target database to be synchronized corresponding to each current source database to be synchronized; the generating module is further specifically configured to generate a corresponding data synchronization target task according to the data information of each current source database to be synchronized, the data information of the target database corresponding to the current source database to be synchronized, and the task creating template.
In a third aspect, the present application provides an electronic device comprising a memory storing a computer program and a processor that when executing the computer program performs the method of any of the alternative implementations of the first aspect.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method of any of the alternative implementations of the first aspect.
In a fifth aspect, the present application provides a computer program product which, when run on a computer, causes the computer to perform the method of any one of the alternative implementations of the first aspect.
The foregoing description is only an overview of the technical solutions of the present application, and may be implemented according to the content of the specification in order to make the technical means of the present application more clearly understood, and in order to make the above-mentioned and other objects, features and advantages of the present application more clearly understood, the following detailed description of the present application will be given.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a first flowchart of a data synchronization task generating method provided in an embodiment of the present application;
fig. 2 is a second flowchart of a data synchronization task generating method provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a data synchronization task generating device according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Icon: 300-an acquisition module; 310-a generation module; 320-creating a storage module; 4-an electronic device; 401-a processor; 402-memory; 403-communication bus.
Detailed Description
Embodiments of the technical solutions of the present application will be described in detail below with reference to the accompanying drawings. The following examples are only for more clearly illustrating the technical solutions of the present application, and thus are only examples, and are not intended to limit the scope of protection of the present application.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions.
In the description of the embodiments of the present application, the technical terms "first," "second," etc. are used merely to distinguish between different objects and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, a particular order or a primary or secondary relationship. In the description of the embodiments of the present application, the meaning of "plurality" is two or more unless explicitly defined otherwise.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In the description of the embodiments of the present application, the term "and/or" is merely an association relationship describing an association object, which means that three relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
In the description of the embodiments of the present application, the term "plurality" refers to two or more (including two), and similarly, "plural sets" refers to two or more (including two), and "plural sheets" refers to two or more (including two).
In the description of the embodiments of the present application, the orientation or positional relationship indicated by the technical terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. are based on the orientation or positional relationship shown in the drawings, and are merely for convenience of describing the embodiments of the present application and for simplifying the description, rather than indicating or implying that the apparatus or element referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the embodiments of the present application.
In the description of the embodiments of the present application, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured" and the like are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally formed; or may be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the embodiments of the present application will be understood by those of ordinary skill in the art according to the specific circumstances.
With the rapid increment of the application of informatization high-speed development of big data, the Internet of things, cloud computing and the like, the volume of data is increased explosively. The digital construction in the enterprise breaks the information island, and the data synchronization is converged to the data center for wide application, so that the data synchronization task is used in a large amount.
The inventor notes that the creation of the data synchronization task is currently done manually by a user, and the manual creation of the data synchronization task requires configuration of execution information required for each step of data synchronization, such as data cleansing, conversion components, and the like, which takes a lot of time, thereby making the creation of the data synchronization task inefficient. In addition, in order to increase the creation speed, there are cases where tasks are created by multiple users at the same time, but the simultaneous creation of database lock tables by multiple users may result in partial operations or task loss.
In this regard, the present inventors devised a data synchronization task generating method, apparatus, and computer-readable storage medium, which pre-formulates a task creation template, and then automatically generates a data synchronization target task based on the task creation template, data information of a source database to be synchronized, and data information of a target database to be synchronized, thereby reducing human input and improving efficiency of data synchronization task creation; meanwhile, the problem of locking the table during the creation of the data synchronization task is avoided.
Based on the above-mentioned idea, the present application provides a data synchronization task generating method, which can be applied to a computing device, where the computing device includes, but is not limited to, a computer, a server, a chip, a controller, and the like, and as shown in fig. 1, the data synchronization task generating method can be implemented by:
step S100: and acquiring the data information of the source database to be synchronized currently and the data information of the target database to be synchronized currently.
Step S110: a task creation template is obtained.
Step S120: and generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template.
In the above embodiment, the data synchronization task is a task of synchronizing data in the source database to the target database, and the data information of the current source database to be synchronized may include various data information of the source database to be synchronized, for example, may be information such as a table name, a table chinese name, a field, a data type, a data length, a primary key, and a source database connection name of the source database to be synchronized; the data information of the target database currently to be synchronized may include a target database connection name.
As a possible implementation, the data information of the database to be currently synchronized and the data information of the target database to be currently synchronized may be obtained through a user's current implementation input or configuration.
As another possible implementation manner, the user may configure and store the data information of the source database to be synchronized and the database information of the target database to be synchronized in advance in the configuration file or the database, and on this basis, when the data synchronization needs to be created for the source database and the target database, the stored configuration file or database is automatically read by the program, so as to obtain the data information of the database to be synchronized currently and the data information of the target database to be synchronized currently.
In this embodiment, the task creation template may include step execution information of each step required for creating the data synchronization task, and in particular, the step execution information may include source database data and target database data required for execution, and may further include each component required for creation of the data synchronization task. The task creation module may be created and stored in advance, and is obtained by calling the task creation module when executing step S110.
As one possible implementation, as shown in fig. 2, the task creation module may create by:
step S200: and acquiring the step execution information of each step required for creating the data synchronization task.
Step S210: integrating each step required for creating the data synchronization task and the step execution information of each step into one file, and setting corresponding preset parameter variables for the source database data and the target database data required in the step execution information of each step to create the task creation template.
In this embodiment, since different data synchronization tasks are basically the same flow, and the difference is that the data information of the source database and the data information of the target database are different, the invariant component required for generating the data synchronization task is used as a template, the corresponding variable generated for the data synchronization task is set as the corresponding preset parameter variable, and further the automatic generation of the data synchronization task can be realized by automatically replacing or filling the preset parameter variable by the data of the source database and the data of the target database to be synchronized.
The above embodiments may be specifically illustrated by the following examples, for example, the various steps required to create a data synchronization task may include: (1) creating database connection information: and connecting the source database and the target database. (2) The selection table input component configures source database connections, tables, field information. (3) The necessary cleaning components required for data synchronization are selected. (4) The select table output component configures the target database connection, table, and field mapping information.
When the template is created, the steps are acquired, then the table input in the step (2) is parameterized, namely, the source database connection information, the table, the field information and the like are set as corresponding preset parameter variables, the table output in the step (4) is parameterized, namely, the target database connection, the table, the field information and the like are set as corresponding preset parameter variables, and therefore, the steps are stored in one file, and the task creation template can be generated. The purpose of setting the preset parameter variables is to set the data of the source database and the data of the target database required in the task creation template to be in a state to be filled or replaced, so that filling or replacement can be performed during application, and when different data synchronization tasks are generated for different source databases and different target databases, only the preset parameter variables in the template need to be filled or replaced by utilizing the data information of the source database and the data information of the target database to be synchronized.
It should be noted that, the step S100 of acquiring the data information of the source database to be synchronized, the data information of the target database to be synchronized, and the acquisition task creation template of step S110 may be performed simultaneously, for example, the scheme may simultaneously read the data information of the source database to be synchronized, the configuration file corresponding to the data information of the target database to be synchronized, and the stored task creation template by configuring the related processes. Of course, the step S100 and the step S110 of the present embodiment may be performed based on a sequential order, which is not particularly limited in the present application.
After the data information of the source database to be synchronized, the data information of the target database to be synchronized and the task creation template are obtained in the above manner, the scheme can generate the data synchronization target task according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creation template.
As a possible implementation manner, the foregoing describes that the task creation template includes source database data and target database data required for execution, based on which, the solution may configure the source database data required for step execution in the task creation template as data information of a source database to be currently synchronized, and configure the target database data required for step execution in the task creation template as data information of a target database to be currently synchronized, so as to obtain a synchronous execution file; and generating a data synchronization target task according to the synchronization execution file.
Specifically, the foregoing describes that, the data of the source database and the data of the target database required in the task creation template are both preset parameter variables, and on the basis, the scheme can replace the preset parameter variables of the source database data required by step execution according to the data information of the source database to be synchronized currently; and replacing the preset parameter variables of the target database data required by the step execution by utilizing the data information of the target database to be synchronized, thereby obtaining the synchronous execution file. For example, the database type of the source database required in the task creation template is "database type: { type } ", wherein { type } is a preset parameter variable described above, and the type of the source database to be synchronized in the data information of the source database to be synchronized is MySQL, then the present solution replaces the preset parameter variable { type } with MySQL, so that the database type of the source database required in the generated synchronization execution file is" database type: mySQL). For another example, a target database connection for a desired target database in the task creation template is "target database connection: { connect } ", and the connection of the target database to be synchronized in the data information of the target database to be synchronized is" http:// www.a.com/1.Jpg ", then the preset parameter variable { connect } is replaced by" http:// www.a.com/1.Jpg "in this scheme, so that the connection of the target database to be synchronized required in the generated synchronization execution file is" target database connection ": http:// www.a.com/1. Jpg).
In an alternative implementation manner of this embodiment, the data synchronized by the data synchronization is generally a plurality of data, on the basis of which, preset parameter variables set in the task creation template may be different according to different corresponding data, for example, the preset parameter variable corresponding to the database type exemplified above is { type }, and the preset parameter variable corresponding to the database connection is { connect }; further, in order to distinguish the types and the connection of the source database and the target database, the preset parameter variable of the type of the source database may be specifically set to { type1}, and the preset parameter variable of the target database may be set to { type2}, thereby realizing the distinction. On the basis, when the scheme is replaced, the corresponding preset parameter variables in the execution information of each data of the current to-be-synchronized source database and each data searching step of the current to-be-synchronized target database can be firstly searched, for example, if the data of the current to-be-synchronized source database is the type of the source database, the { type1} is automatically searched in the task creating module, so that the replacement position is determined, and the type of the current to-be-synchronized source database is replaced by the { type1 }.
And searching and replacing each data of the current source database to be synchronized and each data of the current target database to be synchronized in the mode, so as to obtain a synchronous execution file, and further converting the synchronous execution file to generate a data synchronous target task.
In addition, it should be noted that, in addition to the above alternative manner, the present solution may also adopt a data filling manner, that is, a manner of setting a preset parameter variable in the task creation template to be in a state to be filled, so that the present solution is realized by a manner of filling blank data. For example, the database type of the source database required in the task creation template exemplified above is "database type: "then based on the type of source database to be synchronized as MySQL versus" database type: filling, and then obtaining the database type: mySQL). Different states to be filled can be identified through the text part in the front of the colon.
As a possible implementation manner, in order to ensure the integrity and format of the task creation template, the solution may determine whether the task creation template is a template file in a preset format, for example, when the task creation template is created, an XML format is used for creating and storing, then the solution determines whether the task creation template is a template file in the XML format, if yes, it continues to determine whether the step execution information of each step in the task creation template is completed, if yes, it performs a step of configuring the source database data required for step execution in the task creation template as the data information of the source database to be currently synchronized; if the task creation template is not the XML-format template file or the step execution information is incomplete, the data synchronization task creation is not performed.
In addition to determining the task creation template, the method can also determine the information integrity of the synchronous execution file after generating the synchronous execution file, so as to ensure the data integrity of the generated synchronous execution file.
According to the data synchronization task generation method, the task creation template is created in advance and comprises step execution information of all steps required for creating the data synchronization task, then when the data synchronization task is generated, the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creation template are obtained, so that the data synchronization target task is automatically generated according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creation template, and the labor investment is reduced, and the efficiency of creating the data synchronization task is improved.
In an optional implementation manner of this embodiment, under a condition that a processing capability is relatively strong, the scheme may also generate a plurality of data synchronization tasks at the same time, specifically, the source database to be synchronized includes a plurality of source databases to be synchronized, each source database to be synchronized corresponds to a target database to be synchronized, on this basis, the scheme may acquire data information of each current source database to be synchronized and data information of each target database to be synchronized corresponding to each current source database to be synchronized, and then acquire a task creation template, on this basis, the scheme may create a plurality of data synchronization task generation processes, each data synchronization task generation process generates a corresponding data synchronization target task according to data information of one current source database to be synchronized, data information of a target database corresponding to the current source database to be synchronized, and a task creation template, thereby further improving efficiency of data synchronization task creation by multiple processes.
Fig. 3 shows a schematic block diagram of a data synchronization task generating device provided in the present application, and it should be understood that the device corresponds to the embodiment of the method performed in fig. 1-2, and is capable of performing the steps involved in the foregoing method, and specific functions of the device may be referred to in the foregoing description, and detailed descriptions thereof are omitted herein as appropriate to avoid redundancy. The device includes at least one software functional module that can be stored in memory in the form of software or firmware (firmware) or cured in an Operating System (OS) of the device. Specifically, the device comprises: an acquisition module 300 and a generation module 310; the acquiring module 300 is configured to acquire data information of a source database to be currently synchronized and data information of a target database to be currently synchronized; acquiring a task creation template; the task creation template comprises step execution information of each step required for creating the data synchronization task; the generating module 310 is configured to generate a data synchronization target task according to the data information of the source database to be synchronized currently, the data information of the target database to be synchronized currently, and the task creation template.
According to the data synchronization task generating device, the task creating template is pre-created and comprises step execution information of all steps required for creating the data synchronization task, then when the data synchronization task is generated, the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creating template are obtained, so that the data synchronization target task is automatically generated according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creating template, and the labor investment is reduced and the efficiency of creating the data synchronization task is improved.
In an alternative implementation manner of the present embodiment, the step execution information includes source database data and target database data required for execution; the generating module 310 is specifically configured to configure source database data required by step execution in a task creation template as data information of a source database to be currently synchronized, and configure target database data required by step execution in the task creation template as data information of a target database to be currently synchronized, so as to obtain a synchronization execution file; and generating a data synchronization target task according to the synchronization execution file.
In an optional implementation manner of this embodiment, source database data and target database data required for execution in the step execution information are preset parameter variables; the generating module 310 is further specifically configured to replace a preset parameter variable of the source database data required for executing the step with the data information of the source database to be synchronized; and replacing the preset parameter variables of the target database data required by the step execution by utilizing the data information of the target database to be synchronized at present to obtain a synchronous execution file.
In an optional implementation manner of this embodiment, the data information of the current to-be-synchronized source database and the data information of the current to-be-synchronized target database each include a plurality of data, and the generating module 310 is further specifically configured to execute corresponding preset parameter variables in the information according to each data of the current to-be-synchronized source database and each data searching step of the current to-be-synchronized target database, where preset parameter variables corresponding to different data are different; and replacing the searched corresponding preset parameter variable by each data to obtain the synchronous execution file.
In an optional implementation manner of this embodiment, the apparatus further includes a creation storage module 320, configured to create a task creation template, and store the created task creation template.
In an optional implementation manner of this embodiment, the creation storage module 320 is specifically configured to obtain each step required for creating the data synchronization task and step execution information of each step; integrating each step required for creating the data synchronization task and the step execution information of each step into one file, and setting corresponding preset parameter variables for the source database data and the target database data required in the step execution information of each step to create the task creation template.
In an optional implementation manner of this embodiment, the source databases to be synchronized include a plurality of source databases to be synchronized, and each source database to be synchronized corresponds to one target database to be synchronized; the acquiring module 300 is specifically configured to acquire data information of each current to-be-synchronized source database and data information of a to-be-synchronized target database corresponding to each current to-be-synchronized source database; the generating module 310 is further specifically configured to generate a corresponding data synchronization target task according to the data information of each current source database to be synchronized, the data information of the target database corresponding to the current source database to be synchronized, and the task creation template.
According to some embodiments of the present application, as shown in fig. 4, the present application provides an electronic device 4, including: a processor 401 and a memory 402, the processor 401 and the memory 402 being interconnected and communicating with each other by a communication bus 403 and/or other form of connection mechanism (not shown), the memory 402 storing a computer program executable by the processor 401, which when run by a computing device, is executed by the processor 401 to perform the methods performed in the aforementioned implementations, e.g. steps S100 to S120: acquiring data information of a source database to be synchronized currently and data information of a target database to be synchronized currently; acquiring a task creation template; and generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template.
The present application provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the method of the preceding implementation.
The storage medium may be implemented by any type of volatile or nonvolatile Memory device or combination thereof, such as static random access Memory (Static Random Access Memory, SRAM), electrically erasable Programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), erasable Programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk.
The present application provides a computer program product which, when run on a computer, causes the computer to perform the aforementioned method.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the embodiments, and are intended to be included within the scope of the claims and description. In particular, the technical features mentioned in the respective embodiments may be combined in any manner as long as there is no structural conflict. The present application is not limited to the specific embodiments disclosed herein, but encompasses all technical solutions falling within the scope of the claims.
Claims (10)
1. A method for generating a data synchronization task, the method comprising:
acquiring data information of a source database to be synchronized currently and data information of a target database to be synchronized currently;
Acquiring a task creation template; the task creation template comprises step execution information of each step required for creating the data synchronization task;
and generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template.
2. The method of claim 1, wherein the step execution information includes source database data and target database data required for execution;
the generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template comprises the following steps:
configuring source database data required by step execution in a task creation template as data information of the source database to be synchronized currently, and configuring target database data required by step execution in the task creation template as data information of the target database to be synchronized currently, so as to obtain a synchronous execution file;
and generating the data synchronization target task according to the synchronization execution file.
3. The method according to claim 2, wherein the source database data and the target database data required for execution in the step execution information are both preset parameter variables;
The configuring the source database data required by the step execution in the task creation template as the data information of the source database to be synchronized currently, and configuring the target database data required by the step execution in the task creation template as the data information of the target database to be synchronized currently includes:
replacing preset parameter variables of source database data required by step execution by utilizing the data information of the source database to be synchronized currently;
and replacing preset parameter variables of target database data required by step execution by utilizing the data information of the target database to be synchronized currently, so as to obtain the synchronous execution file.
4. The method of claim 2, wherein the data information of the source database to be currently synchronized and the data information of the target database to be currently synchronized each include a plurality of data;
replacing preset parameter variables of source database data required by step execution by utilizing the data information of the source database to be synchronized currently; and replacing the preset parameter variables of the target database data required by the step execution by utilizing the data information of the target database to be synchronized currently, wherein the method comprises the following steps:
According to each data of the source database to be synchronized currently and each data of the target database to be synchronized currently, searching corresponding preset parameter variables in the step execution information, wherein the preset parameter variables corresponding to different data are different;
and replacing the searched corresponding preset parameter variable by each data to obtain the synchronous execution file.
5. The method of claim 2, wherein prior to said configuring source database data required for step execution in the task creation template as data information of the source database currently to be synchronized, the method further comprises:
judging whether the task creation template is a template file in a preset format or not;
if the task creation template is judged to be a template file in a preset format, judging whether step execution information of each step in the task creation template is complete or not;
and if the step execution information of each step in the task creation template is complete, executing the step of configuring the source database data required by step execution in the task creation template as the data information of the source database to be synchronized currently.
6. The method of claim 1, wherein prior to the obtaining the data information of the source database to be currently synchronized and the data information of the target database to be currently synchronized, the method further comprises:
And creating the task creation template and storing the created task creation template.
7. The method of claim 6, wherein creating the task creation template comprises:
acquiring step execution information of each step required for creating a data synchronization task;
integrating each step required for creating the data synchronization task and the step execution information of each step into a file, and setting corresponding preset parameter variables for the source database data and the target database data required in the step execution information of each step to create the task creation template.
8. The method of claim 1, wherein the source databases to be synchronized include a plurality of source databases to be synchronized, each corresponding to a target database to be synchronized;
the obtaining the data information of the current to-be-synchronized source database and the data information of the current to-be-synchronized target database includes: acquiring data information of each current source database to be synchronized and data information of a target database to be synchronized corresponding to each current source database to be synchronized;
The generating a data synchronization target task according to the data information of the current to-be-synchronized source database, the data information of the current to-be-synchronized target database and the task creation template comprises the following steps: and generating a corresponding data synchronization target task according to the data information of each current source database to be synchronized, the data information of the target database corresponding to the current source database to be synchronized and the task creation template.
9. A data synchronization task generating device, the device comprising: the device comprises an acquisition module and a generation module;
the acquisition module is used for acquiring the data information of the source database to be synchronized currently and the data information of the target database to be synchronized currently; acquiring a task creation template; the task creation template comprises step execution information of each step required for creating the data synchronization task;
the generating module is used for generating a data synchronization target task according to the data information of the current source database to be synchronized, the data information of the current target database to be synchronized and the task creating template.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310155859.9A CN116204587B (en) | 2023-02-21 | 2023-02-21 | Data synchronization task generation method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310155859.9A CN116204587B (en) | 2023-02-21 | 2023-02-21 | Data synchronization task generation method, device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116204587A true CN116204587A (en) | 2023-06-02 |
| CN116204587B CN116204587B (en) | 2024-01-30 |
Family
ID=86507352
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310155859.9A Active CN116204587B (en) | 2023-02-21 | 2023-02-21 | Data synchronization task generation method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116204587B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116578650A (en) * | 2023-07-11 | 2023-08-11 | 太平金融科技服务（上海）有限公司 | Data synchronization method, device, computer equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017114199A1 (en) * | 2015-12-31 | 2017-07-06 | 阿里巴巴集团控股有限公司 | Data synchronisation method and apparatus |
| CN112148788A (en) * | 2020-08-25 | 2020-12-29 | 珠海市卓轩科技有限公司 | Data synchronization method and system for heterogeneous data source |
| CN113076365A (en) * | 2021-04-07 | 2021-07-06 | 杭州数梦工场科技有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113449031A (en) * | 2021-04-14 | 2021-09-28 | 上海漫微信息技术有限公司 | Data synchronization management method and device and server |
| CN114547206A (en) * | 2022-03-07 | 2022-05-27 | 牧原食品股份有限公司 | Data synchronization method and data synchronization system |
| CN114780641A (en) * | 2022-05-07 | 2022-07-22 | 湖南长银五八消费金融股份有限公司 | Multi-library multi-table synchronization method and device, computer equipment and storage medium |
| CN115146000A (en) * | 2022-06-20 | 2022-10-04 | 彩讯科技股份有限公司 | Database data synchronization method and device, electronic equipment and storage medium |
-
2023
- 2023-02-21 CN CN202310155859.9A patent/CN116204587B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017114199A1 (en) * | 2015-12-31 | 2017-07-06 | 阿里巴巴集团控股有限公司 | Data synchronisation method and apparatus |
| CN112148788A (en) * | 2020-08-25 | 2020-12-29 | 珠海市卓轩科技有限公司 | Data synchronization method and system for heterogeneous data source |
| CN113076365A (en) * | 2021-04-07 | 2021-07-06 | 杭州数梦工场科技有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113449031A (en) * | 2021-04-14 | 2021-09-28 | 上海漫微信息技术有限公司 | Data synchronization management method and device and server |
| CN114547206A (en) * | 2022-03-07 | 2022-05-27 | 牧原食品股份有限公司 | Data synchronization method and data synchronization system |
| CN114780641A (en) * | 2022-05-07 | 2022-07-22 | 湖南长银五八消费金融股份有限公司 | Multi-library multi-table synchronization method and device, computer equipment and storage medium |
| CN115146000A (en) * | 2022-06-20 | 2022-10-04 | 彩讯科技股份有限公司 | Database data synchronization method and device, electronic equipment and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116578650A (en) * | 2023-07-11 | 2023-08-11 | 太平金融科技服务（上海）有限公司 | Data synchronization method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116204587B (en) | 2024-01-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116204587B (en) | Data synchronization task generation method, device and computer readable storage medium | |
| CN109857451B (en) | Method, device, equipment and medium for generating IP core file | |
| CN116821437B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN117474504A (en) | Device information management method, device, electronic equipment and medium | |
| US20200379435A1 (en) | Method and electronic generation device for generating at least one configuration file for an automation tool, related computer program | |
| US8196093B2 (en) | Apparatus and method for componentizing legacy system | |
| CN104462036A (en) | Method and system for synchronously editing format information of document | |
| CN112685040B (en) | Method, device, equipment and storage medium for generating interface file in android system | |
| CN108334333B (en) | Method and device for updating source code base | |
| CN117827218A (en) | Development method, device and electronic equipment for supporting multiple languages by low-code application | |
| CN108664505B (en) | Method and device for exporting database table structure | |
| CN117234582A (en) | Project code processing method and device, electronic equipment and storage medium | |
| CN112486952A (en) | Flow creation method, system, storage medium and electronic device | |
| EP3425510B1 (en) | Automated testing method for application program user interface, electronic device, system, and storage medium | |
| CN116049215A (en) | Method, device, electronic equipment and medium for inquiring options | |
| CN115587021A (en) | Simulation test message generation method and device | |
| JP2015148925A (en) | Program generation device and method | |
| CN114239529A (en) | Document generation method, device, equipment and medium based on template engine | |
| CN112860248B (en) | Source code generation method and device | |
| CN111045696A (en) | Method and device for automatically installing single machine operation system, computer equipment and storage medium | |
| CN114416104B (en) | Structured data file processing method and device | |
| JP2024146079A (en) | Design and development support device, design and development support method, and program | |
| JP2015095035A (en) | Parameter setting support system, parameter setting support method, and parameter setting support program | |
| CN113033137B (en) | Method and device for establishing digital component model and server | |
| CN115809087A (en) | Version updating method, version updating device, version updating equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |