CN113722394A

CN113722394A - Data synchronization method, device and storage medium

Info

Publication number: CN113722394A
Application number: CN202110945065.3A
Authority: CN
Inventors: 丁伟; 陈宝玉; 孙振江; 边伟
Original assignee: Beijing Baiwu Tech Co ltd
Current assignee: Beijing Baiwu Tech Co ltd
Priority date: 2021-08-17
Filing date: 2021-08-17
Publication date: 2021-11-30

Abstract

In the embodiment of the application, a database information and starting mode are configured for a data synchronization program deployed on a database in a file configuration mode, a client and a server can be configured in an integrated mode, and unified management and maintenance are facilitated. And moreover, the database information is managed independently in the form of configuration files, the parameter configuration modes of the client and the server are extracted from the database into independent files, the configuration files can be flexibly expanded and changed according to specific requirements, and the data synchronization between different databases is facilitated to be universally processed.

Description

Data synchronization method, device and storage medium

Technical Field

The present application relates to the field of database technologies, and in particular, to a data synchronization method, device, and storage medium.

Background

In a conventional database synchronization method, data change information is determined by acquiring a database log, and a synchronization task is executed in a target database based on the database log to synchronize the data change information in a source database to the target database. Database synchronization based on database logs requires that the source database be of the same type and table structure as the target database.

The existing database synchronization mode is difficult to solve the data synchronization problem among different types of databases and different table structures, and cannot solve the synchronization problem of partial data such as a designated table or a designated column in the table.

Disclosure of Invention

Various aspects of the present application provide a data synchronization method, device and storage medium, so as to implement data synchronization between different types of databases and improve compatibility of data synchronization.

The embodiment of the application provides a data synchronization method, which is applicable to a first database, wherein a data synchronization program and a corresponding synchronization configuration file are deployed on the first database, and a starting mode of the data synchronization program is specified in the synchronization configuration file, and the method comprises the following steps: starting a data synchronization program on the first database according to a starting mode specified in the synchronization configuration file, wherein the starting mode is a client or a server corresponding to the service role of the data synchronization program; under the condition that a data synchronization program on the first database is started as a client, taking the first database as a target database, and reading the identifier of a source database, a first data table of data to be synchronized in the source database and the identifier of a first data column from the synchronization configuration file; sending a first data synchronization request to the source database according to the source database, the first data table and the identifier of the first data column; and receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, analyzing the first character string and storing the analyzed first character string into a first database.

An embodiment of the present application further provides a computer device, including: a processor and a memory storing a computer program; the processor to execute the computer program to: starting a data synchronization program according to a starting mode designated in a synchronization configuration file, wherein the starting mode is a client or a server corresponding to the service role of the data synchronization program; under the condition that the data synchronization program is started as a client, reading the identifier of a source database, a first data table of data to be synchronized in the source database and the identifier of a first data column from the synchronization configuration file; sending a first data synchronization request to the source database according to the source database, the first data table and the identifier of the first data column; and receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, analyzing the first character string and storing the analyzed first character string into a local database.

An embodiment of the present application further provides a computer device, including: a processor and a memory storing a computer program; the processor to execute the computer program to: starting a data synchronization program according to a starting mode designated in a synchronization configuration file, wherein the starting mode is a client or a server corresponding to the service role of the data synchronization program; under the condition that the data synchronization program is started as a server, taking a local database as a source database, and receiving a second data synchronization request sent by a target database, wherein the second data synchronization request comprises a second data table of data to be synchronized in the local database and an identifier of a second data column; reading data in a second data column from the second data table according to the second data table and the identification of the second data column; and converting the data into a JSON format to obtain a second character string, returning the second character string to the target database, and storing the second character string in the target database after the second character string is analyzed.

Embodiments of the present application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the embodiments of the method of the present application.

In the embodiment of the application, the database information and the starting mode are configured for the data synchronization program deployed on the database in a file configuration mode, the client and the server can be configured in an integrated mode, and unified management and maintenance are facilitated. And moreover, the database information is managed independently in the form of configuration files, the parameter configuration modes of the client and the server are extracted from the database into independent files, the configuration files can be flexibly expanded and changed according to specific requirements, and the data synchronization between different databases is facilitated to be universally processed.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1a is a flowchart of a data synchronization method according to an embodiment of the present application;

FIG. 1b is a flow chart of another data synchronization method provided in the embodiments of the present application;

fig. 1c is a flowchart of another data synchronization method provided in the embodiment of the present application;

fig. 2 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In order to solve the problems existing in the existing data synchronization mode, embodiments of the present application provide a data synchronization tool, where the data synchronization tool individually configures and manages data synchronization tasks, database information, and contents such as cache, log, and start script in the form of independent files, and establishes a corresponding relationship between a source database and a target database by adding configuration files in the case of a data synchronization requirement, so as to synchronize data from the source database to the target database according to the corresponding relationship when a data synchronization program is executed.

In this embodiment of the application, the data synchronization tool may be deployed to a server where the source database is located, or may be deployed to a client where the target database is located. Optionally, the data synchronization tool may be provided with a configuration item of a starting manner, and by configuring different values for the configuration item, it may be identified that the data synchronization program in which the data synchronization tool is deployed is started in different corresponding manners. For example, when the value of the startup mode configuration item is 0, it indicates that the data synchronization program is started in the server role, and accordingly, the database corresponding to the data synchronization program is the source database; when the value of the starting mode configuration item is 1, the data synchronization program is started in a client role, and correspondingly, the corresponding database on the data synchronization program is the target database. Of course, the manner of the partition data synchronization procedure is not limited thereto, and is merely exemplary.

Further, according to the value of the configuration item of the starting mode, the database information can be configured in the database information configuration file provided by the data synchronization tool, and other relevant information can be configured in other relevant configuration files. In this embodiment of the present application, a configuration manner of the database information is not limited, and optionally, the database information corresponding to the start-up manner may be configured according to a value of the start-up manner configuration item, or the source database information and the target database information may be configured at the same time, and the corresponding database information is set to be valid according to a value of the start-up manner configuration item and/or another database information is set to be invalid. For example, if the value of the startup mode configuration item is 0, only the information corresponding to the source database is configured in the database information configuration file; and if the value of the starting mode configuration item is 1, only configuring the information corresponding to the target database in the database information configuration file. For another example, the information of the source database and the information of the target database are configured in the database information configuration file at the same time, and if the value of the start mode configuration item is 0, the information of the source database is set to be valid and/or the information of the target database is set to be invalid; and if the value of the starting mode configuration item is 1, setting the information of the source database to be invalid and/or setting the information of the target database to be valid.

In the following, two cases of starting up in a client role and starting up in a server role on a data synchronization program deployed on a first database will be described in detail with respect to configuration contents in each configuration file provided by a data synchronization tool.

A: a data synchronization program deployed on a first database is started in a client role

In an embodiment of the present application, the data synchronization tool provides a database information configuration file for a data synchronization program started in a client role, the database information configuration file is used for configuring information of a first database, and the first database is used as a target database when the data synchronization program is started in the client role. The database information configuration items in the database information configuration file include but are not limited to: the database driving type, the database connection address, the user name and the password of the database, the check statement for connecting the database and the like. After the database information configuration file is established, the data synchronization program can establish a connection pool of a target database according to the information in the configuration file, and if the configuration file changes, for example, the configuration file is deleted, the database synchronization program can automatically close the previously established connection pool and establish a new connection pool. Under the condition that the data synchronization requirement exists, the data synchronization program can start and execute a data synchronization task, read the value of each database information configuration item from the database information configuration file, and determine the target database so as to synchronize data to the target database under the condition that the data is acquired from the source database.

Further, the data synchronization tool provides a data synchronization task configuration file for a data synchronization program started in a client role, wherein the data synchronization task configuration file is used for configuring task information of a data synchronization task, and the information configuration items include but are not limited to: the name of the data synchronization task, the address (source database address) accessed by the data synchronization task, the database name (source database name) of the data to be synchronized, the SQL statement for performing data synchronization, the local database name (target database name), the SQL statement for updating the local database, the identification of whether the data synchronization task is valid, and the user name and password corresponding to the data synchronization program for performing the data synchronization task.

Further optionally, the data synchronization task configuration file may further include other information configuration items for managing the data synchronization tasks, for example, the data synchronization task configuration file may further include setting an interval time between data synchronization tasks that are continuously started, so that different data synchronization tasks may be sequentially started in a single-thread manner, so as to reduce processing pressure of the server. Of course, the embodiment of the present application is not limited to starting the data synchronization task in a single thread manner, and under the condition that the processing capability of the server is strong, the data synchronization task may be executed in parallel by a plurality of threads. Further optionally, each time a data synchronization task is started, a data synchronization task configuration file may be added to the data synchronization tool, and in order to identify a correspondence between the data synchronization task configuration file and the data synchronization task, a file name of the data synchronization task configuration file may correspond to a name of the data synchronization task, so that when the data synchronization task is started, configuration information corresponding to the data synchronization task is determined according to the correspondence.

In the embodiment of the application, the data synchronization tool comprises a cache file storage directory, and the data synchronization program can automatically generate the cache file in the operation process. Optionally, the cache file storage location of the data synchronization task may be configured as a cache file storage directory, so that the generated cache file is stored in the cache file storage directory when the data synchronization task is executed. In this embodiment, a storage manner of the cache files corresponding to different data synchronization tasks is not limited, for example, the cache files generated during the execution of the data synchronization program may be directly stored in the cache file storage directory, or may be separately stored in the cache file storage directory according to different data synchronization tasks. Further, the configuration manner of the storage locations of the cache files is not limited, for example, the storage locations of all the cache files may be configured in a unified manner by using a single file, or the storage locations of the corresponding cache files may be configured for different data synchronization tasks.

In an optional embodiment of the present application, for a data synchronization task that has already been executed, if it is desired to execute the data synchronization task again under an initial condition, the data synchronization program may be closed and the cache file of the data synchronization task may be deleted, so that after the data synchronization program is restarted, the data synchronization task may be executed according to the initial condition. Further optionally, the data synchronization task may be renamed and a data synchronization task configuration file with a corresponding name may be created, so that when the data synchronization task is executed again, a new data synchronization task is executed, and the data synchronization task may be executed according to the initial condition.

In addition to the configuration files, the data synchronization tool according to the embodiment of the present application may further provide a configuration file with other information for the data synchronization program started as the client role, for example, the configuration file may further include a start script configuration file for configuring various command statements for starting the data synchronization program, and a start manner of the data synchronization program may also be configured in the configuration file for starting the client, so that the data synchronization program may be started in the client role when the data synchronization program is started. Further optionally, the data synchronization tool further provides a configuration file with information such as a synchronization mode for table data and network access duration, and corresponding information can be flexibly configured according to specific requirements of the data synchronization task. For example, the corresponding synchronization mode can be configured as a dynamic synchronization mode for the time data; for information such as connection time, data reading time, data writing time and the like in the data synchronization task execution process, corresponding timeout time can be configured according to requirements, and optionally, the time can be in milliseconds, so that corresponding processing or prompting can be performed under the condition of timeout.

B: a data synchronization program deployed on a first database is started in a server role

In this embodiment of the present application, the data synchronization tool provides a database information configuration file for a data synchronization program started as a server role, where the database information configuration file is used to configure source database information of data to be synchronized, where the database information configuration items include but are not limited to: the database driving type, the database connection address, the user name and the password of the database, the check statement for connecting the database and the like. After the database information configuration file is established, the data synchronization program can establish a connection pool of a source database according to the information in the configuration file, and if the configuration file changes, for example, the configuration file is deleted, the database synchronization program can automatically close the previously established connection pool and establish a new connection pool. Further optionally, in order to establish a corresponding relationship between the source database and the data synchronization task, a name of the database information configuration file may be set as a name of the source database, where the name corresponds to a database name of data to be synchronized in the data synchronization task configuration file of the client. Therefore, when the data synchronization program executes the data synchronization task, the information of the source database can be determined according to the name of the data synchronization task, and the data can be read from the source database.

In the embodiment of the application, the client can access the server and obtain data from the server, and for the server, the client can be accessed by any client in principle. Optionally, in order to ensure the security and reliability of data synchronization, the data synchronization tool provides an account configuration file for a data synchronization program started as a server role, where the account configuration file is used to configure the access right of the client. Optionally, the information configuration items in the account configuration file include, but are not limited to: account name and password, identification of whether the account is valid or not, database address allowing access and the like; the account name is an identity of a data synchronization tool used by the server; the database addresses allowed to be accessed are the corresponding database addresses of the clients, which may be one or more.

In this embodiment of the application, one server uses one account, each account corresponds to one account configuration file, optionally, in order to identify account information corresponding to different accounts, a name of an account configuration file may be set as a corresponding account name, and when a certain account configuration file is changed or deleted, an access right of a client that the server corresponding to the account allows to access will be affected. Further optionally, an account configuration file storage directory is provided in the data synchronization tool, and the account configuration file configured for each server may be stored in the account configuration file storage directory. In this embodiment, a storage manner of the account files corresponding to different servers is not limited, for example, the account files corresponding to different servers may be directly stored in the account configuration file storage directory, or may be separately stored in the account configuration file storage directory according to different servers.

In the embodiment of the application, a corresponding log file can be generated in the execution process of the data synchronization program, the change information of the database can be determined by checking the log file, the database data can be maintained according to the change information of the database, and the data synchronization program can be debugged according to the running log recorded in the log file under the condition that the running of the data synchronization program is wrong. Further optionally, under the condition that there is a data synchronization requirement, the same database change operation may be performed on the target database according to the source database log recorded in the log file. Based on this, in the embodiment of the present application, the data synchronization tool further provides a log configuration file for the data synchronization program started in the role of the server, where the configuration items include, but are not limited to, a source database name, and a storage location of the log file corresponding to the source database, for example, the configuration log file is stored in an installation directory of the data synchronization program. In this way, when the target database needs to be changed as the source database, the change information of the source database can be determined according to the log file corresponding to the source database, so as to perform the same change on the target database.

In addition to the configuration file, the data synchronization tool of the embodiment of the present application may also provide the client with a configuration file of other information, for example, the data synchronization tool may further include a start script configuration file for configuring various command statements for starting the data synchronization program, and the data synchronization program may also be configured in the configuration file to start the client, so that when the data synchronization program is started, the data synchronization program may be started in the form of a server.

In the above embodiment, the content of the configuration file is described from two perspectives of starting the data synchronization program by the client or the server, and in the embodiment of the present application, the manner in which the data synchronization tool provides the configuration file is not limited. Optionally, the data synchronization tool may provide a corresponding configuration file according to the start role of the data synchronization program, or may also provide the configuration file at the same time, and set the corresponding configuration file to be valid according to the start role of the data synchronization program and/or set the configuration file corresponding to the other end to be invalid. Of course, besides the configuration files, other configuration files may be added according to actual needs, and are not described in the embodiment of the present application.

In the embodiment of the application, the database information, the data synchronization task and other information are managed separately in the form of the configuration file, the parameter configuration modes of the client and the server are extracted from the database into the independent file, and the configuration file can be expanded and changed flexibly according to specific requirements. In addition, the client and the server are distinguished by roles in a starting mode, and the configuration information corresponding to the client and the server is integrated, so that the same maintenance is facilitated.

Based on the above, an embodiment of the present application further provides a data synchronization method, which is applicable to the data synchronization tool in the foregoing embodiment, in the embodiment of the present application, a database corresponding to the data synchronization tool is referred to as a first database; the first database is provided with a data synchronization program and a corresponding synchronization configuration file. Optionally, the synchronization configuration file includes a database information configuration file, a data synchronization task configuration file, an account configuration file, a cache information configuration file, a log configuration file, a start script configuration file, and other information configuration files. For specific contents in each configuration file and functions of the configuration files, reference may be made to the above embodiments, which are not described herein again.

Fig. 1a is a flowchart of a data synchronization method provided in an embodiment of the present application, and as shown in fig. 1a, the method includes:

s1a, starting a data synchronization program on a first database according to a starting mode specified in a synchronization configuration file, wherein the starting mode corresponds to the service role of the data synchronization program and is a client or a server;

s2a, under the condition that the data synchronization program on the first database is started as a client, taking the first database as a target database, and reading the identifier of the source database, the first data table of the data to be synchronized in the source database and the identifier of the first data column from the synchronization configuration file;

s3a, sending a first data synchronization request to the source database according to the source database, the first data table and the identifier of the first data column;

and S4a, receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, analyzing the first character string and storing the analyzed first character string into the first database.

In the embodiment of the application, the database synchronization program supports a client starting mode and a server starting mode, and the service role of the data synchronization program can be determined to be the client or the server according to the corresponding starting modes. In the embodiment of the present application, the synchronization configuration file is provided with an SQL statement for performing data synchronization, and a data table and a data column of data to be synchronized may be specified in the SQL statement. In this embodiment of the present application, the source database includes a first data table, the first data table includes a first data column, and when it is determined that the service role of the data synchronization program is the client according to the starting manner, the first database corresponding to the data synchronization program may be used as the target database, and according to the SQL statement for performing data synchronization in the synchronization configuration file, the identifier of the source database, the first data table of the data to be synchronized in the source database, and the identifier of the first data column may be read. And then, according to the source database, the first data table and the identifier of the first data column, sending a first data synchronization request to the source database, so that the source database queries data in the first data column of the first data table according to the first data synchronization request.

In the embodiment of the present application, the corresponding data synchronization task information and the corresponding source database information may be configured according to the data synchronization requirement. Optionally, the number of the configured data synchronization tasks may be multiple, and in the case of a data synchronization requirement, the data synchronization program may sequentially start the data synchronization tasks and store the started data synchronization tasks in the local cache queue. Under the condition that the execution condition is met, the data synchronization program can read the database synchronization tasks from the local cache queue in sequence, determine the corresponding synchronization configuration file according to the database synchronization tasks, read the identifier of the source database, the first data table of the data to be synchronized in the source database and the identifier of the first data column from the synchronization configuration file, carry the read information in the first data synchronization request and send the first data synchronization request to the source database, so that the source database can query the data from the first data column according to the first data synchronization request. In this embodiment, the execution condition of the data synchronization task is not limited, optionally, the corresponding data synchronization task may be executed when the specified time arrives, or the data synchronization task may be executed according to a trigger of a user or other conditions, and may be flexibly set according to a requirement.

In this embodiment of the application, a manner in which the target database sends the first data request to the source database is not limited, and optionally, the first data request may be sent in an http manner. Further optionally, the first data request includes an identifier of the first data table and an identifier of the first data column, so that the source database queries the data to be synchronized from the first data table and the identifier of the first data column. In an optional embodiment, an SQL statement for performing data synchronization may be directly carried in the first data synchronization request, so that the source database determines the identifier of the first data table and the first data column according to the SQL statement; in another optional embodiment, the target database may also determine the identifiers of the first data table and the first data column according to the SQL statement, and carry the identifiers of the first data table and the first data column in the first data synchronization request. Further, under the condition that the source database queries corresponding data according to the first data synchronization request, the queried data can be returned to the target database, and the target database can store the data returned by the source database into the target database after receiving the data.

Further optionally, in order to improve the compatibility of data synchronization between different databases, the data types stored in the different databases may be subjected to a universal process, and optionally, the source database may perform format conversion after querying the data, and package the data into data in a specified format. When the target database receives the data in the specified format, the data can be analyzed into a data type matched with the target database and stored. Optionally, the source database may convert the data in the first data table into a first character string in the JSON format and return the first character string to the target database, and after receiving the first character string, the target database may parse the first character string and store the first character string in the target database.

In the embodiment of the application, the number of pieces of data, which are requested by the target database to be synchronized with the source database, is multiple, and the first data synchronization request can be sent to the source database in batches under the condition of very large data volume. In an optional embodiment, when the first data synchronization request is sent to the source database each time, the number of the data to be synchronized may be carried in the first data synchronization request, so that the source database may query the corresponding number of data according to the number. Further optionally, in the case that the target database receives data returned by the source database each time, the identifier of the last piece of data received this time may be stored in the cache as a breakpoint identifier, and when a first data synchronization request is sent to the source database next time, the breakpoint identifier is sent to the source database, so that the source database queries a corresponding number of pieces of data from the first data column according to the breakpoint identifier, the number of pieces of data to be synchronized, the first data table, and the identifier of the first data column. Further, the source database may convert, according to the request information, a plurality of rows of data queried in the first data column into a JSON format under the condition that the first data synchronization request is received, and then return the converted first character string to the target database. Further, the target database may analyze the first character string and store the first character string in the target database, and replace the breakpoint identifier in the cache with the identifier of the last piece of data obtained through analysis.

In an embodiment of the present application, the data synchronization program deployed on the first database may also be started in a server role. Fig. 1b is a flowchart of another data synchronization method provided in this embodiment, where the flowchart corresponds to a process in which a data synchronization program deployed on a first database is started in a server role and performs data synchronization. As shown in fig. 1b, the method comprises:

s1b, starting a data synchronization program on a first database according to a starting mode specified in a synchronization configuration file, wherein the starting mode corresponds to the service role of the data synchronization program and is a client or a server;

s2b, under the condition that a data synchronization program on a first database is started as a server, taking the first database as a source database, and receiving a second data synchronization request sent by a target database, wherein the second data synchronization request comprises a second data table of data to be synchronized in the first database and an identifier of a second data column;

s3b, reading the data in the second data column from the second data table according to the second data table and the identification of the second data column; converting the data into a JSON format to obtain a second character string, and returning the second character string to the target database; and the second character string is analyzed and then stored in the target database.

In this embodiment of the application, when the data synchronization program deployed on the first database is started as a server, the first database may be used as a source database, and the source database may further receive a second data synchronization request sent from the target database, and return data that is requested to be synchronized to the target database according to information in the second data synchronization request. In this embodiment of the present application, the source database includes a second data table, and the second data table includes a second data column, and optionally, the second data synchronization request includes a second data table of data to be synchronized in the source database and an identifier of the second data column. Based on this, the source database may query data from the second data column in the second data table and return the queried data to the target database. Further optionally, when the source database queries data from the second data column of the second data table, the source database may convert the queried data into a second character string in the JSON format, and return the second character string to the target database, and the target data may parse the second character string and store the parsed second character string in the target database after receiving the second character string.

In the embodiment of the present application, the synchronization configuration file is stored in the folder of the repository file, and the required database information may be configured in the synchronization configuration file before the data synchronization program deployed on the first database is started. Optionally, the synchronization configuration file at least includes the first configuration information, the second configuration information, the identifier of the first database, and the starting mode of the data synchronization program. The first information corresponds to a target database and comprises an identifier of a source database to be accessed, a data table of data to be synchronized in the source database and an identifier of a data column; the second information corresponds to the source database and includes an identifier of a target database having a right to access the source database. Optionally, the target database corresponding to the first configuration information is a first database, and the source database in the first configuration information is a second database; the source database corresponding to the second configuration information is the first database, and the target database with the authority to access the source database is the third database.

In an optional embodiment of the present application, the configuration modes of the first configuration information and the second configuration information are not limited, and in an optional embodiment, the corresponding database information may be configured according to the starting mode of the data synchronization program, for example, when the starting mode corresponds to the client role, the first configuration information is configured in the synchronization configuration file; and when the starting mode corresponds to the server role, configuring second configuration information in the synchronous configuration file. In another optional embodiment, the first configuration information and the second configuration information may be configured in the synchronization configuration file at the same time, and the validity of both may be set according to the starting mode of the data synchronization program, for example, when the starting mode corresponds to the client role, the first configuration information is set to be effective; and when the starting mode corresponds to the server role, setting the second configuration information to take effect.

Further optionally, the first configuration information, the second configuration information, the identifier of the first database, and the configuration location of the starting manner of the data synchronization program are not limited, in an optional embodiment, the configuration information may be configured in different configuration files, and the specific configuration location of each configuration information may refer to the above embodiment. In another optional embodiment, the configuration information may be configured in one configuration file at the same time, in which case, the required configuration information may be set to be valid and/or the unnecessary information may be set to be invalid according to specific requirements.

In an optional embodiment of the present application, the first configuration information includes an SQL statement used by the second database for performing data synchronization, and optionally, in order to reduce understanding cost, the SQL statement used by the first configuration information for performing data synchronization may be set as a pseudo first SQL statement, where the pseudo first SQL statement includes a first data table to be synchronized and a first data column, and when a data synchronization program deployed on the first database is started in a client role, the first database (target database) sends, to the source database, the SQL statement used by the first database for performing data synchronization, which is carried by the first data synchronization request, as the pseudo first SQL statement. Correspondingly, when the data synchronization program deployed on the first database is started in the role of a server, a second data synchronization request received by the first database (source database) contains a second pseudo SQL statement required by the first database to perform data synchronization, wherein the second pseudo SQL statement comprises a second data table and an identifier of a second data column; the first database can read the data in the second data column from the second data table according to the second data table and the identification of the second data column.

In an alternative embodiment of the present application, before reading the data, the first database may convert the second pseudo SQL statement into an executable SQL statement, execute the converted SQL statement, and read the data in the second data column from the second data table. Further optionally, in order to ensure the security of data synchronization, before the first database reads the data in the second data column from the second data table according to the second data table and the identifier of the second data column, the first database may check the target database performing data access, and determine that the target database has sufficient authority to access the first database. Optionally, the identifier of the third database allowed to be accessed in the synchronization configuration file may be read, and whether the identifier of the third database includes the identifier of the target database is determined; and if the data is not contained, returning a prompt of forbidding access or failure of access to the target database.

In the embodiment of the present application, it is not limited whether the types of the source database and the target database are the same, and a connection may be established with the data synchronization program deployed thereon through an adapted database connection according to the type of the source database or the target database and the development environment on which the data synchronization program deployed thereon depends. For example, taking a JAVA-based development environment as an example, a connection between a first database and a Data synchronization program deployed on the first database may be established through JAVA database connection (JDBC); for another example, based on a development environment of another development language, an Open Database interconnection (ODBC) may be used to establish a connection between the first Database and the data synchronization program deployed thereon, which may be determined according to actual requirements. And the source database returns the queried data to the target database in a character string of an execution format, such as a JSON format, and the target database can be analyzed into an adaptive data type after receiving the data of the specified format, so that the compatibility of data synchronization between the source database and the target database can be realized, and the stability of the data synchronization is facilitated.

Based on the above, the overall flowchart of the data synchronization method provided in the embodiment of the present application may refer to fig. 1c, and as shown in fig. 1c, the method includes two processes:

scheme 1:

s1, starting the data synchronization program on the first database according to the starting mode specified in the synchronization configuration file, wherein the service role of the data synchronization program corresponding to the starting mode is a client or a server;

s21, under the condition that the data synchronization program on the first database is started as a client, taking the first database as a target database, and reading the identifier of the source database, the first data table of the data to be synchronized in the source database and the identifier of the first data column from the synchronization configuration file;

s31, sending a first data synchronization request to the source database according to the source database, the first data table and the identifier of the first data column;

s41, the source database receives a first data synchronization request sent by the target database, wherein the first data synchronization request comprises a first data table of data to be synchronized and an identifier of a first data column;

s51, reading the data in the first data column from the first data table according to the first data table and the identification of the first data column; converting the data into a JSON format to obtain a first character string, and returning the first character string to a first database;

s61, the first database receives a first character string returned by the source database after converting the data in the first data column into the JSON format, and the first character string is analyzed and then stored in the first database.

And (2) a flow scheme:

s22, the target database reads the identification of the first database, the second data table of the data to be synchronized in the first database and the identification of the second data column from the synchronization configuration file;

s32, sending a second data synchronization request to the first database according to the first database, the second data table and the identification of the second data column;

s42, under the condition that the data synchronization program on the first database is started as a client, receiving a second data synchronization request sent by a target database, wherein the second data synchronization request comprises a first data table of data to be synchronized in the first database and an identifier of a first data column;

s52, reading the data in the first data column from the first data table according to the identifications of the first data table and the first data column; converting the data into a JSON format to obtain a first character string, and returning the first character string to the target database;

and S62, the target database receives a second character string returned after the data in the second data column is converted into the JSON format by the first database, and the second character string is analyzed and then stored in the target database.

Steps S1, S21, S31, and S61 in the flowchart 1 are steps executed when the data synchronization program deployed on the first database is started as a client, and steps S41 and S51 are steps executed for the source database of the corresponding server; steps S1, S42, and S52 in flow 2 are steps performed when the data synchronization program deployed on the first database is started as a server, and steps S22, S32, and S62 are steps performed for a target database of the corresponding client. For the specific implementation process and details of each step, reference may be made to the above embodiments, which are not described herein again.

In the embodiment of the application, the database information and the starting mode are configured for the data synchronization program deployed on the database in a file configuration mode, the client and the server can be configured in an integrated mode, and unified management and maintenance are facilitated. And moreover, the database information, the data synchronization task and other information are managed independently in the form of configuration files, the parameter configuration modes of the client and the server are extracted from the databases into the independent files, the configuration files can be flexibly expanded and changed according to specific requirements, and the data synchronization between different databases is facilitated to be universally processed.

It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of steps S1a to S4a may be device a; for another example, the execution subject of steps S1 to S31 may be device a, and the execution subject of steps S41 and S51 may be device B; and so on.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations occurring in a specific order are included, but it should be clearly understood that the operations may be executed out of the order they appear herein or in parallel, and the sequence numbers of the operations, such as S1a, S2a, etc., are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor limit the types of "first" and "second" to be different.

An embodiment of the present application further provides a computer device, which is suitable for a client, and fig. 2 is a schematic structural diagram of the computer device provided in the embodiment of the present application, and as shown in fig. 2, the computer device includes: a processor 21 and a memory 22 in which a computer program is stored; the processor 21 and the memory 22 may be one or more.

The memory 22 is mainly used for storing computer programs, and these computer programs can be executed by the processor 21, so that the processor 21 controls the computer device to implement corresponding functions, and complete corresponding actions or tasks. In addition to storing computer programs, the memory 22 may be configured to store other various data to support operations on the computer device. Examples of such data include instructions for any application or method operating on a computer device.

The memory 22, which may be implemented by any type of volatile or non-volatile memory device or combination thereof, may include, for example, a Static Random Access Memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

In the embodiment of the present application, the implementation form of the processor 21 is not limited, and may be, for example, but not limited to, a CPU, a GPU, an MCU, or the like. The processor 21 may be regarded as a control system of the computer device and may be adapted to execute a computer program stored in the memory 22 to control the computer device to perform a corresponding function, to perform a corresponding action or task. It should be noted that, according to the implementation form and the scene of the computer device, the functions, actions or tasks to be implemented may be different; accordingly, the computer programs stored in the memory 22 may vary, and execution of different computer programs by the processor 21 may control the computer device to perform different functions, perform different actions or tasks.

In some alternative embodiments, as shown in fig. 2, the computer device may further include: a display 23, a power supply component 24, and a communication component 25. Only some of the components are schematically shown in fig. 2, which does not mean that the computer device only comprises the components shown in fig. 2, but the computer device may also comprise other components for different application requirements, for example, in case of a need for voice interaction, as shown in fig. 2, the computer device may also comprise an audio component 26. The components that can be included in the computer device may depend on the product form of the computer device, and are not limited herein.

In the embodiment of the present application, when the processor 21 executes the computer program in the memory 22, it is configured to: starting a data synchronization program according to a starting mode specified in a synchronization configuration file, wherein a service role of the data synchronization program corresponding to the starting mode is a client or a server; under the condition that a data synchronization program is started as a client, reading the identifier of a source database, a first data table of data to be synchronized in the source database and the identifier of a first data column from a synchronization configuration file; sending a first data synchronization request to the source database according to the source database, the first data table and the identifier of the first data column; and receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, analyzing the first character string and storing the analyzed first character string into the local database.

In an optional embodiment, when reading the identifier of the source database, the first data table of the data to be synchronized in the source database, and the identifier of the first data column from the synchronization configuration file, the processor 21 is configured to: and reading the database synchronization tasks from the local cache queue in sequence, determining a corresponding synchronization configuration file according to the database synchronization tasks, and reading the identifier of the source database, the first data table of the data to be synchronized in the source database and the identifier of the first data column from the synchronization configuration file.

In an optional embodiment, when sending the first data synchronization request to the source database according to the source database, the first data table, and the identifier of the first data column, the processor 21 is configured to: and sending a first data synchronization request to the source database according to the breakpoint identifier in the cache, the source database, the first data table and the identifier of the first data column, wherein the first data synchronization request comprises the breakpoint identifier.

In an optional embodiment, when receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, and storing the parsed first character string in the first database, the processor 21 is configured to: receiving a first character string returned by the source database after converting a plurality of rows of data in the first data column into a JSON format, analyzing the first character string, storing the analyzed first character string into the first database, and replacing the breakpoint identifier in the cache with the identifier of the last piece of data obtained through analysis.

In an optional embodiment, the synchronization configuration file includes first configuration information, second configuration information, an identifier of the first database, and a starting mode of the data synchronization program, and the synchronization configuration file is in a folder of the repository file; when the starting mode corresponds to the client role, the first configuration information takes effect; when the starting mode corresponds to the server role, the second configuration information takes effect; the first information comprises an identifier of a second database to be accessed, a data table of data to be synchronized in the second database and an identifier of a data column, and the second information comprises an identifier of a third database with authority to access the first database.

In an optional embodiment, the first configuration information further includes a first pseudo SQL statement used by the second database to perform data synchronization, where the first pseudo SQL statement includes a first data table and a first data column to be synchronized; the processor 21 is further configured to: and carrying the pseudo first SQL statement in the first data synchronization request and sending the pseudo first SQL statement to the source database.

Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program can implement the steps performed by the computer device serving as the client in the foregoing method embodiments when executed.

An embodiment of the present application further provides a computer device, which is suitable for a server, and has a structure similar to that of the computer device serving as a client shown in fig. 2, and specifically, refer to the schematic structural diagram shown in fig. 2. In this embodiment, the computer device includes: a processor and a memory storing a computer program; the processor and the memory may be one or more, among others.

The memory is mainly used for storing computer programs, and the computer programs can be executed by the processor to enable the processor to control the computer equipment to realize corresponding functions and complete corresponding actions or tasks. In addition to storing computer programs, the memory may be configured to store other various data to support operations on the computer device. Examples of such data include instructions for any application or method operating on a computer device.

The memory may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

In the embodiment of the present application, the implementation form of the processor is not limited, and for example, the processor may be, but not limited to, a CPU, a GPU, an MCU, or the like. The processor may be regarded as a control system of the computer device and may be configured to execute a computer program stored in the memory to control the computer device to implement the corresponding functions and to perform the corresponding actions or tasks. It should be noted that, according to the implementation form and the scene of the computer device, the functions, actions or tasks to be implemented may be different; accordingly, the computer programs stored in the memory may vary, and execution of different computer programs by the processor may control the computing device to perform different functions, perform different actions or tasks.

In some optional embodiments, the computer device may further comprise: the present embodiment only shows some components schematically, and does not mean that the computer device only includes these components, and the computer device may also include other components for different application requirements, for example, in the case of a voice interaction requirement, the computer device may also include an audio component. The components that can be included in the computer device may depend on the product form of the computer device, and are not limited herein.

In an embodiment of the application, the processor, when executing the computer program in the memory, is configured to: starting a data synchronization program according to a starting mode specified in a synchronization configuration file, wherein a service role of the data synchronization program corresponding to the starting mode is a client or a server; under the condition that the data synchronization program is started as a server, taking a local database as a source database, and receiving a second data synchronization request sent by a target database, wherein the second data synchronization request comprises a second data table of data to be synchronized in the local database and an identifier of a second data column; reading data in the second data column from the second data table according to the second data table and the identification of the second data column; and converting the data into a JSON format to obtain a second character string, returning the second character string to the target database, and storing the second character string in the target database after the second character string is analyzed.

In an optional embodiment, the second data synchronization request further includes a second pseudo SQL statement required by the first database to perform data synchronization, where the second pseudo SQL statement includes a second data table and an identifier of a second data column; accordingly, the processor, when reading the data in the second data column from the second data table according to the second data table and the identification of the second data column, is configured to: and converting the second pseudo SQL statement into an executable SQL statement, and executing the executable SQL statement to read the data in the second data column from the second data table.

In an optional embodiment, the processor is further configured to, before reading the data in the second data column from the second data table according to the second data table and the identification of the second data column: reading the identifier of a third database which is allowed to be accessed in the synchronous configuration file, and judging whether the identifier of the third database contains the identifier of the target database; and in the case of inclusion, performing an operation of reading data in the second data column from the second data table according to the second data table and the identification of the second data column.

Accordingly, the present application further provides a computer-readable storage medium storing a computer program, where the computer program can implement the steps performed by the computer device serving as the server in the foregoing method embodiments when executed.

The communication component in the above embodiments is configured to facilitate communication between the device in which the communication component is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as a WiFi, a 2G, 3G, 4G/LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

The display in the above embodiments includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

The power supply assembly of the above embodiments provides power to various components of the device in which the power supply assembly is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

The audio component in the above embodiments may be configured to output and/or input an audio signal. For example, the audio component includes a Microphone (MIC) configured to receive an external audio signal when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal may further be stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data synchronization method, adapted to a first database, where a data synchronization program and a corresponding synchronization configuration file are deployed on the first database, and a starting manner of the data synchronization program is specified in the synchronization configuration file, the method comprising:

starting a data synchronization program on the first database according to a starting mode specified in the synchronization configuration file, wherein the starting mode is a client or a server corresponding to the service role of the data synchronization program;

under the condition that a data synchronization program on the first database is started as a client, taking the first database as a target database, and reading the identifier of a source database, a first data table of data to be synchronized in the source database and the identifier of a first data column from the synchronization configuration file;

sending a first data synchronization request to the source database according to the source database, the first data table and the identifier of the first data column; and

and receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, analyzing the first character string and storing the analyzed first character string into a first database.

2. The method of claim 1, wherein reading the identifier of the source database and the identifiers of the first data table and the first data column of the data to be synchronized in the source database from the synchronization configuration file comprises:

the method comprises the steps of reading database synchronization tasks from a local cache queue in sequence, determining a corresponding synchronization configuration file according to the database synchronization tasks, and reading an identifier of a source database, a first data table of data to be synchronized in the source database and an identifier of a first data column from the synchronization configuration file.

3. The method of claim 1, wherein sending a first data synchronization request to the source database based on the source database, the first data table, and the identifier of the first data column comprises:

and sending a first data synchronization request to the source database according to the breakpoint identifier in the cache, the source database, the first data table and the identifier of the first data column, wherein the first data synchronization request comprises the breakpoint identifier.

4. The method according to claim 3, wherein receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, and storing the parsed first character string in the first database comprises:

receiving a first character string returned by the source database after converting a plurality of rows of data in the first data column into a JSON format, analyzing the first character string, storing the analyzed first character string into the first database, and replacing the breakpoint identifier in the cache with the identifier of the last piece of data obtained through analysis.

5. The method of claim 1, further comprising:

under the condition that a data synchronization program on the first database is started as a server, taking the first database as a source database, and receiving a second data synchronization request sent by a target database, wherein the second data synchronization request comprises a second data table of data to be synchronized in the first database and an identifier of a second data column;

reading data in a second data column from the second data table according to the second data table and the identification of the second data column; and converting the data into a JSON format to obtain a second character string, returning the second character string to the target database, and storing the second character string in the target database after the second character string is analyzed.

6. The method of claim 5, prior to initiating a data synchronization procedure on the first database according to the initiation mode specified in the synchronization profile, comprising:

the synchronous configuration file comprises first configuration information, second configuration information, an identifier of a first database and a starting mode of the data synchronization program, and the synchronous configuration file is in a folder of a repository file;

when the starting mode corresponds to a client role, the first configuration information takes effect; when the starting mode corresponds to a server role, the second configuration information takes effect;

the first information comprises an identifier of a second database to be accessed, a data table of data to be synchronized in the second database and an identifier of a data column, and the second information comprises an identifier of a third database with authority to access the first database.

7. The method according to claim 6, wherein the first configuration information further includes a first pseudo SQL statement used by the second database to perform data synchronization, and the first pseudo SQL statement includes a first data table and a first data column to be synchronized;

the method further comprises the following steps: and carrying the pseudo first SQL statement in the first data synchronization request and sending the pseudo first SQL statement to the source database.

8. The method according to claim 5, wherein the second data synchronization request further includes a second pseudo SQL statement required by the first database to perform data synchronization, and the second pseudo SQL statement includes the identifier of the second data table and the second data column;

correspondingly, reading the data in the second data column from the second data table according to the second data table and the identification of the second data column, including:

and converting the second pseudo SQL statement into an executable SQL statement, and executing the executable SQL statement to read the data in the second data column from the second data table.

9. The method of claim 6, further comprising, before reading the data in the second column of data from the second data table based on the second data table and the identification of the second column of data:

reading the identifier of a third database which is allowed to be accessed in the synchronous configuration file, and judging whether the identifier of the third database contains the identifier of the target database;

and if the data is contained, performing an operation of reading the data in the second data column from the second data table according to the second data table and the identification of the second data column.

10. A computer device, comprising: a processor and a memory storing a computer program;

the processor to execute the computer program to:

starting a data synchronization program according to a starting mode designated in a synchronization configuration file, wherein the starting mode is a client or a server corresponding to the service role of the data synchronization program;

under the condition that the data synchronization program is started as a client, reading the identifier of a source database, a first data table of data to be synchronized in the source database and the identifier of a first data column from the synchronization configuration file;

and receiving a first character string returned by the source database after converting the data in the first data column into the JSON format, analyzing the first character string and storing the analyzed first character string into a local database.

11. A computer device, comprising: a processor and a memory storing a computer program;

the processor to execute the computer program to:

under the condition that the data synchronization program is started as a server, taking a local database as a source database, and receiving a second data synchronization request sent by a target database, wherein the second data synchronization request comprises a second data table of data to be synchronized in the local database and an identifier of a second data column;

reading data in a second data column from the second data table according to the second data table and the identification of the second data column;

and converting the data into a JSON format to obtain a second character string, returning the second character string to the target database, and storing the second character string in the target database after the second character string is analyzed.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 9.