CN111984723A - Data synchronization method and device and terminal equipment - Google Patents
Data synchronization method and device and terminal equipment Download PDFInfo
- Publication number
- CN111984723A CN111984723A CN201910438796.1A CN201910438796A CN111984723A CN 111984723 A CN111984723 A CN 111984723A CN 201910438796 A CN201910438796 A CN 201910438796A CN 111984723 A CN111984723 A CN 111984723A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- service
- width table
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention is suitable for the technical field of data processing, and provides a data synchronization method, a device and terminal equipment, wherein the data synchronization method comprises the following steps: if the change of the target data of the target database is monitored through the data transmission service DTS, the changed target data is updated to a service width table, wherein the target database is stored in a target server; and reading the data of the service width table, and synchronizing the data of the service width table to a target client. The embodiment of the invention can reduce the pressure on the database server during data synchronization.
Description
Technical Field
The invention belongs to the technical field of data processing, and particularly relates to a data synchronization method, a data synchronization device and terminal equipment.
Background
With the development of the big data era, more and more data service providers providing cloud computing and data monitoring provide important data support for various industries. In the prior art, a data service provider is usually required to read data of each database of a database server according to a client requirement and synchronize the data into a client report. However, in the existing data synchronization process, there is often a situation that the database server needs to perform reading and writing at the same time, that is, the database server performs data updating and writing into the database according to new monitoring information and calculation information, and also responds to the request of the client in time to perform database reading operation to implement data synchronization to the client, so that the problems of large database server pressure and low data synchronization efficiency exist in the existing data synchronization process.
Disclosure of Invention
In view of this, embodiments of the present invention provide a data synchronization method, an apparatus, and a terminal device, so as to solve a problem in the prior art how to reduce pressure on a database server during data synchronization.
A first aspect of an embodiment of the present invention provides a data synchronization method, including:
if the change of the target data of the target database is monitored through the data transmission service DTS, the changed target data is updated to a service width table, wherein the target database is stored in a target server;
and reading the data of the service width table, and synchronizing the data of the service width table to a target client.
A second aspect of an embodiment of the present invention provides a data synchronization apparatus, including:
the monitoring unit is used for updating the changed target data to the service width table if the change of the target data of the target database is monitored through the data transmission service DTS, wherein the target database is stored in the target server;
and the synchronization unit is used for reading the data of the service width table and synchronizing the data of the service width table to the target client.
A third aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the data synchronization method when executing the computer program.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the steps of the data synchronization method as described.
Compared with the prior art, the embodiment of the invention has the following beneficial effects: in the embodiment of the invention, the change of the target data is monitored and updated to the service width table through the data transmission service DTS, and the data is synchronized to the target client through reading the service width table, so that the data synchronization can be realized by avoiding directly performing query reading operation on the target database, and the process of realizing the data synchronization through the reading operation is separated from the target database, thereby realizing the data read-write separation, reducing the pressure on the target server and improving the data synchronization efficiency.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flow chart of an implementation of a first data synchronization method according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of an implementation of a second data synchronization method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a data synchronization apparatus provided in an embodiment of the present invention;
fig. 4 is a schematic diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
In addition, in the description of the present application, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
The first embodiment is as follows:
fig. 1 shows a schematic flow chart of a first data synchronization method provided in an embodiment of the present application, which is detailed as follows:
in S101, if it is monitored through the data transmission service DTS that the target data of the target database is changed, the changed target data is updated to the service width table, where the target database is stored in the target server.
Data Transmission Service (DTS) is a Data cloud service, and can implement Data streaming service for Data interaction between multiple Data sources, and can monitor Data changes of subscribed databases through DTS subscription.
The execution subject of the embodiment of the present application is a server, and the server is hereinafter referred to as a synchronization server in order to be distinguished from a target server. The target database is a database which is monitored by the synchronization server in real time through DTS, wherein one or more target databases are stored in one or more target servers, and the target servers are different from the synchronization server. Specifically, various service data are stored in the target database, and the synchronization server monitors specified service data, i.e., target data, in the target database in real time by means of the DTS.
The service data changes with the change of specific services, such as the operation of a client or the change of monitoring data of a sensor, so that the target server storing each service data writes new service data into a target database in the target server along with the change of the services, so as to realize real-time update. When the target data is changed due to update writing of services, such as operations of adding, modifying, deleting and the like to the data, the DTS platform monitors the change of the target data or monitors the update writing operation of the target database in the target data server, and reports the changed target data information to the synchronization server, that is, the synchronization server can monitor whether the target data of the target database is changed through the DTS. When the synchronous server monitors that the target data of the target database is changed through the DTS, the synchronous server updates the changed target data to the service width table stored in the synchronous server. The service width table is specifically a data table including each service data (i.e., target data of each target database), and the target data of each target data table originally dispersed in each target database is collected in the service width table.
Optionally, the database type of the target database includes any one or more of MySQL, Monogo, Oracle, and SQL Server.
The target databases monitored by the synchronization Server through the DTS may be one or more, and the types of the target databases may also be one or more, specifically, the database types of the target databases may include any one or more of MySQL, Monogo, Oracle, and SQL Server. The data of the target database of any one or more database types can be simultaneously monitored through the DTS.
Specifically, when the synchronization server monitors that the target data changes through the DTS, the synchronization server performs data cleaning on the changed target data, for example, performs format conversion on the target data of various database types, uniformly converts the target data into a data format corresponding to the service wide table, and then updates the target data after the data cleaning into the service wide table.
In S102, the data of the service width table is read, and the data of the service width table is synchronized to the target client.
The target client needs to request specific target data through the self report form need. The service width table includes each target data, and since the target data stored in the service width table is monitored through the DTS and updated in real time, that is, the target data stored in the service width table is the latest, the service width table is a data table including the most complete and latest target data. And reading the data of the service width table according to the report form requirement of the target client, synchronizing the data of the service width table to the target client, meeting the report form requirement of the target client and presenting the latest target data. Optionally, the synchronization server may simultaneously read corresponding data in the service wide table according to different report requirements of the multiple target clients, and synchronize the data required by each client to the report of each target client.
When the target data is obtained through a single service wide table, the target data can be read only through simple database query statements, and the target data can be obtained without the mode of obtaining the target data from original different target databases and target data tables through multiple layers and query statements, so that the traversal query of data statements can be reduced, dozens of or even hundreds of rows of data query statements are changed into one row, and the data reading synchronization efficiency is improved.
Optionally, the step S102 includes:
and receiving a request of a target client, reading the data of the service width table, and synchronizing the data of the service width table to the target client.
And when a request of a target client is received, reading the data of the service width table, and synchronizing the data of the service width table to the target client. The request comprises the service data information requested by the target client, correspondingly reads the data of the service wide table according to the service data information carried by the request, and synchronizes the read data to the target client.
Because the data of the service wide table is synchronized to the target client when the request of the target client is received, the data synchronization times can be reduced, the requirement of the target client can be responded in time, and the resource consumption of the system is saved.
Optionally, the step S102 includes:
and automatically reading the data of the service width table at preset time intervals, and synchronizing the data of the service width table to a target client.
And pre-storing service data information and preset time required by one or more target clients in the synchronous server. According to the preset time, automatically reading corresponding data in the service width table according to the service data information required by the target client at intervals of the preset time (for example, 1s or 5 s), and synchronizing the service data to the corresponding target client. Alternatively, the preset time may be set by receiving a setting instruction. Optionally, before the automatically reading the data of the service width table at preset intervals and synchronizing the data of the service width table to the target client, the method further includes: and receiving an automatic updating request of a target client, wherein the automatic updating request comprises service data information and preset time information required by the target client.
The data of the service wide table can be automatically synchronized to the target client at preset time intervals, so that the target client can timely and automatically obtain the latest service data, and the real-time performance of data synchronization is improved.
In the embodiment of the invention, the change of the target data is monitored and updated to the service width table through the data transmission service DTS, and the data is synchronized to the target client through reading the service width table, so that the data synchronization can be realized by avoiding directly performing query reading operation on the target database, and the process of realizing the data synchronization through the reading operation is separated from the target database, thereby realizing the data read-write separation, reducing the pressure on the target server and improving the data synchronization efficiency.
Example two:
fig. 2 shows a schematic flow chart of a second data synchronization method provided in an embodiment of the present application, which is detailed as follows:
in step S201, at least one target data table of at least one target database is subscribed through a data transmission service DTS, where the target data table is used for storing target data.
The execution subject of the embodiment of the present application is a server, and the server is hereinafter referred to as a synchronization server in order to be distinguished from a target server. The synchronization server subscribes to one or more target data tables of one or more target databases through a Data Transmission Service (DTS), wherein the target data tables contain target data. Specifically, the DTS platform subscribes a target data table of a target database, and an interface function of the DTS platform is added into a program of the synchronization server, so that the synchronization server can subscribe to monitor data through the DTS platform; or after the DTS platform subscribes the target data table of the target database, the synchronization server accesses the DTS platform in a wired or wireless mode so that the synchronization server can subscribe the monitored data through the DTS platform; or, subscribing a target data table of a target database on the DTS platform, and setting the synchronization server as the monitoring server corresponding to the target database through setting so that the synchronization server can subscribe the monitored data through the DTS platform.
In step S202, a service width table is constructed according to the target data table.
The target data table is a data table storing target data, and generally, since the service data is numerous, the target data specifically includes various service data stored in different target data tables of a plurality of different target databases. In different target data tables, data of the same object is usually identified by a uniform ID, for example, in different target data tables for counting user service data, a user ID exists in each target data table for identifying service data corresponding to the same user. According to the common ID of each target data table, data of different target data tables can be integrated, all target data corresponding to the same ID and from different target data tables are integrated into a row, the target data corresponding to all IDs are finally integrated, a service width table containing all the target data is constructed and stored in the synchronous server.
In step S203, if it is monitored through the data transmission service DTS that the target data of the target database is changed, the changed target data is updated to the service width table, where the target database is stored in the target server.
Specifically, the step S203 includes:
s203a 1: and if the change of the target data of the target database is monitored through the data transmission service DTS, generating a corresponding database operation statement according to the changed target data.
When the data transmission service DTS monitors that the target data of the target database changes, the DTS sends monitoring information to the synchronous server, wherein the monitoring information comprises database storage position information and target data updating value information corresponding to the changed target data, and the database storage position information corresponding to the target data indicates that the changed target data is specifically the target data in which row and column of which target data table of which target database is located. The synchronization server receives the monitoring information, analyzes that the changed target data is located at a specific row position of the service width table according to the monitoring information and the association relationship between the target data table and the service width table (for example, the corresponding relationship of the column name of the target data table in the service width table), and generates a database operation statement according to the specific row position information and the target data update value information in the monitoring information. For example, if it is detected that the changed target data is specifically located in the row of the target data table a with the user ID "10" and in the column named "age", the changed value of the target data is 28, and therefore the database storage location information for locating the target data is (table: a, row: where ID is 10, column: age), and the updated value of the target data is 28, and the monitored information is obtained. Generating a database operation statement according to the monitoring information and the association relationship between the target data table a and the service width table Widetable prestored in the synchronization server, for example, the column name of the age column in the target data table a in the service width table Widetable is age 1: the uedate consumable set 1-28 where ID-10.
S203a 2: and operating the service width table according to the database operation statement, and updating the changed target data to the service width table.
And operating the service width table according to the database operation statement, positioning the position of the changed target data corresponding to the service width table, and updating the data of the position into a target data updating value, thereby realizing the updating of the changed target data to the service width table.
In step S204, the data of the service width table is read, and the data of the service width table is synchronized to the target client.
In this embodiment, S204 is the same as S102 in the previous embodiment, and please refer to the related description of S102 in the previous embodiment, which is not repeated herein.
In the embodiment of the invention, the target data of the target data table is subscribed through the data transmission service DTS, the service width table is constructed according to the target data table, the service width table is updated when the change of the target data is monitored, and the data is synchronized to the target client by reading the service width table, so that the data synchronization can be realized by avoiding directly inquiring and reading the target database, and the process of realizing the data synchronization through the reading operation is separated from the target database, thereby realizing the data reading and writing separation, reducing the pressure on the target server and improving the data synchronization efficiency.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Example three:
fig. 3 shows a schematic structural diagram of a data synchronization apparatus provided in an embodiment of the present application, and for convenience of description, only parts related to the embodiment of the present application are shown:
the data synchronization apparatus includes: a listening unit 31, a synchronization unit 32. Wherein:
the monitoring unit 31 is configured to update the changed target data to the service bandwidth table if it is monitored through the data transmission service DTS that the target data of the target database is changed, where the target database is stored in the target server.
Data Transmission Service (DTS) is a Data cloud service, and can implement Data streaming service for Data interaction between multiple Data sources, and can monitor Data changes of subscribed databases through DTS subscription.
The execution subject of the embodiment of the present application is a server, and the server is hereinafter referred to as a synchronization server in order to be distinguished from a target server. The target database is a database which is monitored by the synchronization server in real time through DTS, wherein one or more target databases are stored in one or more target servers, and the target servers are different from the synchronization server. Specifically, various service data are stored in the target database, and the synchronization server monitors specified service data, i.e., target data, in the target database in real time by means of the DTS.
The service data changes with the change of specific services, such as the operation of a client or the change of monitoring data of a sensor, so that the target server storing each service data writes new service data into a target database in the target server along with the change of the services, so as to realize real-time update. When the target data is changed due to update writing of services, such as operations of adding, modifying, deleting and the like to the data, the DTS platform monitors the change of the target data or monitors the update writing operation of the target database in the target data server, and reports the changed target data information to the synchronization server, that is, the synchronization server can monitor whether the target data of the target database is changed through the DTS. When the synchronous server monitors that the target data of the target database is changed through the DTS, the synchronous server updates the changed target data to the service width table stored in the synchronous server. The service width table is specifically a data table including each service data (i.e., target data of each target database), and the target data of each target data table originally dispersed in each target database is collected in the service width table.
Optionally, the database type of the target database includes any one or more of MySQL, Monogo, Oracle, and SQL Server.
Optionally, the data synchronization apparatus further includes:
the device comprises a subscription unit and a Data Transmission Service (DTS), wherein the subscription unit is used for subscribing at least one target data table of at least one target database through the DTS, and the target data table is used for storing target data.
Optionally, the data synchronization apparatus further includes:
and the construction unit is used for constructing a service width table according to the target data table.
Optionally, the monitoring unit 31 includes a database operation statement generating module and an operation module:
and the database operation statement generation module is used for generating a corresponding database operation statement according to the changed target data if the change of the target data of the target database is monitored through the data transmission service DTS.
And the operation module is used for operating the service width table according to the database operation statement and updating the changed target data to the service width table.
And the synchronization unit 32 is configured to read the data of the service width table and synchronize the data of the service width table to the target client.
The target client needs to request specific target data through the self report form need. The service width table includes each target data, and since the target data stored in the service width table is monitored through the DTS and updated in real time, that is, the target data stored in the service width table is the latest, the service width table is a data table including the most complete and latest target data. And reading the data of the service width table according to the report form requirement of the target client, synchronizing the data of the service width table to the target client, meeting the report form requirement of the target client and presenting the latest target data. Optionally, the synchronization server may simultaneously read corresponding data in the service wide table according to different report requirements of the multiple target clients, and synchronize the data required by each client to the report of each target client.
When the target data is obtained through a single service wide table, the target data can be read only through simple database query statements, and the target data can be obtained without the mode of obtaining the target data from original different target databases and target data tables through multiple layers and query statements, so that the traversal query of data statements can be reduced, dozens of or even hundreds of rows of data query statements are changed into one row, and the data reading synchronization efficiency is improved.
Optionally, the synchronization unit 32 includes:
and the first synchronization module is used for receiving a request of a target client, reading the data of the service width table and synchronizing the data of the service width table to the target client.
Optionally, the synchronization unit 32 includes:
and the second synchronization module is used for automatically reading the data of the service width table at preset time intervals and synchronizing the data of the service width table to the target client.
In the embodiment of the invention, the change of the target data is monitored and updated to the service width table through the data transmission service DTS, and the data is synchronized to the target client through reading the service width table, so that the data synchronization can be realized by avoiding directly performing query reading operation on the target database, and the process of realizing the data synchronization through the reading operation is separated from the target database, thereby realizing the data read-write separation, reducing the pressure on the target server and improving the data synchronization efficiency.
Example four:
fig. 4 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 4, the terminal device 4 of this embodiment includes: a processor 40, a memory 41 and a computer program 42, such as a data synchronization program, stored in said memory 41 and operable on said processor 40. The processor 40 implements the steps in the above-described embodiments of the data synchronization method, such as the steps S101 to S102 shown in fig. 1, when executing the computer program 42. Alternatively, the processor 40, when executing the computer program 42, implements the functions of the modules/units in the above-mentioned device embodiments, such as the functions of the units 31 to 32 shown in fig. 3.
Illustratively, the computer program 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 42 in the terminal device 4. For example, the computer program 42 may be divided into a listening unit and a synchronization unit, and each unit has the following specific functions:
And the monitoring unit is used for updating the changed target data to the service width table if the change of the target data of the target database is monitored through the data transmission service DTS, wherein the target database is stored in the target server.
And the synchronization unit is used for reading the data of the service width table and synchronizing the data of the service width table to the target client.
The terminal device 4 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 40, a memory 41. Those skilled in the art will appreciate that fig. 4 is merely an example of a terminal device 4 and does not constitute a limitation of terminal device 4 and may include more or fewer components than shown, or some components may be combined, or different components, e.g., the terminal device may also include input-output devices, network access devices, buses, etc.
The Processor 40 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or a memory of the terminal device 4. The memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal device 4. The memory 41 is used for storing the computer program and other programs and data required by the terminal device. The memory 41 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (10)
1. A method of data synchronization, comprising:
if the change of the target data of the target database is monitored through the data transmission service DTS, the changed target data is updated to a service width table, wherein the target database is stored in a target server;
and reading the data of the service width table, and synchronizing the data of the service width table to a target client.
2. The data synchronization method of claim 1, wherein the database type of the target database comprises any one or more of MySQL, Monogo, Oracle, and SQL Server.
3. The data synchronization method of claim 1, wherein before the updating the changed target data to the traffic width table if the change of the target data of the target database is monitored through the data transfer service DTS, the method further comprises:
subscribing to at least one target data table of at least one target database through a Data Transfer Service (DTS), wherein the target data table is used for storing target data.
4. The data synchronization method of claim 3, wherein before the updating the changed target data to the traffic width table if the change of the target data of the target database is monitored through the data transfer service DTS, the method comprises:
and constructing a service width table according to the target data table.
5. The data synchronization method of claim 1, wherein if it is monitored through the data transfer service DTS that the target data of the target database is changed, updating the changed target data to the traffic width table comprises:
if the change of the target data of the target database is monitored through the data transmission service DTS, generating a corresponding database operation statement according to the changed target data;
and operating the service width table according to the database operation statement, and updating the changed target data to the service width table.
6. The data synchronization method of claim 1, wherein the reading the data of the service width table and synchronizing the data of the service width table to a target client comprises:
and receiving a request of a target client, reading the data of the service width table, and synchronizing the data of the service width table to the target client.
7. The data synchronization method of claim 1, wherein the reading the data of the service width table and synchronizing the data of the service width table to a target client comprises:
and automatically reading the data of the service width table at preset time intervals, and synchronizing the data of the service width table to a target client.
8. A data synchronization apparatus, comprising:
the monitoring unit is used for updating the changed target data to the service width table if the change of the target data of the target database is monitored through the data transmission service DTS, wherein the target database is stored in the target server;
and the synchronization unit is used for reading the data of the service width table and synchronizing the data of the service width table to the target client.
9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910438796.1A CN111984723A (en) | 2019-05-24 | 2019-05-24 | Data synchronization method and device and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910438796.1A CN111984723A (en) | 2019-05-24 | 2019-05-24 | Data synchronization method and device and terminal equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111984723A true CN111984723A (en) | 2020-11-24 |
Family
ID=73436095
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910438796.1A Pending CN111984723A (en) | 2019-05-24 | 2019-05-24 | Data synchronization method and device and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111984723A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112765277A (en) * | 2021-01-28 | 2021-05-07 | 树根互联股份有限公司 | Data synchronization method, device and system |
| CN113760836A (en) * | 2021-01-27 | 2021-12-07 | 北京京东振世信息技术有限公司 | Wide table calculation method and device |
| CN114048255A (en) * | 2021-10-27 | 2022-02-15 | 北京金山云网络技术有限公司 | Data calling method and device, storage medium and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101661491A (en) * | 2008-08-31 | 2010-03-03 | 阿里巴巴集团控股有限公司 | Updating method of wide table inside data warehouse |
| CN108287886A (en) * | 2018-01-16 | 2018-07-17 | 阿里巴巴集团控股有限公司 | The method and device of synchrodata modification information |
| CN108596561A (en) * | 2018-03-29 | 2018-09-28 | 客如云科技（成都）有限责任公司 | A kind of people's effect service system and method based on big data framework |
-
2019
- 2019-05-24 CN CN201910438796.1A patent/CN111984723A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101661491A (en) * | 2008-08-31 | 2010-03-03 | 阿里巴巴集团控股有限公司 | Updating method of wide table inside data warehouse |
| CN108287886A (en) * | 2018-01-16 | 2018-07-17 | 阿里巴巴集团控股有限公司 | The method and device of synchrodata modification information |
| CN108596561A (en) * | 2018-03-29 | 2018-09-28 | 客如云科技（成都）有限责任公司 | A kind of people's effect service system and method based on big data framework |
Non-Patent Citations (1)
| Title |
|---|
| 胡可云 等: "《数据挖掘理论与应用》", 30 April 2008 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113760836A (en) * | 2021-01-27 | 2021-12-07 | 北京京东振世信息技术有限公司 | Wide table calculation method and device |
| CN113760836B (en) * | 2021-01-27 | 2024-04-12 | 北京京东振世信息技术有限公司 | Wide table calculation method and device |
| CN112765277A (en) * | 2021-01-28 | 2021-05-07 | 树根互联股份有限公司 | Data synchronization method, device and system |
| CN114048255A (en) * | 2021-10-27 | 2022-02-15 | 北京金山云网络技术有限公司 | Data calling method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109947668B (en) | Method and device for storing data | |
| CN111198859B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN107977396B (en) | Method and device for updating data table of KeyValue database | |
| CN110532322B (en) | Operation and maintenance interaction method, system, computer readable storage medium and equipment | |
| AU2014209697A1 (en) | Method and system for using a recursive event listener on a node in hierarchical data structure | |
| CN111984723A (en) | Data synchronization method and device and terminal equipment | |
| CN111143382A (en) | Data processing method, system and computer readable storage medium | |
| CN112948498A (en) | Method and device for generating global identification of distributed system | |
| CN112306700A (en) | Abnormal RPC request diagnosis method and device | |
| CN112269830A (en) | Big data analysis method, system, computer equipment and storage medium thereof | |
| CN113672627A (en) | Elasticissearch search engine index construction method and device | |
| CN111343267A (en) | Configuration management method and system | |
| CN111338834A (en) | Data storage method and device | |
| CN113220342A (en) | Centralized configuration method and device, electronic equipment and storage medium | |
| CN110727694B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN111274104B (en) | Data processing method, device, electronic equipment and computer readable storage medium | |
| CN112148705A (en) | Data migration method and device | |
| CN111274032A (en) | Task processing system and method, and storage medium | |
| CN115982148A (en) | Database table processing method and device, electronic equipment and storage medium | |
| CN114756173A (en) | Method, system, device and computer readable medium for file merging | |
| CN113076343A (en) | Data query method, device, equipment and storage medium | |
| CN112235332B (en) | Method and device for switching reading and writing of clusters | |
| CN109902067B (en) | File processing method and device, storage medium and computer equipment | |
| US11226882B2 (en) | Method and device for data center management | |
| CN113779048A (en) | Data processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201124 |