CN111797167B - Data synchronization system and method and data processing center - Google Patents
Data synchronization system and method and data processing center Download PDFInfo
- Publication number
- CN111797167B CN111797167B CN202010625437.XA CN202010625437A CN111797167B CN 111797167 B CN111797167 B CN 111797167B CN 202010625437 A CN202010625437 A CN 202010625437A CN 111797167 B CN111797167 B CN 111797167B
- Authority
- CN
- China
- Prior art keywords
- data
- processor
- security domain
- central
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Mathematical Physics (AREA)
- Hardware Redundancy (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides a data synchronization system, a data synchronization method and a data processing center, wherein the system comprises the following components: a data processing center and a plurality of security domains; the data processing center comprises a security domain center node and a plurality of center front-end processors; the security domain central node is configured to receive, from a first central front-end processor of the data processing center, data to be synchronized from a first database of the first security domain sent by the first boundary front-end processor of the first security domain, and send the data to be synchronized to a second central front-end processor of the data processing center, so that the second central front-end processor sends the data to be synchronized to a second database of the second security domain through a second boundary front-end processor of the second security domain, and destination database information carried in the data to be synchronized points to the second database of the second security domain. The data processing center is in star connection with each security domain, so that the configuration of a front-end processor and the configuration of a transmission process can be reduced, the maintenance workload is reduced, and the expansion is convenient.
Description
Technical Field
The embodiment of the invention relates to the technical field of security domains, in particular to a data synchronization system and method and a data processing center.
Background
For security management needs, security domain partitioning is often required: and drawing all databases with the same security level and the same security requirement into the same network segment, and performing access control at the boundary of the network segment. The access principle between different security domains is not allowed to be directly reachable.
In the prior art, the data synchronization can be performed by adding a front-end processor between databases belonging to two security domains, and the two databases are connected through the front-end processors respectively.
However, in the case of multiple security domains, the number of front-end processors to be set is very large, so that maintenance is not easy, and the workload for performing database addition/deletion processing is also very large.
Disclosure of Invention
Embodiments of the present invention provide a data synchronization system, a method, and a data processing center to improve the first aspect, where the embodiments of the present invention provide a data synchronization system, including:
a data processing center and a plurality of security domains;
the plurality of security domains includes a first security domain and a second security domain; the first security domain comprises a first database and a first boundary front-end processor; the second security domain comprises a second database and a second boundary front-end processor;
the data processing center comprises a security domain center node and a plurality of center front-end processors; the plurality of central pre-processors comprise a first central pre-processor and a second central pre-processor; the first central pre-processor is communicated with the first boundary pre-processor, and the second central pre-processor is communicated with the second boundary pre-processor;
the security domain central node is configured to receive, from a first central pre-processor, data to be synchronized from a first database in the first security domain sent by the first boundary pre-processor, and send the data to be synchronized to the second central pre-processor, so that the second central pre-processor sends the data to be synchronized to a second database in the second security domain through the second boundary pre-processor, and destination database information carried in the data to be synchronized points to the second database in the second security domain.
In one possible design, each of the security domains includes: and a plurality of databases, each of the databases being in communication with a boundary front-end processor, each of the boundary front-end processors being in communication with one of the central front-end processors.
In one possible design, each of the security domains includes: the system comprises a plurality of databases, a central node and a boundary front-end processor, wherein each database is communicated with the boundary front-end processor through the central node, and the boundary front-end processor is communicated with one central front-end processor.
In one possible design, the data to be synchronized includes modified data or operational information of the data change.
In one possible design, the central node is further configured to receive data to be synchronized sent by a source database in the multiple databases in the first security domain, and send the data to be synchronized to a target database in the multiple databases in the first security domain, so that the target database realizes data synchronization with the source database according to the data to be synchronized.
In one possible design, the data to be synchronized is log data;
the source database is used for recording operation information of data change in log data, capturing the log data through a capturing process and sending the log data to the central node through a first transmission process;
the central node is specifically configured to receive the log data, and send the log data to a target database in the multiple databases through a second transmission process;
the target end database is used for reading data change information from the log data through a delivery process, creating standard sentences according to the data change information, executing the standard sentences and realizing data synchronization with the source end database.
In one possible design, the standard statement is an SQL statement.
In a second aspect, an embodiment of the present invention provides a data synchronization method applied to the data processing center described in the first aspect and the possible designs of the first aspect, where the method includes:
the first database sends the data to be synchronized to the first boundary front-end processor;
the first boundary front-end processor sends the data to be synchronized to a first central front-end processor;
the first central front-end processor sends the data to be synchronized to a security domain central node;
the security domain central node sends the data to be synchronized to a second central front-end processor;
the second central front-end processor sends the data to be synchronized to a second database; and the destination database information carried in the data to be synchronized points to a second database of the second security domain.
In a third aspect, an embodiment of the present invention provides a data processing center, including: a security domain central node and a plurality of central front-end processors; the plurality of central pre-processors comprise a first central pre-processor and a second central pre-processor; the first central pre-processor is in communication with a first boundary pre-processor of a first security domain, and the second central pre-processor is in communication with a second boundary pre-processor of a second security domain;
the security domain central node is configured to receive, from a first central pre-processor, data to be synchronized from a first database in the first security domain sent by the first boundary pre-processor, and send the data to be synchronized to the second central pre-processor, so that the second central pre-processor sends the data to be synchronized to a second database in the second security domain through the second boundary pre-processor, and destination database information carried in the data to be synchronized points to the second database in the second security domain.
In a fourth aspect, an embodiment of the present invention provides a data synchronization method, applied to a data processing center according to the design of the third aspect, including: the security domain central node receives data to be synchronized from a first database of the first security domain, which is sent by a first boundary front-end processor, from a first central front-end processor;
the security domain central node sends the data to be synchronized to the second central front-end processor, so that the second central front-end processor sends the data to be synchronized to a second database of the second security domain through the second boundary front-end processor, and destination database information carried in the data to be synchronized points to the second database of the second security domain.
The data synchronization system, the data synchronization method and the data processing center provided by the embodiment, wherein the system comprises: a data processing center and a plurality of security domains; the plurality of security domains includes a first security domain and a second security domain; the first security domain comprises a first database and a first boundary front-end processor; the second security domain comprises a second database and a second boundary front-end processor; the data processing center comprises a security domain center node and a plurality of center front-end processors; the plurality of central pre-processors comprise a first central pre-processor and a second central pre-processor; the first central pre-processor is communicated with the first boundary pre-processor, and the second central pre-processor is communicated with the second boundary pre-processor; the security domain central node is configured to receive, from a first central pre-processor, data to be synchronized from a first database in the first security domain sent by the first boundary pre-processor, and send the data to be synchronized to the second central pre-processor, so that the second central pre-processor sends the data to be synchronized to a second database in the second security domain through the second boundary pre-processor, and destination database information carried in the data to be synchronized points to the second database in the second security domain. According to the data processing system provided by the embodiment, the data processing center is arranged, so that the data processing center and each security domain form star connection, when data synchronization is carried out between different security domains, the data processing center can be used as a medium, and compared with the prior art, the data processing system has the advantages that the front-end processor and the transmission process are required to be independently configured between any two security domain databases, the configuration of the front-end processor and the transmission process can be greatly reduced, the maintenance workload is reduced, and the security domains are convenient to expand.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it will be obvious that the drawings in the following description are some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a schematic diagram of a prior art inter-security domain data synchronization system;
FIG. 2 is a schematic diagram of a data synchronization system according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a data synchronization system according to another embodiment of the present invention;
FIG. 4 is a schematic diagram of a data synchronization system according to another embodiment of the present invention;
fig. 5 is a flowchart of a data synchronization method according to an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
For security management needs, security domain partitioning is often required: and drawing all databases with the same security level and the same security requirement into the same network segment, and performing access control at the boundary of the network segment. The access principle between different security domains is not allowed to be directly reachable.
In the business process, data synchronization is often required between different databases, for example, when a new traffic business is created, the foreground stores accepted user data into a business database, and when billing and accounting are performed, the accounting database also needs to use the user data, so that the user data needs to be synchronized from the business database to the accounting database. If two databases requiring data synchronization belong to different security domains, data replication across the security domains is required.
In the prior art, the data synchronization can be performed by adding a front-end processor between databases belonging to two security domains, and the two databases are connected through the front-end processors respectively. Fig. 1 is a schematic diagram of a prior art inter-security domain data synchronization system, and as shown in fig. 1, a first security domain 10 and a second security domain 20 respectively include 4 databases. Direct data transmission can be performed between 4 databases in the first security domain 10, but in order to ensure data security, data transmission between databases belonging to different security domains needs to be performed by deploying a front-end processor. Taking the database 11 in the first security domain 10 as a source database and the database 21 in the second security domain 20 as a destination database as an example, when performing data synchronization across the security domains, the database 11 needs to be connected to the front end processor A1 and a transfer process from the database 11 to the front end processor A1 is deployed, the database 21 needs to be connected to the front end processor B1 and a transfer process from the front end processor A1 to the front end processor B1 is deployed, and a transfer process from the front end processor B1 to the database 21 is deployed. Up to this point, data transmission from the database 11 to the database 21 can be realized. Correspondingly, if the database 11 is used as the source database and the database 22 is used as the destination database, two front-end processors are added and three transmission processes are deployed to realize the data transmission from the database 11 to the database 22. That is, to realize data transmission between the 4 databases in the first security domain and the 4 databases in the second security domain, 32 front-end processors need to be provided, and 32×3=96 transmission processes are deployed.
Therefore, when the number of security domains is increased, the need for adding a front-end processor and a transmission process is greatly increased. Based on the above, the embodiment of the invention provides a data synchronization system to improve the accuracy of data synchronization.
In the present embodiment, a data processing center is provided. A plurality of security domains are connected with the data processing center in a star shape. When data synchronization is carried out between different security domains, the data processing center can be used as a medium, and compared with the prior art that the front-end processor and the transmission process are required to be configured independently between any two security domains, the configuration of the front-end processor and the transmission process can be greatly reduced, the maintenance workload is reduced, and the security domains are convenient to expand.
The technical scheme of the invention is described in detail below by specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.
Fig. 2 is a schematic structural diagram of a data synchronization system according to an embodiment of the invention. As shown in fig. 2, the system includes:
a data processing center 200, and a plurality of security domains.
The plurality of security domains includes a first security domain 110 and a second security domain 120; the first security domain 110 includes a first database and a first boundary-layer; the second security domain 120 includes a second database and a second boundary capper.
The data processing center 200 comprises a security domain central node and a plurality of central front-end processors; the plurality of central pre-processors comprise a first central pre-processor and a second central pre-processor; the first central pre-processor communicates with the first boundary pre-processor, and the second central pre-processor communicates with the second boundary pre-processor.
The security domain central node is configured to receive, from a first central pre-processor, data to be synchronized from a first database of the first security domain 110 sent by the first boundary pre-processor, and send the data to be synchronized to the second central pre-processor, so that the second central pre-processor sends the data to be synchronized to a second database of the second security domain 120 through the second boundary pre-processor, and destination database information carried in the data to be synchronized points to the second database of the second security domain 120.
Optionally, the data to be synchronized includes modified data or operation information of data change. Specifically, the original data in the source database is changed through a series of adding, deleting and changing operations, and the data to be synchronized can be changed data or a series of adding, deleting and changing operation information, namely the operation information of data change.
In practical application, a database in the first security domain 110 is taken as a source database, a database in the second security domain 120 is taken as a destination database, a data synchronization process between the source database and the destination database is described by taking an example, the source database sequentially passes through a first boundary front end processor of the first security domain 110, a first center front end processor of the data processing center 200, a security domain central node of the data processing center 200, a second center front end processor of the data processing center 200 and a second boundary front end processor of the second security domain 120, and is communicated with the destination database of the second security domain 120, specifically, a transmission process between the source database and the first boundary front end processor, and a transmission process between the first boundary front end processor and the first center front end processor, a transmission process between the first center front end processor and the security domain central node, a transmission process between the second center front end processor and the second boundary front end processor, and the destination database are deployed, and the data is synchronized from the destination database to the destination database is completed.
According to the data synchronization system provided by the embodiment, by arranging the data processing center 200, when the data processing center 200 is in star connection with each security domain and data synchronization is performed between different security domains, the data processing center 200 can be used as a medium, and compared with the prior art that a front end processor and a transmission process are required to be configured between databases of any two security domains, the configuration of the front end processor and the configuration of the transmission process can be greatly reduced, the maintenance workload is reduced, and the security domains are convenient to expand.
Fig. 3 is a schematic structural diagram of a data synchronization system according to another embodiment of the present invention. As shown in fig. 3, the internal architecture of each security domain is described in detail in this embodiment based on the embodiment shown in fig. 2, and each security domain includes: and a plurality of databases, each of the databases being in communication with a boundary front-end processor, each of the boundary front-end processors being in communication with one of the central front-end processors.
In practical applications, a boundary front-end processor is configured for each database in each security domain, and correspondingly, a central front-end processor is configured for each boundary front-end processor in the security domain in the data processing center 200. The following describes, by taking a database 113 in the first security domain 110 as a source database and taking a database 123 in the second security domain 120 as a destination database as an example, a data synchronization process between the source database and the destination database, where the source database sequentially passes through the first boundary pre-processor 112 of the first security domain 110, the first central pre-processor 202 of the data processing center 200, the security domain core node 201 of the data processing center 200, the second central pre-processor 203 of the data processing center 200, and the second boundary pre-processor 122 of the second security domain 120, and is in communication with the destination database of the second security domain 120, specifically, a transfer process between the source database and the first boundary pre-processor 112 and a transfer process between the first boundary pre-processor 112, a transfer process between the first central pre-processor 202 and the security domain core node 201, a transfer process between the security domain core node 201 and the second central pre-processor 203, a transfer process between the second boundary pre-processor 203 and the second boundary pre-processor 122, and a data synchronization process between the destination database and the destination pre-processor 122 are performed, and the data synchronization process is completed by deploying the transfer processes between the source database and the destination pre-processor and the destination database.
In general, in this embodiment, since the front-end processor in the data processing center 200 and the transmission processes between the front-end processor and the central node can be multiplexed, only 16 front-end processors and 24 transmission processes need to be provided. Compared with the 32 front-end processors and 96 transmission processes which are required to be arranged in the prior art shown in fig. 1, the configuration of the front-end processors and the deployment of the transmission processes can be greatly reduced, the cost is saved, and the workload is reduced. And when the security domain needs to be newly added, the front-end processor which needs to be added is also greatly reduced compared with the prior art.
According to the data synchronization system provided by the embodiment, by arranging the data processing center 200, when the data processing center 200 is in star connection with each security domain and data synchronization is performed between different security domains, the data processing center 200 can be used as a medium, and compared with the prior art that a front end processor and a transmission process are required to be configured between databases of any two security domains, the configuration of the front end processor and the configuration of the transmission process can be greatly reduced, the maintenance workload is reduced, and the security domains are convenient to expand.
Fig. 4 is a schematic structural diagram of a data synchronization system according to another embodiment of the present invention. As shown in fig. 4, the internal architecture of each security domain is described in detail in this embodiment based on the embodiment shown in fig. 2, where each security domain includes: the system comprises a plurality of databases, a central node and a boundary front-end processor, wherein each database is communicated with the boundary front-end processor through the central node, and the boundary front-end processor is communicated with one central front-end processor.
In practical applications, a central node may be configured for a database in each security domain, such that the central node in each security domain is connected with the databases in the security domain in a star-type. Taking a database 113 in the first security domain 110 as a source database, taking a database 123 in the second security domain 120 as a destination database as an example, describing a data synchronization process between the source database and the destination database, the source database sequentially passes through the central node 111 of the first security domain 110, the first border pre-processor 112, the first central pre-processor 202 of the data processing center 200, the security domain central node 201 of the data processing center 200, the second central pre-processor 203 of the data processing center 200 and the second border pre-processor 122 of the second security domain 120, the central node 121, and a transfer process between the central node 111 and the first border pre-processor 112, and a transfer process between the first border pre-processor 112 and the first central pre-processor 202, a transfer process between the first central pre-processor 202 and the security domain central node 201, a transfer process between the second border pre-processor 203 of the security domain central node 203 and the second security domain 120, and a transfer process between the second border pre-processor 203 of the second security domain 120, and the second border pre-processor 122, and a transfer process between the central node 111 and the second border pre-processor 122, and a transfer process between the second border pre-processor and a transfer process is deployed.
In general, in this embodiment, a central node is configured for each security domain, and if each database in the security domain is star-connected to the central node, the front-end processor in the security domain can be multiplexed, and only 4 front-end processors and 14 transmission processes need to be set in total. Compared with the 32 front-end processors and 96 transmission processes which are required to be arranged in the prior art shown in fig. 1, the configuration of the front-end processors and the deployment of the transmission processes can be greatly reduced, the cost is saved, and the workload is reduced. And when the security domain needs to be newly added, the front-end processor which needs to be added is also greatly reduced compared with the prior art.
Optionally, the central node is further configured to receive data to be synchronized sent by a source database in the multiple databases in the first security domain 110, and send the data to be synchronized to a target database in the multiple databases in the first security domain 110, so that the target database realizes data synchronization with the source database according to the data to be synchronized.
Optionally, the data to be synchronized is log data;
the source database is used for recording operation information of data change in log data, capturing the log data through a capturing process and sending the log data to the central node through a first transmission process;
the central node is specifically configured to receive the log data, and send the log data to a target database in the multiple databases through a second transmission process;
the target end database is used for reading data change information from the log data through a delivery process, creating standard sentences according to the data change information, executing the standard sentences and realizing data synchronization with the source end database.
Optionally, the standard statement is an SQL statement.
Taking the data to be synchronized as log data as an example, the data synchronization process is described below. In the concrete implementation process:
firstly, the Log information (such as Online Redo Log Online redox Log or Archive Log) is read at the source database by using the capture Process (Extract Process), then is analyzed, only the data changes such as adding, deleting and modifying operations are extracted, and the related information is converted into an OGG (Oracle GoldenGate, oracle database) to be stored in a local queue file (local trail file) in a self-defined intermediate format. The queue file is transferred to the central node of the first security domain 110 via TCP/IP using a transfer Process (Pump Process). The queue file corresponds to log data.
In the second step, the central node of the first security domain 110 receives the data change and buffers it in the remote queue file (Remote trail file), and waits for the transfer Process (Pump Process) to read the data.
A third step, a transfer Process (Pump Process), transfers the queue file to the boundary front-end processor of the first security domain 110 via TCP/IP.
Fourth, the boundary pre-processor of the first security domain 110 receives the data change and buffers it in the remote queue file (Remote trail file) to wait for the transfer Process (Pump Process) to read the data.
A fifth step, a transmission Process (Pump Process), transmits the queue file to the boundary front-end processor of the data processing center 200 and the first security domain 110 through TCP/IP.
In the sixth step, the data processing center 200 and the boundary front end processor of the first security domain 110 receive the data change and buffer it in the remote queue file (Remote trail file), and wait for the transfer Process (Pump Process) to read the data.
A seventh step, a transmission Process (Pump Process) transmits the queue file to the security domain central node of the data processing center 200 through TCP/IP.
Eighth, the security domain central node of the data processing center 200 receives the data changes and buffers them in the remote queue file (Remote trail file), and waits for the transmission Process (Pump Process) to read the data.
A ninth step, a transmission Process (Pump Process) transmits the queue file to the boundary front-end processor of the data processing center 200 and the second security domain 120 through TCP/IP.
Tenth, the boundary pre-processor of the data processing center 200 and the second security domain 120 receives the data change and buffers it into the remote queue file (Remote trail file) to wait for the transfer Process (Pump Process) to read the data.
An eleventh step, a transfer Process (Pump Process), transfers the queue file to the boundary front-end processor of the second security domain 120 via TCP/IP.
In the twelfth step, the boundary pre-processor of the second security domain 120 receives the data change and buffers it in the remote queue file (Remote trail file), and waits for the transfer Process (Pump Process) to read the data.
A thirteenth, transfer Process (Pump Process) transfers the queue file over TCP/IP to the central node of the second security domain 120.
In a fourteenth step, the central node of the second security domain 120 receives the data change buffer in the remote queue file (Remote trail file) and waits for the transfer Process (Pump Process) to read the data.
A fifteenth, transfer Process (Pump Process) transfers the queue file over TCP/IP to the destination database of the second security domain 120.
Sixteenth, the destination database receives the data change and caches the data change in the remote queue file (Remote trail file), and waits for a delivery process (reply process) to read the data.
Seventeenth, a delivery process (replying process) reads the data change from the queue and creates a corresponding SQL sentence, and the SQL sentence is executed through a local interface of the database and submitted to a destination database.
According to the data synchronization system provided by the embodiment, the central node is arranged in the security domain, so that the central node is in star connection with all databases in the security domain, different databases can be communicated with the boundary front-end processor of the security domain through the central node, and when data synchronization is carried out between different security domains, the front-end processor and the transmission process are required to be configured independently relative to the databases in any two security domains in the prior art by taking the data processing center 200 as a medium, the configuration of the front-end processor and the configuration of the transmission process can be greatly reduced, the maintenance workload is reduced, and the expansion of the security domains is facilitated.
Specifically, the data synchronization system provided in this embodiment has the following advantages:
first, star type simple structure: the network structure is also called as the connection between the irregular nodes, is random and irregular, and the main advantage of the network topology is that the system has high reliability, but the structure is complex, and the management and maintenance are difficult. The star structure refers to that all database nodes in the network are connected to one central node, all data transmission and information exchange are completed through the central node, the structure is simple, the connection is convenient, the management and maintenance are relatively easy, the expansibility is strong, and the reliability of the system can be ensured by deploying the main and standby double central nodes.
Secondly, the maintenance workload of the network access control strategy is small: the newly added nodes in the mesh structure need to add access control strategies of the newly added nodes of the database and all the original nodes, and the newly added quantity is related to the number N of the original nodes, namely 2N. The newly added nodes in the star structure only need to add the access strategy between the newly added nodes and the central node, and the newly added quantity is irrelevant to the number N of the original nodes, namely, only 2
Third, add background capture process EP (Extract Process) for small: the new node in the mesh structure needs to add the background capturing process EP of the new node and all the original nodes, and the new increment is related to the number N of the original nodes, namely 2N. The newly added nodes in the star structure only need to adjust the access strategy between the newly added nodes and the central node, and the newly added quantity is irrelevant to the number N of the original nodes, namely, only 2
Fourth, data replication across security domains may be implemented: data replication between nodes in a mesh structure is directly abutted, and is limited because access across security domains is not allowed to be directly reachable. Under the general principle of violating the management of the security domain, the data replication in the security domain adopts a star structure, the data replication among the security domains adopts a mode of a central node and a boundary front end processor, and the data replication is realized by adopting a full star structure with the central security domain as the center.
Fifth, the system has small exposed surface, which is convenient for centralized safety protection: since two nodes of any database in a mesh structure can access each other, once one node suffers a security event, it can quickly spread to other nodes. In the star structure, only the nodes and the central node can access each other in the access strategy, so that the central node is well protected in a centralized way, and one node can be prevented from being subjected to a security event and spreading to other nodes.
Fig. 5 is a flowchart of a data synchronization method according to an embodiment of the invention. Applied to the data synchronization system shown in fig. 2 to 4, as shown in fig. 5, the method includes:
501. the first database sends the data to be synchronized to the first boundary front-end processor.
502. And the first boundary front-end processor sends the data to be synchronized to the first central front-end processor.
503. And the first central front-end processor sends the data to be synchronized to a security domain central node.
504. And the security domain central node transmits the data to be synchronized to a second central front-end processor.
505. And the second central front-end processor sends the data to be synchronized to a second database. And the destination database information carried in the data to be synchronized points to a second database of the second security domain.
According to the data synchronization method, star connection is formed through the security domains of the data processing center, when data synchronization is carried out between different security domains, the data processing center can be used as a medium, and compared with the prior art that a front end processor and a transmission process are required to be configured between databases of any two security domains, the configuration of the front end processor and the configuration of the transmission process can be greatly reduced, maintenance workload is reduced, and the security domains are convenient to expand.
The data synchronization method provided by the embodiment of the invention can be applied to the data synchronization system embodiment, and the implementation principle and the technical effect are similar, and the embodiment is not repeated here.
The embodiment of the invention also provides a data processing center, which comprises: a security domain central node and a plurality of central front-end processors; the plurality of central pre-processors comprise a first central pre-processor and a second central pre-processor; the first central pre-processor communicates with a first boundary pre-processor of a first security domain and the second central pre-processor communicates with a second boundary pre-processor of a second security domain.
The security domain central node is configured to receive, from a first central pre-processor, data to be synchronized from a first database in the first security domain sent by the first boundary pre-processor, and send the data to be synchronized to the second central pre-processor, so that the second central pre-processor sends the data to be synchronized to a second database in the second security domain through the second boundary pre-processor, and destination database information carried in the data to be synchronized points to the second database in the second security domain.
The data processing center provided by the embodiment forms star connection with each security domain, when data synchronization is performed between different security domains, the data processing center can be used as a medium, and compared with the prior art, the data processing center has the advantages that the front-end processor and the transmission process are required to be configured independently between databases of any two security domains, the configuration of the front-end processor and the transmission process can be greatly reduced, the maintenance workload is reduced, and the security domains are convenient to expand.
The data processing center provided by the embodiment of the present invention is similar to the data processing center in the embodiment of the data synchronization system, and its implementation principle and technical effects are similar, and the embodiment is not repeated here.
The embodiment of the invention also provides a data synchronization method, which is applied to the data processing center described in the above embodiment, and comprises the following steps:
and the security domain central node receives data to be synchronized from a first database of the first security domain, which is sent by the first boundary front-end processor, from a first central front-end processor.
The security domain central node sends the data to be synchronized to the second central front-end processor, so that the second central front-end processor sends the data to be synchronized to a second database of the second security domain through the second boundary front-end processor, and destination database information carried in the data to be synchronized points to the second database of the second security domain.
According to the data synchronization method, star connection is formed through the security domains of the data processing center, when data synchronization is carried out between different security domains, the data processing center can be used as a medium, and compared with the prior art that a front end processor and a transmission process are required to be configured between databases of any two security domains, the configuration of the front end processor and the configuration of the transmission process can be greatly reduced, maintenance workload is reduced, and the security domains are convenient to expand.
The data synchronization method provided by the embodiment of the invention can be applied to the data synchronization system embodiment, and the implementation principle and the technical effect are similar, and the embodiment is not repeated here.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (10)
1. A data synchronization system, comprising: a data processing center and a plurality of security domains;
the plurality of security domains includes a first security domain and a second security domain; the first security domain comprises a first database and a first boundary front-end processor; the second security domain comprises a second database and a second boundary front-end processor;
the data processing center comprises a security domain center node and a plurality of center front-end processors; the plurality of central pre-processors comprise a first central pre-processor and a second central pre-processor; the first central pre-processor is communicated with the first boundary pre-processor, and the second central pre-processor is communicated with the second boundary pre-processor;
the security domain central node is configured to receive, from a first central pre-processor, data to be synchronized from a first database in the first security domain sent by the first boundary pre-processor, and send the data to be synchronized to the second central pre-processor, so that the second central pre-processor sends the data to be synchronized to a second database in the second security domain through the second boundary pre-processor, and destination database information carried in the data to be synchronized points to the second database in the second security domain.
2. The data synchronization system of claim 1, wherein each of the security domains comprises: and a plurality of databases, each of the databases being in communication with a boundary front-end processor, each of the boundary front-end processors being in communication with one of the central front-end processors.
3. The data synchronization system of claim 1, wherein each of the security domains comprises: the system comprises a plurality of databases, a central node and a boundary front-end processor, wherein each database is communicated with the boundary front-end processor through the central node, and the boundary front-end processor is communicated with one central front-end processor.
4. A data synchronization system according to any one of claims 1-3, characterized in that the data to be synchronized comprises altered data or operation information of data changes.
5. A data synchronization system according to claim 3, wherein the central node is further configured to receive data to be synchronized sent by a source database in the plurality of databases in the first security domain, and send the data to be synchronized to a target database in the plurality of databases in the first security domain, so that the target database performs data synchronization with the source database according to the data to be synchronized.
6. The data synchronization system of claim 5, wherein the data to be synchronized is log data;
the source database is used for recording operation information of data change in log data, capturing the log data through a capturing process and sending the log data to the central node through a first transmission process;
the central node is specifically configured to receive the log data, and send the log data to a target database in the multiple databases through a second transmission process;
the target end database is used for reading data change information from the log data through a delivery process, creating standard sentences according to the data change information, executing the standard sentences and realizing data synchronization with the source end database.
7. The data synchronization system of claim 6, wherein the standard statement is an SQL statement.
8. A data synchronization method, applied to the data synchronization system according to any one of claims 1 to 7, comprising:
the first database sends the data to be synchronized to the first boundary front-end processor;
the first boundary front-end processor sends the data to be synchronized to a first central front-end processor;
the first central front-end processor sends the data to be synchronized to a security domain central node;
the security domain central node sends the data to be synchronized to a second central front-end processor;
the second central front-end processor sends the data to be synchronized to a second database; and the destination database information carried in the data to be synchronized points to a second database of the second security domain.
9. A data processing center, comprising: a security domain central node and a plurality of central front-end processors; the plurality of central pre-processors comprise a first central pre-processor and a second central pre-processor; the first central pre-processor is in communication with a first boundary pre-processor of a first security domain, and the second central pre-processor is in communication with a second boundary pre-processor of a second security domain;
the security domain central node is configured to receive, from a first central pre-processor, data to be synchronized from a first database in the first security domain sent by the first boundary pre-processor, and send the data to be synchronized to the second central pre-processor, so that the second central pre-processor sends the data to be synchronized to a second database in the second security domain through the second boundary pre-processor, and destination database information carried in the data to be synchronized points to the second database in the second security domain.
10. A data synchronization method applied to the data processing center of claim 9, the method comprising:
the security domain central node receives data to be synchronized from a first database of the first security domain, which is sent by a first boundary front-end processor, from a first central front-end processor;
the security domain central node sends the data to be synchronized to the second central front-end processor, so that the second central front-end processor sends the data to be synchronized to a second database of the second security domain through the second boundary front-end processor, and destination database information carried in the data to be synchronized points to the second database of the second security domain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010625437.XA CN111797167B (en) | 2020-07-02 | 2020-07-02 | Data synchronization system and method and data processing center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010625437.XA CN111797167B (en) | 2020-07-02 | 2020-07-02 | Data synchronization system and method and data processing center |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111797167A CN111797167A (en) | 2020-10-20 |
| CN111797167B true CN111797167B (en) | 2023-05-16 |
Family
ID=72810982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010625437.XA Active CN111797167B (en) | 2020-07-02 | 2020-07-02 | Data synchronization system and method and data processing center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111797167B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1437810A (en) * | 2000-03-10 | 2003-08-20 | 黎明网络有限公司 | An information switch |
| CN101188599A (en) * | 2007-12-04 | 2008-05-28 | 四方电气(集团)有限公司 | Implementation method for load balance design of electric monitoring front system of power plant |
| CN102663661A (en) * | 2012-04-10 | 2012-09-12 | 华为技术有限公司 | Health information system |
| CN103605698A (en) * | 2013-11-06 | 2014-02-26 | 广东电子工业研究院有限公司 | Cloud database system used for distributed heterogeneous data resource integration |
| CN108897641A (en) * | 2018-06-21 | 2018-11-27 | 武汉达梦数据库有限公司 | A kind of log analysis under the active and standby environment of database services real-time synchronization system |
| CN109714325A (en) * | 2018-12-19 | 2019-05-03 | 北京奇安信科技有限公司 | A kind of one-way optical gate data transmission method, system, electronic equipment and medium |
| CN110502915A (en) * | 2019-08-30 | 2019-11-26 | 恩亿科(北京)数据科技有限公司 | A kind of method, apparatus and system of data processing |
| CN110825797A (en) * | 2019-10-25 | 2020-02-21 | 烨链（上海）科技有限公司 | Data exchange method and device |
| CN111258780A (en) * | 2020-01-15 | 2020-06-09 | 深圳市华傲数据技术有限公司 | System and method for data exchange based on front-end processor |
-
2020
- 2020-07-02 CN CN202010625437.XA patent/CN111797167B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1437810A (en) * | 2000-03-10 | 2003-08-20 | 黎明网络有限公司 | An information switch |
| CN101188599A (en) * | 2007-12-04 | 2008-05-28 | 四方电气(集团)有限公司 | Implementation method for load balance design of electric monitoring front system of power plant |
| CN102663661A (en) * | 2012-04-10 | 2012-09-12 | 华为技术有限公司 | Health information system |
| CN103605698A (en) * | 2013-11-06 | 2014-02-26 | 广东电子工业研究院有限公司 | Cloud database system used for distributed heterogeneous data resource integration |
| CN108897641A (en) * | 2018-06-21 | 2018-11-27 | 武汉达梦数据库有限公司 | A kind of log analysis under the active and standby environment of database services real-time synchronization system |
| CN109714325A (en) * | 2018-12-19 | 2019-05-03 | 北京奇安信科技有限公司 | A kind of one-way optical gate data transmission method, system, electronic equipment and medium |
| CN110502915A (en) * | 2019-08-30 | 2019-11-26 | 恩亿科(北京)数据科技有限公司 | A kind of method, apparatus and system of data processing |
| CN110825797A (en) * | 2019-10-25 | 2020-02-21 | 烨链（上海）科技有限公司 | Data exchange method and device |
| CN111258780A (en) * | 2020-01-15 | 2020-06-09 | 深圳市华傲数据技术有限公司 | System and method for data exchange based on front-end processor |
Non-Patent Citations (4)
| Title |
|---|
| A security domain isolation and data exchange system based on VMM;G. Dong 等;《2009 3rd International Conference on Signal Processing and Communication Systems》;1-5 * |
| The Data Exchange Research of E-Government Application System Basing on Spatial Information;Y. Zhu 等;《2011 International Conference on Management and Service Science》;1-5 * |
| 基于SOA的数据共享与交换平台分析与设计;金保华 等;《郑州轻工业学院学报(自然科学版)》;第26卷(第1期);102-106 * |
| 数据交换平台信息安全问题研究与实现;张秀伟;《中国优秀硕士学位论文全文数据库 信息科技辑》(第（2007）6期);I138-72 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111797167A (en) | 2020-10-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104935680B (en) | A kind of the recurrence Domain Name Service System and method of multi-layer shared buffer memory | |
| CN101183377B (en) | High availability data-base cluster based on message middleware | |
| CN104539681B (en) | The processing method of distributed GIS acceleration systems and GIS service | |
| CN113347164B (en) | Block chain-based distributed consensus system, method, device and storage medium | |
| CN104734915A (en) | Composite multiprocess and multithread multi-network concurrence dynamic simulation method | |
| CN103763155A (en) | Multi-service heartbeat monitoring method for distributed type cloud storage system | |
| CN106484713A (en) | A kind of based on service-oriented Distributed Request Processing system | |
| CN105094997A (en) | Method and system for sharing physical memory among cloud computing host nodes | |
| CN105740248B (en) | A kind of method of data synchronization, apparatus and system | |
| CN105187464A (en) | Data synchronization method, device and system in distributed storage system | |
| CN103488526A (en) | System and method for locking business resource in distributed system | |
| CN103618762A (en) | System and method for enterprise service bus state pretreatment based on AOP | |
| CN101383814B (en) | Device and method implementing data access based on connection pool | |
| CN106326372A (en) | Git central warehouse management system and control method | |
| WO2016082594A1 (en) | Data update processing method and apparatus | |
| CN110177144B (en) | Method for copying application environment by one key based on private cloud | |
| CN114374701B (en) | Transparent sharing device for sample model of multistage linkage artificial intelligent platform | |
| CN102306148A (en) | Share memory database access method | |
| CN101882290A (en) | Service integration method based on situation ontologies under internet environment | |
| CN111797167B (en) | Data synchronization system and method and data processing center | |
| CN101789963A (en) | Data synchronization system | |
| CN116668191B (en) | Internet of things application virtual gateway with data encryption convergence function | |
| CN109635161B (en) | Cross-domain allopatric collaborative design method in mass data environment | |
| CN110445859A (en) | The processing method of distributed GIS acceleration system and GIS service | |
| CN114793244A (en) | Resource processing method, device, equipment and medium for block chain |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |