CN113486030B - Real-time database synchronization method and device based on industrial protocol analysis - Google Patents
Real-time database synchronization method and device based on industrial protocol analysis Download PDFInfo
- Publication number
- CN113486030B CN113486030B CN202110724228.5A CN202110724228A CN113486030B CN 113486030 B CN113486030 B CN 113486030B CN 202110724228 A CN202110724228 A CN 202110724228A CN 113486030 B CN113486030 B CN 113486030B
- Authority
- CN
- China
- Prior art keywords
- real
- time database
- time
- data stream
- synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention discloses a real-time database synchronization method and a real-time database synchronization device based on industrial protocol analysis, wherein the real-time database synchronization method based on the industrial protocol analysis is to add one or more synchronization devices on a communication channel connecting an intranet and an extranet and synchronously update data of a real-time database arranged on the extranet side through the synchronization devices. The invention realizes the real-time database synchronization by capturing the access flow, analyzing the protocol content, synchronously updating the real-time database information and externally providing a real-time database access interface.
Description
Technical Field
The invention relates to the technical field of network information transmission. In particular to a real-time database synchronization method and a device based on industrial protocol analysis.
Background
The real-time database generally integrates a large number of industrial protocol interfaces, and can analyze and transmit various industrial protocols. The real-time database collects and stores flow information for guiding process improvement, reducing materials and increasing yield. Carry out efficient data compression and long-term historical storage to real-time, accurate real-time data, provide fast-speed real-time, historical data service simultaneously, for the managers of enterprise can be timely, comprehensive master production, the sales condition, promote resource utilization and production reliability to the core competitiveness of reinforcing enterprise. Traditional database synchronization is based on methods such as triggers and binlog query, structured data stored in a relational database are synchronized, and the synchronization methods cannot be applied to real-time databases.
Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to provide a real-time database synchronization method and apparatus based on industrial protocol analysis, which implement real-time database synchronization by capturing access traffic, analyzing protocol content, synchronously updating real-time database information, and providing a real-time database access interface to the outside.
In order to solve the technical problems, the invention provides the following technical scheme:
a real-time database synchronization method based on industrial protocol analysis is characterized in that one or more synchronization devices are added to a communication channel connecting an internal network and an external network, and the real-time database arranged on the external network side is synchronously updated through the synchronization devices.
The real-time database synchronization method based on the industrial protocol analysis comprises the following steps of synchronously updating the real-time database through synchronization equipment:
1) copying an access data stream between an intranet side client and an intranet side server to synchronous equipment, and analyzing the access data stream by the synchronous equipment;
2) acquiring the value of the measured point and the time point of the measured point analyzed according to the accessed data flow analysis result in the step 1), when the value of the measured point is the same as the value of the corresponding point of the measured point in the real-time database, not updating the real-time database, otherwise, comparing the time difference between the last updating time point of the measured point and the time point of the measured point analyzed in the time data flow analysis with a threshold, when the time difference between the last updating time point of the measured point and the time point of the measured point analyzed in the time data flow analysis is greater than or equal to the threshold, updating the real-time database, otherwise, not updating the real-time database.
In the method for synchronizing the real-time database based on the industrial protocol analysis, in step 1), when analyzing the access data stream, the analysis result of the access data stream at least includes the value of the measured point, the time point at which the measured point is analyzed, and the identification information of the data packet in which the measured point is located, where the identification information of the data packet in which the measured point is located includes the IP address of the data packet, the service port sent by the data packet, the register address of the data packet, and the value of the register.
In the real-time database synchronization method based on the industrial protocol analysis, the synchronization equipment takes the IP address of the data packet, the sending service port of the data packet and the register address of the data packet as keys (metadata) and synchronously updates the value of the register of the data packet as a key value to the real-time database.
According to the real-time database synchronization method based on the industrial protocol analysis, the synchronization equipment is arranged in an optical gate or/and a network gate which are connected with an outer network and an inner network.
The device for synchronously updating the real-time database by utilizing the real-time database synchronization method based on the industrial protocol analysis comprises data capturing equipment, synchronization equipment and the real-time database, wherein the real-time database is in communication connection with the synchronization equipment, and the synchronization equipment is in communication connection with a communication channel connecting an intranet side client and an intranet side server through the data capturing equipment.
According to the device, the intranet side client is in communication connection with the intranet side server through the switch, the data grabbing equipment is in communication connection with the switch, and the access data stream between the intranet side client and the intranet side server is copied in a mirror image mode.
According to the device, the data capturing equipment is in communication connection with the intranet side client and the intranet side server through the network bridge.
According to the device, the synchronization equipment comprises a data flow receiving module, a data flow storage module, a data flow analysis module, a data flow packaging module and a data flow sending module, and the data capturing equipment, the data flow receiving module, the data flow storage module, the data flow analysis module, the data flow packaging module, the data flow sending module and the real-time database are sequentially in communication connection.
The device and the synchronization equipment are arranged in the optical gate or/and the network gate and are in communication connection with the real-time database through the optical gate or/and the network gate.
The technical scheme of the invention achieves the following beneficial technical effects:
1. the invention can efficiently and synchronously update the data of the real-time database in real time, and abandons the defects of the traditional inquiry based on triggers and binlog.
2. Under the condition that physical isolation is arranged between the internal network and the external network, the synchronization equipment transmits real-time data to the real-time database through the network gate or the optical gate, so that real-time information of the internal network side is transmitted in a one-way mode to the external network side, and the method is safe and reliable.
3. The external network user can only check the access, thereby preventing unauthorized misoperation such as downlink control and the like.
4. And the decision system is updated, so that unnecessary synchronous updating is reduced.
ETL conversion processing, supporting SQL, NoSql, data integration operation and the like, and providing RTDB interfaces (OPCDA, OPCUA, MQTT, MODBUS and the like).
Drawings
FIG. 1 is a schematic diagram of the present invention of a device for synchronously updating a real-time database using a real-time database synchronization method based on industrial protocol analysis;
FIG. 2 is a flow chart of real-time database synchronization according to the present invention.
Detailed Description
As shown in fig. 1, an apparatus for synchronously updating a real-time database by using a real-time database synchronization method based on industrial protocol analysis includes a data capture device, a synchronization device, and a real-time database, where the real-time database is in communication connection with the synchronization device, and the synchronization device is in communication connection with a communication channel connecting an intranet-side client and an intranet-side server through the data capture device.
The data capturing equipment is in communication connection with the switch and copies access data streams between the intranet side client and the intranet side server in a mirror image mode; the synchronization equipment is arranged in the optical gate or/and the network gate and is in communication connection with the real-time database through the optical gate or/and the network gate; the real-time database provides RTDB interface service externally. The synchronous equipment comprises a data flow receiving module, a data flow storage module, a data flow analysis module, a data flow packaging module and a data flow sending module, and the data capturing equipment, the data flow receiving module, the data flow storage module, the data flow analysis module, the data flow packaging module, the data flow sending module and the real-time database are sequentially in communication connection. The data flow receiving module receives the data flow which is copied and sent by the data capturing device and sends the data flow to the data flow storage module for storage, the data flow analysis module calls the data flow from the data flow storage module for analysis and processing, the data which can be synchronously updated are sent to the data packaging module for packaging, and the packaged data are sent to the real-time database through the data flow sending module to update corresponding data. When the real-time database is updated synchronously, the data stream storage module of the synchronization device at least stores the data stream captured by the data capture device, the analysis result of the data stream analysis module and the data information updated into the real-time database latest, that is, the data stream storage module stores two data packets related to the same detected point, and the time points of the two data packets analyzed are adjacent in the process of updating the real-time database.
In the invention, the specific steps of synchronously updating the real-time database by using the device are as follows:
1) copying an access data stream between an intranet side client and an intranet side server to synchronous equipment, and analyzing the access data stream by the synchronous equipment; when the access data flow is analyzed, the analysis result of the access data flow at least comprises the value of the measured point, the time point of the analyzed measured point and the identification information of the data packet where the measured point is located, wherein the identification information of the data packet where the measured point is located comprises the IP address of the data packet, the service port sent by the data packet, the register address of the data packet and the value of the register;
2) acquiring the value of the measured point and the time point of the measured point analyzed according to the accessed data flow analysis result in the step 1), when the value of the measured point is the same as the value of the corresponding point of the measured point in the real-time database, not updating the real-time database, otherwise, comparing the time difference between the last updating time point of the measured point and the time point of the measured point analyzed in the time data flow analysis with a threshold, when the time difference between the last updating time point of the measured point and the time point of the measured point analyzed in the time data flow analysis is greater than or equal to the threshold, updating the real-time database, otherwise, not updating the real-time database. The threshold value can be set by the user, and can be reduced or increased according to the accuracy required by the user for the real-time performance. For example, let the measured point be P 1 And P is 1 Corresponding real-time database point P 0 ，P 1 Value of is Val 1 ，P 0 Value of is Val 0 ，P 1 Analyzed at time point T 1 ，P 0 The latest update time point is T 0 The threshold value isΔ T, then when Val 1 ＝Val 0 When the real-time database is updated, the real-time database is not updated correspondingly, otherwise, T is updated 1 And T 0 Introducing a comparison when T 1 -T 0 If the time is less than delta T, the real-time database is not correspondingly updated, otherwise, the real-time database is correspondingly updated.
As shown in FIG. 2, Val 1 ＝p_val，Val 0 ＝p_val_last，T 1 ＝p_t，T 0 P _ T _ last, the time difference is diff _ time, and the threshold Δ T is 5 ms. When p _ val is equal to p _ val _ last, the real-time database is not updated, otherwise, the time difference is compared with the threshold value, when diff _ time is less than 5ms, the real-time database does not need to be updated, otherwise, the real-time database is updated. When the real-time database is not updated, the data which is stored in the data flow storage module and is related to the current measured point is cleared, and the original data which is related to the current measured point is reserved so as to be used for analyzing the next data flow; when the real-time database is updated, the original data related to the current measured point is removed, and the current data related to the current measured point is retained so as to be used for analyzing and comparing the next data stream.
In the invention, a synchronization device takes a data packet IP address, a data packet sending service port and a data packet register address as keys and synchronously updates the value of a data packet register as a key value to a real-time database, which specifically comprises the following steps: in the synchronous device, a data flow analysis module obtains a service IP, a service port, a register address and a value of a register by analyzing a message in a data flow, wherein the service IP is 10.1.1.1, and the service port: 502, register address: 40001, value of register: 40, when the value of the detected point needs to be synchronized to the real-time database, the data stream packing module uses the information to obtain a key: key 10.1.1.1_502_40001, and its key value is: and delivering information with a format of 10.1.1.1_502_40001|40 as a payload (wherein,' is a division point of a key and a key value) to the data stream sending module to send the information to the real-time database so as to update corresponding data. At this time, the extranet host only needs one real-time database to provide service to the outside. When the extranet client obtains the real-time database service by accessing the real-time database, the real-time data corresponding to the measured point bit can be found through the key (metadata).
In the prior art, if the intranet only has one MODBUS service, the extranet host only needs to provide the real-time database service of one IP port, but if the intranet has a plurality of MODBUS services, the extranet host needs to provide the real-time database services of a plurality of IP ports, and one-to-one correspondence is achieved. In the invention, the data of a plurality of IP ports can be synchronously updated to a real-time database through the synchronization equipment, and then the real-time database service accessed by a plurality of IP ports is provided through the real-time database.
In the practical application process, a simulation device for simulating the synchronization device and a simulation device for simulating the data capture device can be arranged in the optical gate or the network gate, namely, the data capture device and the synchronization device are simulated in the optical gate or the network gate, so that the devices in a communication link can be reduced, the time delay and the information loss rate of information transmission are reduced, the outer network and the inner network can be physically isolated, and the information safety of the inner network is ensured.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. This need not be, nor should it be exhaustive of all embodiments. And obvious variations or modifications derived therefrom are intended to be within the scope of the claims of this patent.
Claims (8)
1. A real-time database synchronization method based on industrial protocol analysis is characterized in that one or more synchronization devices are added on a communication channel connecting an intranet and an extranet, and the synchronization devices are used for carrying out data synchronization update on a real-time database arranged on the extranet side; the synchronous equipment is arranged in an optical gate or/and a network gate which is connected with the outer network and the inner network; the synchronous updating of the real-time database by the synchronization equipment comprises the following steps:
1) copying an access data stream between an intranet side client and an intranet side server to synchronous equipment, and analyzing the access data stream by the synchronous equipment;
2) acquiring the value of the measured point and the time point of the measured point analyzed according to the accessed data flow analysis result in the step 1), when the value of the measured point is the same as the value of the corresponding point of the measured point in the real-time database, not updating the real-time database, otherwise, comparing the time difference between the last updating time point of the measured point and the time point of the measured point analyzed in the time data flow analysis with a threshold, when the time difference between the last updating time point of the measured point and the time point of the measured point analyzed in the time data flow analysis is greater than or equal to the threshold, updating the real-time database, otherwise, not updating the real-time database.
2. The real-time database synchronization method based on industrial protocol analysis as claimed in claim 1, wherein in step 1), when analyzing the access data stream, the analysis result of the access data stream at least includes the value of the measured point, the time point at which the measured point is analyzed, and the packet identification information where the measured point is located includes the packet IP address, the packet sending service port, the packet register address, and the packet register value.
3. The real-time database synchronization method based on industrial protocol analysis of claim 2, wherein the synchronization device uses the packet IP address, the packet egress service port, and the packet register address as keys and synchronously updates the value of the packet register to the real-time database as a key value.
4. A device for synchronously updating a real-time database by using the real-time database synchronization method based on industrial protocol analysis as claimed in any one of claims 1 to 3, which is characterized by comprising data capture equipment, synchronization equipment and a real-time database, wherein the real-time database is in communication connection with the synchronization equipment, and the synchronization equipment is in communication connection with a communication channel connecting an intranet side client and an intranet side server through the data capture equipment.
5. The apparatus according to claim 4, wherein the intranet side client is communicatively connected to the intranet side server via a switch, and the data capture device is communicatively connected to the switch and copies the access data stream between the intranet side client and the intranet side server in a mirror image manner.
6. The apparatus according to claim 4, wherein the data capture device is communicatively connected to the intranet client and the intranet server via a network bridge.
7. The apparatus of claim 4, wherein the synchronization device comprises a data stream receiving module, a data stream storing module, a data stream analyzing module, a data stream packaging module and a data stream sending module, and the data capturing device, the data stream receiving module, the data stream storing module, the data stream analyzing module, the data stream packaging module, the data stream sending module and the real-time database are sequentially connected in communication.
8. An apparatus according to any one of claims 4 to 7 wherein the synchronisation means is provided within the shutter or/and gatekeeper and is communicatively connected to the real time database via the shutter or/and gatekeeper.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110724228.5A CN113486030B (en) | 2021-06-29 | 2021-06-29 | Real-time database synchronization method and device based on industrial protocol analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110724228.5A CN113486030B (en) | 2021-06-29 | 2021-06-29 | Real-time database synchronization method and device based on industrial protocol analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113486030A CN113486030A (en) | 2021-10-08 |
| CN113486030B true CN113486030B (en) | 2022-08-19 |
Family
ID=77937796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110724228.5A Active CN113486030B (en) | 2021-06-29 | 2021-06-29 | Real-time database synchronization method and device based on industrial protocol analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113486030B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105657039A (en) * | 2016-02-15 | 2016-06-08 | 上海带来科技有限公司 | Big data based power plant equipment fault fast positioning system and method |
| CN111200624A (en) * | 2018-11-19 | 2020-05-26 | 浙江宇视科技有限公司 | Cross-gatekeeper data transmission method and system |
| CN112069149A (en) * | 2020-08-25 | 2020-12-11 | 上海上讯信息技术股份有限公司 | Database protocol analysis method and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130191159A1 (en) * | 2012-01-19 | 2013-07-25 | Unitedhealth Group Incorporated | System, method and computer program product for customer-selected care path for treatment of a medical condition |
| CN203225787U (en) * | 2013-03-19 | 2013-10-02 | 山西省电力公司大同供电分公司 | A synchronizer spanning a safe area in power scheduling |
| CN105468795A (en) * | 2015-10-30 | 2016-04-06 | 广西慧云信息技术有限公司 | Sensing device real-time data management method |
| CN106131146A (en) * | 2016-06-28 | 2016-11-16 | 山东中磁视讯股份有限公司 | Data synchronous system under a kind of limited network and method |
| CN110069566A (en) * | 2017-12-01 | 2019-07-30 | 北京京穗蓝盾信息安全技术有限公司 | Heterogeneous database synchronization method in a kind of one-way import system |
| CN110557251B (en) * | 2019-09-27 | 2022-07-22 | 武汉控安融科技有限公司 | Industrial data safety isolation acquisition system and internal and external network data one-way transmission method |
-
2021
- 2021-06-29 CN CN202110724228.5A patent/CN113486030B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105657039A (en) * | 2016-02-15 | 2016-06-08 | 上海带来科技有限公司 | Big data based power plant equipment fault fast positioning system and method |
| CN111200624A (en) * | 2018-11-19 | 2020-05-26 | 浙江宇视科技有限公司 | Cross-gatekeeper data transmission method and system |
| CN112069149A (en) * | 2020-08-25 | 2020-12-11 | 上海上讯信息技术股份有限公司 | Database protocol analysis method and device |
Non-Patent Citations (1)
| Title |
|---|
| 冶金类Web数据库在物理隔离网络间安全同步的方法研究;贺韬等;《计算机与应用化学》;20141128(第11期);第121-123页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113486030A (en) | 2021-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6134514A (en) | Large-scale network simulation method and apparatus | |
| US5600632A (en) | Methods and apparatus for performance monitoring using synchronized network analyzers | |
| US6263444B1 (en) | Network unauthorized access analysis method, network unauthorized access analysis apparatus utilizing the method, and computer-readable recording medium having network unauthorized access analysis program recorded thereon | |
| CN101090486A (en) | Monitoring device for multimedium monitoring information and its monitoring method | |
| CN110311990A (en) | A kind of configurable internet of things data acquisition system and configuration method | |
| CN110300120A (en) | The hierarchical synchronization method and system of reverse data under the conditions of a kind of physical isolation | |
| GB2267016A (en) | Multi-equipment routing method and master station for layered protocol communication network system | |
| CN113486030B (en) | Real-time database synchronization method and device based on industrial protocol analysis | |
| CN100493047C (en) | Method for synchronously implementing web tube total web clock | |
| CN101340370B (en) | Link selection method and apparatus | |
| KR100725853B1 (en) | Device and method for testing multiple protocols | |
| CN114338274A (en) | Heterogeneous industrial field bus fusion method and system | |
| CN116208727B (en) | Video matrix switcher based on multi-user modularization and control method thereof | |
| CN1462129A (en) | Multimedia intelligent monitoring method and system | |
| CN114745229B (en) | Embedded AI video gateway and implementation method thereof | |
| CN112312070A (en) | Digital retina cloud software scheduling method | |
| CN105677885A (en) | Receiving and sending device and method and synchronizing system for achieving shipping industry data synchronization | |
| CN108900596B (en) | WTB physical layer data frame acquisition device and data frame acquisition method | |
| CN115484161A (en) | TSN network configuration method, device and computer readable storage medium | |
| CN1300977C (en) | An open method and system for ultra-remote industrial monitoring information integration | |
| KR20090063504A (en) | Oma-dm system, device management server and terminal device | |
| CN101483662B (en) | Packet forwarding apparatus and method for virtual storage network switch | |
| CN114006651B (en) | Satellite internet transmission method suitable for multiple protocols | |
| CN116056173B (en) | Method for solving influence of SCTP multi-destination address on N2 port signaling synthesis | |
| CN112291210B (en) | Method and device for acquiring front-end equipment catalog |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |