CN111200624B - Cross-gatekeeper data transmission method and system - Google Patents
Cross-gatekeeper data transmission method and system Download PDFInfo
- Publication number
- CN111200624B CN111200624B CN201811377980.1A CN201811377980A CN111200624B CN 111200624 B CN111200624 B CN 111200624B CN 201811377980 A CN201811377980 A CN 201811377980A CN 111200624 B CN111200624 B CN 111200624B
- Authority
- CN
- China
- Prior art keywords
- data
- cache
- network
- database
- gatekeeper
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Abstract
The invention discloses a data transmission method and a data transmission system across a network gate. In the invention, data is transmitted unidirectionally in a file form, so that the characteristics of the network gate are ensured. The adoption of the internal intermediate device as data transfer reduces the damage to the storage hard disk. Meanwhile, files under abnormal conditions of the network gate cannot be accumulated, and the file writing is performed by the transmission equipment again, so that the real-time transmission of data is strongly ensured. In addition, the invention is independent of the network gate and the networks on two sides, can be rapidly developed based on the protocol form of each manufacturer, and realizes the data transmission of different manufacturers.
Description
Technical Field
The invention belongs to the technical field of network gate data transmission, and particularly relates to a method and a system for transmitting cross-network gate data.
Background
Video monitoring based on an IP network has gradually developed into a mainstream scheme of security industry, and technologies such as NAT, firewall, security isolation gatekeeper and the like are widely applied to various large projects in the integration of various network islands because of standards and openness of IP and considering security reasons of network communication transmission. The technical application of the network gate equipment has the technical core concept of realizing physical isolation, and physically isolating and blocking all connections with potential attack possibility, so that a hacker cannot invade, attack and damage, and real safety is realized. The isolation technology truly realizes the security of the network, but introduces a new problem, because the data transmission of vehicle information, control information and the like is required to have higher real-time performance, thus having new challenges for realizing the timely performance and the complete performance of the data transmission between network segments.
In the existing application of the video monitoring system, there are often two modes of Oracle database transmission or file transmission:
when the data is transmitted by an Oracle database, under the condition that the database stores the URL of the data picture, the gatekeeper realizes the data transmission in an IP address proxy mode, namely, the traffic from the video private network or the public external network is firstly sent to a proxy IP of the gatekeeper, the gatekeeper modifies the target IP and then forwards the target IP to the intranet, when the intranet needs to acquire the picture through the URL address, the intranet returns visit through the IP address which is modified into the intranet, the video private network or the public external network is returned visit through a port mapped by the gatekeeper, the picture stream is acquired, and then the picture stream is transmitted to the intranet; or the Oracle databases are deployed on both sides of the gateway, the Oracle databases support to directly write the picture stream into the databases, and the gateway supports to directly transmit the picture stream between the two databases.
However, most manufacturers in the video monitoring system industry apply no Oracle databases, and the above solution needs to directly apply the Oracle databases of the video private network or the public external network, and then upload the Oracle databases to the platform of the manufacturer related to each project in the intranet, where the operation performance pressure of the Oracle databases of the video private network or the public external network may be affected, and in the case of huge data volume storage and frequent database I/O operation, the performance of the databases may be bottleneck. When transmitting in the form of a picture stream, for the case that the database is written with single data each time and the daily processing amount of the data is more than 500 ten thousand, the query performance of the single writing and the intranet can be severely tested. Moreover, the Oracle database is a database needing payment, is not widely applied to most factories in the monitoring system industry, is different in most of databases applied by all factories, and needs to be subjected to new butt joint every time a new characteristic and a new requirement are developed, so that the cost is quite high.
The other mode is a file transmission mode, the network gate transmits the file to the intranet in a special network ferrying mode, each manufacturer can store the data in the form of writing the file into equipment in environments such as a video private network, a public extranet and the like according to respective protocols and a butt joint mode, the network gate ferries the catalog of the file stored by the equipment to the specific catalog of the other network segment designated equipment in the mode of transmitting the file by the network gate, and the file is read, analyzed and uploaded according to the receiving mode of an intranet platform by the equipment of the other network segment (temporarily named intranet).
However, when the file transmission mode is adopted, the real-time property of the file cannot be distinguished, namely, how many files are successfully ferred, when the ferry is stopped in a short time due to abnormal conditions of the network gate, and when the ferry is started again, the priority of the real-time data file is not guaranteed to be ferred; in addition, one end of the net gate continuously writes files to the disk, and the disk is cleaned after ferrying is successful, so that the disk has frequent reading, writing and cleaning operations, file fragments can be generated in the storage of the disk due to the frequent operations, and the stored disk is greatly damaged; in addition, the file transmission scheme is more prone to real-time transmission under normal conditions (no abnormal condition of the network gate), and the operation of deleting the file in the transmission process is also carried out due to the fact that no database exists, so that repeated uploading failure of data can be caused once abnormal conditions (power-off, restarting and the like of an upper platform of an intranet) occur in the transmission process, and the integrity of the data cannot be guaranteed.
Disclosure of Invention
The invention aims to provide a cross-gatekeeper data transmission method and a cross-gatekeeper data transmission system, which are used for overcoming the problems in the prior art, meeting the transmission of cross-gatekeeper data among different manufacturers and guaranteeing the real-time performance and the integrity of data transmission.
In order to achieve the above purpose, the technical scheme of the invention is as follows:
the utility model provides a cross-gatekeeper data transmission method for realizing cross-gatekeeper data transmission between first network and second network, be equipped with the gatekeeper between first network and the second network, be equipped with first transmission equipment in the first network, first transmission equipment is equipped with first database and first buffering, cross-gatekeeper data transmission method includes:
the first transmission equipment receives data of a data source, generates a corresponding data record in a first database, stores the data in a setting directory of a first cache in a file form, and initially sets a flag bit in the data record to be in a first state;
the first transmission equipment receives the access of the gatekeeper and ferrys the data to a second network through the gatekeeper;
after the first transmission device is taken away by the gateway, the corresponding flag bit in the data record is updated to be in the second state.
Further, the cross-gatekeeper data transmission method further includes:
when the first transmission device is restarted, the data record with the flag bit in the first state is read from the first database, and the data is stored in a file form in a setting directory of the first cache.
Further, the cross-gatekeeper data transmission method further includes:
when the residual space of the first buffer is insufficient to set a first threshold value, the first transmission device does not store the data in the set directory of the first buffer in the form of a file.
Further, a second transmission device is arranged in the second network, the second transmission device is provided with a second database and a second buffer, and the cross-gatekeeper data transmission method further comprises:
the second transmission equipment receives the data of the gate ferry and stores the data into a setting catalog of a second cache;
the second transmission equipment reads the data stored in the second cache to generate a data record in the second database, generates a duplicate removal table corresponding to the data and stores the duplicate removal table in the second database so as to avoid repeated writing of the data;
the second transmission equipment forwards the data stored in the second cache to the third party platform, and after forwarding, the flag bit in the data record is updated to be in a second state;
and the second transmission equipment updates the flag bit in the data record to be in a first state according to the sending failure information fed back by the third-party platform so as to retransmit the corresponding data to the third-party platform.
Further, the cross-gatekeeper data transmission method further includes:
when the remaining space of the second cache is not enough to set a second threshold value, the second transmission device stores the data exceeding a third threshold value of the storage space of the second cache to the hard disk, generates a data record in the second database, generates a duplicate removal table corresponding to the data and stores the duplicate removal table in the second database so as to avoid repeated writing of the data.
The invention also provides a data transmission system of the cross-gatekeeper, which is used for realizing the data transmission of the cross-gatekeeper between a first network and a second network, wherein the gatekeeper is arranged between the first network and the second network, the data transmission system of the cross-gatekeeper comprises first transmission equipment arranged in the first network, the first transmission equipment is provided with a first database and a first cache, and the first transmission equipment comprises:
the first data receiving module is used for receiving data of a data source, generating a corresponding data record in a first database, storing the data in a setting catalog of a first cache in a file form, and initially setting a flag bit in the data record to be in a first state;
the first data ferrying module is used for receiving the access of the gatekeeper and ferrying the data to the second network through the gatekeeper;
and the first state updating module is used for updating the corresponding flag bit in the data record to be in the second state after the data is taken away by the gateway.
Further, the first transmission device further includes:
and the first retransmission module is used for reading the data record with the flag bit in the first state from the first database when restarting, and storing the data in a file form into a setting directory of the first cache.
Further, the first data receiving module is further configured to store the data in the setting directory of the first cache no longer in a file form when the remaining space of the first cache is less than the set first threshold.
Further, the cross-gatekeeper data transmission system further includes a second transmission device disposed in a second network, where the second transmission device is provided with a second database and a second cache, and the second transmission device includes:
the second data receiving module is used for receiving the data of the gate ferry and storing the data into a setting catalog of the second cache;
the database recording module is used for reading the data stored in the second cache to generate a data record in the second database, generating a duplicate removal table corresponding to the data and storing the duplicate removal table into the second database so as to avoid repeated writing of the data;
the forwarding module is used for forwarding the data stored in the second cache to the third party platform, and updating the flag bit in the data record to be in a second state after forwarding;
and the second retransmission module is used for updating the flag bit in the data record to be in a first state according to the sending failure information fed back by the third party platform so as to retransmit the corresponding data to the third party platform.
Further, the second transmission device further includes:
and the monitoring early warning module is used for storing the data exceeding a third threshold value of the second cache storage space into the hard disk when the remaining space of the second cache is insufficient to the set second threshold value, generating a data record in the second database, generating a duplicate removal table corresponding to the data and storing the duplicate removal table into the second database so as to avoid the repeated writing of the data.
The invention provides a data transmission method and a data transmission system across a network gate, wherein a first transmission device and a second transmission device are arranged on two sides of the network gate, a database and a cache are arranged in the transmission devices, and a flag bit is arranged in the database, so that when an abnormality occurs, data recovery and retransmission are carried out, and the integrity of the data is ensured. In the invention, data is transmitted unidirectionally in a file form, so that the characteristics of the network gate are ensured. The adoption of the internal intermediate device as data transfer reduces the damage to the storage hard disk. Meanwhile, files under abnormal conditions of the network gate cannot be accumulated, and the file writing is performed by the transmission equipment again, so that the real-time transmission of data is strongly ensured. In addition, the invention is independent of the network gate and the networks on two sides, can be rapidly developed based on the protocol form of each manufacturer, and realizes the data transmission of different manufacturers.
Drawings
FIG. 1 is a schematic diagram of a network structure according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a cross-gatekeeper data transmission method on a first network side according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of a cross-gatekeeper data transmission method at a second network side according to an embodiment of the present invention.
Detailed Description
The technical scheme of the present invention will be further described in detail below with reference to the accompanying drawings and examples, which are not to be construed as limiting the present invention.
Taking the video monitoring system as shown in fig. 1 as an example, the first network and the second network are physically isolated by a security gatekeeper. The second network is used as a security network to be isolated, and is also commonly referred to as an intranet, for example, an intranet of a traffic management center, or an intranet of an urban security monitoring center. The first network may be a video private network or some social resource of an external network, such as a video private network of an enterprise, etc. This forms a butt joint of two networks, which is typically physically isolated by a security gatekeeper to ensure network security.
As shown in fig. 2, an embodiment of a method for transmitting data across a gatekeeper is configured to implement data transmission across a gatekeeper between a first network and a second network, where a gatekeeper is disposed between the first network and the second network, a first transmission device is disposed in the first network, and the first transmission device is disposed in a first database and a first cache, and the method for transmitting data across the gatekeeper includes:
the first transmission equipment receives data of a data source, generates a corresponding data record in a first database, stores the data in a setting directory of a first cache in a file form, and initially sets a flag bit in the data record to be in a first state;
the first transmission equipment receives the access of the gatekeeper and ferrys the data to a second network through the gatekeeper;
after the first transmission device is taken away by the gateway, the corresponding flag bit in the data record is updated to be in the second state.
Specifically, the data collected by the video front-end equipment in the first network take snapshot data of the road gate as an example, and include structural data such as license plate numbers, vehicle colors, vehicle types, elapsed time, lane positions and the like, and snapshot pictures. The video front-end devices in the first network are collectively referred to as a data source, and the data source may be other data sources, for example, data sent from other storage devices, which is not limited in the present invention.
The first transmission equipment receives data sent by a data source, generates a corresponding data record in a first database, and when the corresponding data record is generated in the first database, the data record comprises fields corresponding to the structured data.
The data received by the first transmission device in this embodiment also needs to be written into the first buffer in a file form, where the data is stored in the first buffer in a file form, and the naming manner of the data file name corresponding to the structured data in this embodiment is as follows: relative path + unique ID; the naming mode of the picture name corresponding to the picture is as follows: the unique ID+the tail number, the tail number included in the file name corresponding to the picture can ensure the storage of a plurality of pictures. The unique ID is a unique ID identification corresponding to the piece of data. The data file name and picture name, and their corresponding paths, are also included in the data record.
According to the embodiment, data is written into the first buffer memory according to the data record in a time sequence real-time priority mode, and the first buffer memory is used by dividing a part of space from the memory, so that high-speed data reading and writing are supported due to the quick reading and writing performance of the memory.
The data record of this embodiment further includes a flag bit field, where in this embodiment, the flag bit represents two states, where the first state is an initial state, and the second state is a state in which data has been taken away by the gatekeeper. The flag bit may be represented by a number, for example, a first state by 0 and a second state by 2. The invention is not limited to the specific flag bit representation method employed.
In this embodiment, the data source sends data to the first transmission device, and the first transmission device records the data in the first database and the first cache, so that the gatekeeper can conveniently take the accessed data from the first cache. The invention is not limited to the ferrying mode of the gatekeeper, and the invention can access the set catalog of the first transmission equipment through FTP, and the catalog is consistent with the storage catalog in the second network, so as to ensure that the gatekeeper smoothly transmits the data to the corresponding catalog in the second network through FTP.
In this embodiment, after the data is taken by the gatekeeper, the first transmission device updates the corresponding flag bit in the data record to the second state to indicate that the data has been taken from the first cache. The specific implementation method can query the data record in real time, find whether the file still exists in the first cache, judge that the file is taken away by the gatekeeper if the file does not exist, update the zone bit into the second state, and otherwise, do not process.
Based on the method, the first transmission equipment adopts the database and the first cache, so that the data is ferred to the second network in a file form through the gateway. Because the first transmission equipment is provided with the database, when the first transmission equipment is restarted, the data record with the flag bit in the first state can be read from the first database, and the data is stored in the setting catalog of the first cache in the form of a file. When the data is stored in the first buffer memory in the form of a file, the data can be read by the gateway and ferred to the second network, and the description is omitted.
It should be noted that, the first transmission device is restarted, the data in the first buffer does not necessarily disappear, and only the first transmission device is restarted after power failure, but no matter what type of restarting is performed, the data can be recovered from the database, so that the integrity of the data is ensured.
In one embodiment of the present invention, the first transmission device does not store the data in the file form in the setting directory of the first buffer when the remaining space of the first buffer is less than the set first threshold.
In order to prevent the first buffer memory from being full of data and overflowing the data, the first transmission device of the embodiment also realizes monitoring and early warning, namely, when the residual space of the first buffer memory is not enough to be a set first threshold value, the data is not stored in a file form in a set directory of the first buffer memory, and the situation of data loss caused by data overflow is prevented.
It should be noted that, in the technical solution, a hard disk may be further set in the first transmission device, and when the remaining space of the first buffer is not enough to set the first threshold, the data is stored in a file form in a set directory of the hard disk, so as to further enhance the transmission capability of the data. When the data is not transmitted to the second transmission device any more, the first buffer monitoring early warning is triggered, the data and the pictures are temporarily transferred to the hard disk of the first transmission device, and the data and the pictures are gradually transmitted to the first buffer after the normal state is recovered.
The technical scheme further includes that second transmission equipment is arranged in the second network, the second transmission equipment is provided with a second database and a second cache, and the cross-gatekeeper data transmission method further comprises the following steps:
the second transmission equipment receives the data of the gate ferry and stores the data into a setting catalog of a second cache;
the second transmission equipment reads the data stored in the second cache to generate a data record in the second database, generates a duplicate removal table corresponding to the data and stores the duplicate removal table in the second database so as to avoid repeated writing of the data;
the second transmission equipment forwards the data stored in the second cache to the third party platform, and after forwarding, the flag bit in the data record is updated to be in a second state;
and the second transmission equipment updates the flag bit in the data record to be in a first state according to the sending failure information fed back by the third-party platform so as to retransmit the corresponding data to the third-party platform.
Specifically, a second transmission device is arranged in the second network, data transferred from the gatekeeper is firstly archived in a second cached directory, and the second cached directory is consistent with the first cached directory.
In this embodiment, the second transmission device reads the data stored in the second cache, generates a data record in the second database, generates a duplicate-removal table corresponding to the data, and stores the duplicate-removal table in the second database, so as to avoid repeated writing of the data. The data records in the second database are consistent with the data records in the first database, and will not be described in detail herein. In the second database, a deduplication table is also generated, which includes the unique ID for the data, and by comparing the unique IDs, when duplicate data writes are found, the database is not written again.
Since the flag bit may not be updated yet in the data that the first transmission device has been ferred by the gatekeeper when the abnormality occurs, the data is retransmitted when the data is restarted, and the duplicate removal table is set at this time, which is beneficial to preventing duplicate writing of the data.
In this embodiment, the second transmission device forwards the data stored in the second buffer to the third party platform, and updates the flag bit in the data record to the second state after forwarding, so as to indicate that the data is sent to the third party platform.
In order to prevent data from failing in the process of sending to the third party platform, when the sending fails, the second transmission device updates the flag bit in the data record to be in the first state according to the sending failure information fed back by the third party platform. Therefore, the second transmission device considers that the data is not successfully transmitted, the corresponding data can be retransmitted to the third party platform, and the data can be read from the second cache or the hard disk according to the file name and the picture name of the data record in the second database and transmitted to the third party platform during retransmission.
The embodiment also sets a monitoring early warning for the second transmission device, that is, the cross-network gate data transmission method further includes:
when the remaining space of the second cache is not enough to set a second threshold value, the second transmission device stores the data exceeding a third threshold value of the storage space of the second cache to the hard disk, generates a data record in the second database, generates a duplicate removal table corresponding to the data and stores the duplicate removal table in the second database so as to avoid repeated writing of the data.
It should be noted that, when the second transmission device cannot upload the data and the picture to the third party platform due to the abnormal condition, the second buffer memory is accumulated to trigger the second buffer memory monitoring early warning finally, and the new data and the picture can be written into the hard disk for temporary storage. When the data written to the hard disk resumes available for transmission by the third party platform, the accumulated data is uploaded to the third party platform.
According to the technical scheme, when general abnormality occurs, data of the first cache and the second cache are not lost, data stored in the hard disk are not lost, when data retransmission is needed, retransmission can be carried out according to the flag bit in the data record, and when the retransmission is carried out, the data can be read from the cache or the hard disk for retransmission. When a power failure occurs, the data in the cache is lost, but the database is stored in the data record and can be used for recovery, so that only the structured data can be retransmitted, the picture cannot be retransmitted, and the picture needs to be acquired from the data source. By reasonably setting the relation between the writing and the reading of the data in the first cache and the second cache, the accumulated data in the caches can be very little, and the risk of losing the power-off picture is reduced.
Corresponding to the above method, an embodiment of a cross-gatekeeper data transmission system is also provided herein, configured to implement cross-gatekeeper data transmission between a first network and a second network, where a gatekeeper is disposed between the first network and the second network, where the cross-gatekeeper data transmission system includes a first transmission device disposed in the first network, where the first transmission device is provided with a first database and a first cache, and the first transmission device includes:
the first data receiving module is used for receiving data of a data source, generating a corresponding data record in a first database, storing the data in a setting catalog of a first cache in a file form, and initially setting a flag bit in the data record to be in a first state;
the first data ferrying module is used for receiving the access of the gatekeeper and ferrying the data to the second network through the gatekeeper;
and the first state updating module is used for updating the corresponding flag bit in the data record to be in the second state after the data is taken away by the gateway.
In this embodiment, the first transmission device further includes:
and the first retransmission module is used for reading the data record with the flag bit in the first state from the first database when restarting, and storing the data in a file form into a setting directory of the first cache.
In this embodiment, the first data receiving module is further configured to store the data in the setting directory of the first cache no longer in a file form when the remaining space of the first cache is less than the set first threshold.
In this embodiment, the cross-gatekeeper data transmission system further includes a second transmission device disposed in a second network, where the second transmission device is provided with a second database and a second cache, and the second transmission device includes:
the second data receiving module is used for receiving the data of the gate ferry and storing the data into a setting catalog of the second cache;
the database recording module is used for reading the data stored in the second cache to generate a data record in the second database, generating a duplicate removal table corresponding to the data and storing the duplicate removal table into the second database so as to avoid repeated writing of the data;
the forwarding module is used for forwarding the data stored in the second cache to the third party platform, and updating the flag bit in the data record to be in a second state after forwarding;
and the second retransmission module is used for updating the flag bit in the data record to be in a first state according to the sending failure information fed back by the third party platform so as to retransmit the corresponding data to the third party platform.
In this embodiment, the second transmission device further includes:
and the monitoring early warning module is used for storing the data exceeding a third threshold value of the second cache storage space into the hard disk when the remaining space of the second cache is insufficient to the set second threshold value, generating a data record in the second database, generating a duplicate removal table corresponding to the data and storing the duplicate removal table into the second database so as to avoid the repeated writing of the data.
The above embodiments are only for illustrating the technical solution of the present invention and not for limiting it, and those skilled in the art will be able to make various corresponding changes and modifications according to the present invention without departing from the spirit and the essence of the present invention, but these corresponding changes and modifications should fall within the protection scope of the appended claims.
Claims (10)
1. The utility model provides a cross-gatekeeper data transmission method, is used for realizing the data transmission of crossing the gatekeeper between first network and second network, be equipped with the gatekeeper between first network and the second network, its characterized in that, be equipped with first transmission equipment in the first network, first transmission equipment is equipped with first database and first buffering, cross-gatekeeper data transmission method includes:
the first transmission equipment receives data of a data source, generates a corresponding data record in a first database, stores the data in a setting directory of a first cache in a file form, and initially sets a flag bit in the data record to be in a first state;
the first transmission equipment receives the access of the gatekeeper and ferrys the data to a second network through the gatekeeper;
after the first transmission equipment is taken away by the network gate, updating the corresponding zone bit in the data record to be in a second state;
storing the data in a file form into a setting catalog of a first cache, wherein the method comprises the following steps:
storing the structured data and the picture data in a file form;
the generating a corresponding data record in the first database includes:
for structured data, the data record contains a field corresponding to the structured data, and a data file name and a corresponding path;
for picture data, the data record contains a data file name and a corresponding path.
2. The method for transmitting data across a gatekeeper according to claim 1, further comprising:
when the first transmission device is restarted, the data record with the flag bit in the first state is read from the first database, and the data is stored in a file form in a setting directory of the first cache.
3. The method for transmitting data across a gatekeeper according to claim 1, further comprising:
when the residual space of the first buffer is insufficient to set a first threshold value, the first transmission device does not store the data in the set directory of the first buffer in the form of a file.
4. The method for transmitting data across a network gate according to claim 1, wherein a second transmission device is provided in the second network, the second transmission device is provided with a second database and a second cache, and the method for transmitting data across a network gate further comprises:
the second transmission equipment receives the data of the gate ferry and stores the data into a setting catalog of a second cache;
the second transmission equipment reads the data stored in the second cache to generate a data record in the second database, generates a duplicate removal table corresponding to the data and stores the duplicate removal table in the second database so as to avoid repeated writing of the data;
the second transmission equipment forwards the data stored in the second cache to the third party platform, and after forwarding, the flag bit in the data record is updated to be in a second state;
and the second transmission equipment updates the flag bit in the data record to be in a first state according to the sending failure information fed back by the third-party platform so as to retransmit the corresponding data to the third-party platform.
5. The method for transmitting data across a gatekeeper according to claim 4, further comprising:
when the remaining space of the second cache is not enough to set a second threshold value, the second transmission device stores the data exceeding a third threshold value of the storage space of the second cache to the hard disk, generates a data record in the second database, generates a duplicate removal table corresponding to the data and stores the duplicate removal table in the second database so as to avoid repeated writing of the data.
6. The utility model provides a stride floodgate data transmission system for realize crossing the data transmission of floodgate between first network and second network, be equipped with the floodgate between first network and the second network, its characterized in that, stride floodgate data transmission system including locating the first transmission equipment in the first network, first transmission equipment is equipped with first database and first buffer memory, first transmission equipment includes:
the first data receiving module is used for receiving data of a data source, generating a corresponding data record in a first database, storing the data in a setting catalog of a first cache in a file form, and initially setting a flag bit in the data record to be in a first state;
the first data ferrying module is used for receiving the access of the gatekeeper and ferrying the data to the second network through the gatekeeper;
the first state updating module is used for updating the corresponding flag bit in the data record to be in the second state after the data is taken away by the network gate;
storing the data in a file form into a setting catalog of a first cache, wherein the method comprises the following steps:
storing the structured data and the picture data in a file form;
the generating a corresponding data record in the first database includes:
for structured data, the data record contains a field corresponding to the structured data, and a data file name and a corresponding path;
for picture data, the data record contains a data file name and a corresponding path.
7. The cross-gatekeeper data transmission system of claim 6, wherein the first transmission apparatus further comprises:
and the first retransmission module is used for reading the data record with the flag bit in the first state from the first database when restarting, and storing the data in a file form into a setting directory of the first cache.
8. The cross-gatekeeper data transmission system according to claim 6, wherein the first data receiving module is further configured to not store the data in the set directory of the first cache in a file form when the remaining space of the first cache is less than the set first threshold.
9. The cross-gatekeeper data transmission system according to claim 6, further comprising a second transmission device provided in a second network, the second transmission device being provided with a second database and a second cache, the second transmission device comprising:
the second data receiving module is used for receiving the data of the gate ferry and storing the data into a setting catalog of the second cache;
the database recording module is used for reading the data stored in the second cache to generate a data record in the second database, generating a duplicate removal table corresponding to the data and storing the duplicate removal table into the second database so as to avoid repeated writing of the data;
the forwarding module is used for forwarding the data stored in the second cache to the third party platform, and updating the flag bit in the data record to be in a second state after forwarding;
and the second retransmission module is used for updating the flag bit in the data record to be in a first state according to the sending failure information fed back by the third party platform so as to retransmit the corresponding data to the third party platform.
10. The cross-gatekeeper data transmission system of claim 9, wherein said second transmission apparatus further comprises:
and the monitoring early warning module is used for storing the data exceeding a third threshold value of the second cache storage space into the hard disk when the remaining space of the second cache is insufficient to the set second threshold value, generating a data record in the second database, generating a duplicate removal table corresponding to the data and storing the duplicate removal table into the second database so as to avoid the repeated writing of the data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811377980.1A CN111200624B (en) | 2018-11-19 | 2018-11-19 | Cross-gatekeeper data transmission method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811377980.1A CN111200624B (en) | 2018-11-19 | 2018-11-19 | Cross-gatekeeper data transmission method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111200624A CN111200624A (en) | 2020-05-26 |
| CN111200624B true CN111200624B (en) | 2023-05-23 |
Family
ID=70747216
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811377980.1A Active CN111200624B (en) | 2018-11-19 | 2018-11-19 | Cross-gatekeeper data transmission method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111200624B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112492001A (en) * | 2020-11-18 | 2021-03-12 | 深圳市商汤科技有限公司 | Data transmission method, device and equipment and computer storage medium |
| CN112565288B (en) * | 2020-12-21 | 2023-05-09 | 南京南瑞信息通信科技有限公司 | Method and system for executing intranet acquisition and control instruction in extranet |
| CN113064888B (en) * | 2021-03-25 | 2021-12-07 | 珠海格力电器股份有限公司 | Data proofreading method, device and system, server and equipment |
| CN113486030B (en) * | 2021-06-29 | 2022-08-19 | 北京安盟信息技术股份有限公司 | Real-time database synchronization method and device based on industrial protocol analysis |
| CN113918999B (en) * | 2021-12-15 | 2022-02-22 | 天津联想协同科技有限公司 | Method and device for establishing safe ferry channel, network disk and storage medium |
| CN114697118A (en) * | 2022-04-07 | 2022-07-01 | 西安热工研究院有限公司 | Test method and system for reverse through-network gate |
| CN114900473A (en) * | 2022-04-24 | 2022-08-12 | 深圳位置网科技有限公司 | Transmission method for transmitting emergency data to security platform and transmission platform thereof |
| CN116346391B (en) * | 2022-10-26 | 2023-11-10 | 广东省土地调查规划院 | Method, device and storage medium for information interaction between home and abroad survey data and internet |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101820449A (en) * | 2010-04-20 | 2010-09-01 | 江苏电力调度通信中心 | Cross-safety zone application service isolation platform |
| CN102868552A (en) * | 2012-08-21 | 2013-01-09 | 广东电网公司电力科学研究院 | Data sending method |
| WO2013065042A1 (en) * | 2011-11-02 | 2013-05-10 | Ronen Shai | Generating and using a location fingerprinting map |
| CN103532838A (en) * | 2013-10-09 | 2014-01-22 | 中国联合网络通信集团有限公司 | Method and system for realizing data exchange between isolation networks |
| CN106936780A (en) * | 2015-12-30 | 2017-07-07 | 北京明朝万达科技股份有限公司 | A kind of method for monitoring network and system |
| CN107704601A (en) * | 2017-10-13 | 2018-02-16 | 中国人民解放军第三军医大学第附属医院 | Big data search method and system, computer-readable storage medium and electronic equipment |
-
2018
- 2018-11-19 CN CN201811377980.1A patent/CN111200624B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101820449A (en) * | 2010-04-20 | 2010-09-01 | 江苏电力调度通信中心 | Cross-safety zone application service isolation platform |
| WO2013065042A1 (en) * | 2011-11-02 | 2013-05-10 | Ronen Shai | Generating and using a location fingerprinting map |
| CN102868552A (en) * | 2012-08-21 | 2013-01-09 | 广东电网公司电力科学研究院 | Data sending method |
| CN103532838A (en) * | 2013-10-09 | 2014-01-22 | 中国联合网络通信集团有限公司 | Method and system for realizing data exchange between isolation networks |
| CN106936780A (en) * | 2015-12-30 | 2017-07-07 | 北京明朝万达科技股份有限公司 | A kind of method for monitoring network and system |
| CN107704601A (en) * | 2017-10-13 | 2018-02-16 | 中国人民解放军第三军医大学第附属医院 | Big data search method and system, computer-readable storage medium and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 田湾核电站扩建ECC信息安全隔离方案简介;王环宇等;《仪器仪表用户》;20171108(第11期);全文 * |
| 跨安全网闸的内外网数据库同步的实现;董惠勤等;《科技通报》;20070315(第02期);第1-5页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111200624A (en) | 2020-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111200624B (en) | Cross-gatekeeper data transmission method and system | |
| US7934262B1 (en) | Methods and apparatus for virus detection using journal data | |
| US8290994B2 (en) | Obtaining file system view in block-level data storage systems | |
| US7984327B2 (en) | Fast data recovery system | |
| US11397538B2 (en) | Data migration method and apparatus | |
| US10769035B2 (en) | Key-value index recovery by log feed caching | |
| CN103942252B (en) | A kind of method and system for recovering data | |
| US7752403B1 (en) | Methods and systems for secure dispersed redundant data storage | |
| US7900006B2 (en) | Maintaining checkpoints during backup of live system | |
| US20080177806A1 (en) | Method and system for transparent backup to a hierarchical storage system | |
| US7467235B2 (en) | Data transfer method and system | |
| CN104679611A (en) | Data resource copying method and device | |
| US11556400B2 (en) | Handling large messages via pointer and log | |
| CN103530204B (en) | A kind of real time data backup method and system | |
| CN106126374A (en) | Method for writing data, method for reading data and device | |
| US20210326211A1 (en) | Data backup method, apparatus, and system | |
| US11184435B2 (en) | Message transmission method and apparatus in cluster file system | |
| CN106933747A (en) | Data-storage system and date storage method based on multithread | |
| US20060265431A1 (en) | Information processing system, replication method, difference information holding apparatus and program | |
| US20080267176A1 (en) | Selective preservation of network state during a checkpoint | |
| WO2021114774A1 (en) | Method and device for processing aggregated link | |
| US20160037119A1 (en) | Video recording apparatus, and video recording method when temporary network failure occurs | |
| US20150261810A1 (en) | Data transfer apparatus and method | |
| CN103049351B (en) | The log processing method of multivariate data server and device | |
| CN105760398A (en) | Log recording system and log record operating method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |