CN111723116A - Data backfill method, device and system - Google Patents
Data backfill method, device and system Download PDFInfo
- Publication number
- CN111723116A CN111723116A CN202010620522.7A CN202010620522A CN111723116A CN 111723116 A CN111723116 A CN 111723116A CN 202010620522 A CN202010620522 A CN 202010620522A CN 111723116 A CN111723116 A CN 111723116A
- Authority
- CN
- China
- Prior art keywords
- data
- record
- master station
- communication
- cache pool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2358—Change logging, detection, and notification
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
A data backfill method, device and system are disclosed. In an embodiment of the present application, the data backfilling method may include: when the communication with the master station is interrupted, local data monitoring is carried out; when the fact that local data change is monitored, a record is generated according to the changed data, and the record is stored in a preset cache pool, so that the master station can obtain the changed data from the cache pool after communication is recovered. According to the embodiment of the application, backfilling of the data acquired by the SCADA can be realized, and data discontinuity caused by packet loss and the like is avoided.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data backfill method, apparatus, and system.
Background
In the intelligent ring network management system, Data of a Supervisory Control And Data Acquisition (SCADA) system usually needs to be remotely transmitted, And Data packet loss is caused by many uncertain factors such as overlong network delay, network interruption, excessive jumping times, restarting of communication equipment And the like in the transmission process.
Disclosure of Invention
In order to ensure the reliability of data transmission, the embodiments of the present application are expected to provide a data backfill method, apparatus, and system suitable for an intelligent ring network management system.
According to an aspect of the present application, there is provided a data backfilling method, including:
when the communication with the master station is interrupted, local data monitoring is carried out;
when the fact that local data change is monitored, a record is generated according to the changed data, and the record is stored in a preset cache pool, so that the master station can obtain the changed data from the cache pool after communication is recovered.
In some examples, generating a record for the changed data includes: and aiming at one change of one data, generating a record, wherein the content of the record comprises the data address of the data, the data value after the change and a time mark as the acquisition time of the data.
In some examples, the method further comprises: before monitoring local data, verifying whether the timing duration of the last polling message transmitted by the master station reaches a preset duration threshold, and determining that the communication between the current slave station and the master station is interrupted when the timing duration reaches the duration threshold.
In some examples, the method further comprises: and when the communication with the master station is normal, responding to a polling message from the master station, packaging a current data value of local data and a time mark for indicating the local data acquisition time into a data frame, and uploading the data frame to the master station.
In some examples, the performing local data monitoring includes: detecting whether the data value change amplitude of data local to the slave station reaches a preset amplitude threshold value, and determining the data as changed data when the data value change amplitude of the data reaches the amplitude threshold value.
In some examples, after storing the record in a preset cache pool, the method further includes: and responding to a cache clearing instruction from the master station, clearing partial records or all records in the cache pool, wherein the cache clearing instruction is sent by the master station after reading the partial records or all records in the cache pool.
According to an aspect of the present application, there is provided a data backfill apparatus for use in a secondary station, comprising:
the monitoring module is configured to monitor local data when the communication with the master station is interrupted;
the recording module is configured to generate a record aiming at changed data when the local data are monitored to be changed, and store the record into a preset cache pool so that the master station can acquire the changed data from the cache pool after communication is recovered;
a cache pool configured to cache the record.
According to an aspect of the present application, there is provided a data backfilling method, including:
after communication between a master station and a slave station is recovered, reading a record from a preset cache pool in the slave station, wherein the record is generated by the slave station aiming at changed data during communication interruption and is stored in the cache pool;
analyzing the record to obtain data changed during the communication interruption period;
data that changes during the communication interruption is inserted into the database.
According to an aspect of the present application, there is provided a data backfill apparatus for a primary station, comprising:
the reading module is configured to read a record from a cache pool preset in the slave station after communication between the master station and the slave station is recovered, wherein the record is generated by the slave station aiming at changed data during communication interruption and is stored in the cache pool;
the analysis module is configured to analyze the record so as to obtain data changed during the communication interruption period;
and the inserting module is configured to insert the data changed during the communication interruption into the database.
According to one aspect of the application, a data backfill system is provided, and comprises a master station and a slave station, wherein a cache pool is preset in the slave station; wherein the content of the first and second substances,
the slave station is configured to monitor local data when communication with the master station is interrupted; when the local data change is monitored, generating a record aiming at the changed data, and storing the record into a preset cache pool;
and the master station is configured to read the record from the cache pool after the communication between the master station and the slave station is recovered, analyze the record to obtain the data changed during the communication interruption period, and insert the data changed during the communication interruption period into the database.
In the embodiment of the application, a buffer pool is arranged on the slave station, once the slave station data changes, the change record is marked with a time mark to be buffered in a record form, and the master station can access the required data every time to acquire the data which changes during the communication interruption period. Therefore, the embodiment of the application can ensure that the data collected by the SCADA can be backfilled, and data discontinuity caused by data loss and the like is avoided.
Drawings
Fig. 1 is a schematic flow chart of a data backfill method applied to a slave station side in the embodiment of the present application.
Fig. 2 is a schematic flow chart of a data backfilling method applied to a master station side in the embodiment of the present application.
Fig. 3 is a schematic structural diagram of a data backfilling device applied to a slave station side in the embodiment of the present application.
Fig. 4 is a schematic structural diagram of a data backfilling device applied to a master station side in the embodiment of the present application.
Fig. 5 is a schematic diagram of a structure of a data backfilling system and a data backfilling process thereof according to an embodiment of the present application.
Detailed Description
Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings. It should be noted that, in the present application, the embodiments and the features thereof may be arbitrarily combined with each other without conflict.
As described above, in the intelligent ring network management system, the SCADA data usually needs to be transmitted remotely, and there are many uncertain factors such as too long network delay, network interruption, too many hops, and restarting of the communication device during transmission, which results in data packet loss.
However, in the intelligent ring network management system, when the data request of the SCADA does not respond, the request is retransmitted for several times (generally 3 times), and if the data request does not respond (3 times do not respond), the request access is performed again after a certain period of time or a certain number of cycles. Generally, the reason for the data request not responding is complicated, the data request may be recovered by retry in a short period, the manual intervention repair may take a long time, and the scheme of retransmitting the request can only cope with the data loss caused by the short-term self-recoverable fault and cannot cope with the data loss caused by the network interruption in a long period. Therefore, there is a need to improve the reliability of data transmission in intelligent ring network management systems.
In order to solve the technical problem, a basic idea of the embodiments of the present application is to provide a data backfill method, device, and system applicable to an intelligent ring network management system, where data backfill can be performed based on a cache pool during a communication interruption period between a master station and a slave station, so as to ensure that data collected by an SCADA can be backfilled, and avoid data discontinuity caused by network influence.
Herein, the slave station is an instrument, a meter, or an electronic device, etc. responsible for collecting field data and uploading the field data to the master station, and may also be referred to as a slave station device. Accordingly, the master station is a server responsible for data collection, storage, processing, and the like, and may also be referred to as a master server. Herein, the slave station and the master station can communicate through various industrial communication protocols such as Modbus TCP.
Fig. 1 shows a flow of a data backfill method from a station side in an embodiment of the present application. As shown in fig. 1, the data backfill method from the station side may include:
s101, when the communication with a master station is interrupted, local data monitoring is carried out;
step S102, when it is monitored that local data changes, generating a record aiming at the changed data, and storing the record into a preset cache pool, so that the master station can acquire the changed data from the cache pool after communication is recovered.
In step S101, before performing local data monitoring, it may be determined in advance whether communication between the slave station and the master station is interrupted. Specifically, before local data monitoring, whether the timing duration of the last polling message transmitted by the master station reaches a preset duration threshold is verified, and when the timing duration reaches the duration threshold, communication interruption between the current slave station and the master station is determined.
In some examples, determining whether communication between the slave station and the master station is interrupted may include: starting timing when the last polling message from the master station is received; judging whether the timed duration exceeds a preset duration threshold or not; and when the timed duration exceeds the duration threshold, determining that the communication with the master station is interrupted, and automatically starting a data recording function so as to start local data monitoring and recording in the step S102.
Specifically, taking the Modbus TCP protocol as an example, when the master station and the slave station communicate with each other, the Modbus protocol needs to poll the slave station device, and a time threshold may be set to control whether to start the data change record, if the timing time for the slave station to receive the last polling message exceeds the time threshold and the current polling message of the master station is not received yet, the communication is interrupted, and the data change record function is automatically started. And when the polling message is received again, the communication is recovered, and the data change recording function is automatically quitted.
In step S101, performing local data monitoring may include: detecting whether the data value change amplitude of the data local to the slave station reaches a preset amplitude threshold value, and determining the data as changed data when the data value change amplitude of the data reaches the amplitude threshold value.
Specifically, taking the master-slave station adopting the Modbus TCP protocol as an example, during normal communication, the Modbus TCP performs data polling, and once communication is interrupted, the slave station performs local data monitoring, and once it is monitored that the amplitude of data change exceeds a preset amplitude threshold, step S102 is executed, and a record of the data change is generated and stored in the cache pool. For example, suppose that the slave station has 400014000240003 three analog quantities of data to upload to the master station. And in normal polling, the data change recording function is not started, and no data can be stored in the cache pool. And (4) communication interruption, timing from the receiving of the last message by the slave station, and triggering and starting the data change recording function when the timing time exceeds a set value. The system in the slave station monitors 400014000240003 three data changes, assuming that the three values are all 100 when the data change recording function is enabled, the amplitude threshold is all 5%, and when the jump amount of the data is greater than or equal to 5, the data is considered to have changed. For example, when 40001 jumps from 100 to 105, this data changes and needs to be recorded. During this period 40002 stays around between 101 and 102, it is considered that 40002 data has not changed and is not recorded.
In step S102, a record is generated for the changed data, including: and aiming at one change of one data, generating a record, wherein the content of the record comprises the data address of the data, the data value after the change and a time mark as the acquisition time of the data.
Specifically, taking the master-slave station using the Modbus TCP protocol as an example, once communication is interrupted, the step S102 is executed to generate a record of the data change and store the record in the buffer pool after the monitored data change amplitude exceeds the preset amplitude threshold. The cache pool may store data in the form of Modbus data, the contents of which include register locations (i.e., data addresses), changed data values, and times at which the data changes (i.e., time stamps as described herein). Therefore, after the communication is recovered, the master station can read the record of the data changed during the communication interruption from the cache pool in addition to the normal polling data, and extract the data to insert the data into the database, thereby completing the data backfilling.
For example, suppose that the slave station has 400014000240003 three analog quantities of data to upload to the master station. And (4) communication interruption, timing from the receiving of the last message by the slave station, and triggering and starting the data change recording function when the timing time exceeds a set value. And (3) monitoring 400014000240003 three data change conditions from the station system, assuming that the data values of the three data are all 100 when the data change recording function is started, and the amplitude threshold values are all 5%, namely, when the jump amount is greater than or equal to 5, the data are changed and need to be recorded. For example, when 40001 jumps to 105, a record storage is triggered, and at this time, a record is generated in the buffer pool, and the record content includes the data address of 40001, the data value 105 of 40001, and the time stamp of 40001, and the value of the time stamp may be the current time. When 40001 jumps to 100 again, a record is generated again, the content of the record includes the data address of 40001, the data value of 40001, 100, and the time stamp of 40001, the value of the time stamp being the current time at which the record was generated. 40002 lingers between 101 and 102 during this period, no record is generated. 40003 jumps to 95 to generate a record, and then jumps to 90 to generate another record. And then, communication is recovered, the slave station exits from the data change recording mode, and the master station polls and requests the cache pool data.
In this embodiment of the present application, a flag bit may be used in the cache pool, where the flag bit may be used to record a current data size of the cache pool, an operation state of the cache pool, and a control bit to clear a record cached therein. By the marking bit, the condition of the cache pool can be monitored in real time, data in the cache pool can be cleared in time, and data backfill is prevented from being influenced due to the fact that the cache pool is full.
The data backfilling method of the embodiment of the application can further include: and when the communication with the master station is normal, responding to a polling message from the master station, packaging the current data value of the local data and a time mark for indicating the local data acquisition time into a data frame and uploading the data frame to the master station. In the embodiment of the application, the slave station encapsulates time as a part of data into a data frame, and as the time loss exists in each communication, for time-sensitive data, the real-time performance of time marking of the slave station relative to the time marking of the master station is better, and the data acquisition time difference caused by problems of network delay and the like can be avoided.
In the embodiment of the application, the time stamp is the time for stamping the data when the data is acquired from the station. In a general industrial communication protocol, slave station devices are all instruments, meters or devices, and in order to ensure real-time performance of data, the sampling frequency of internal data is usually very high, and after the data passes through communication devices, the real-time performance of the data begins to be reduced. If a device uploads a data a to the server 5 seconds long and the original data does not have a timestamp, the state is already 5 seconds before the data a is received at the primary station. Assuming that the time required for another slave station to transmit a data B to the master station is 3 seconds, if the master station receives data a first and then data B, it cannot be determined which data changes first. Therefore, in the embodiment of the application, the slave station marks the data acquisition time when acquiring the data, and the exact time of the data acquisition can be expressed more clearly, so that the master station can judge the sequence of data change more accurately.
Specifically, taking the master-slave station adopting the Modbus TCP protocol as an example, the Modbus protocol itself does not support the time stamp, and in the embodiment of the present application, the time stamp function of the slave station side is realized by encapsulating the time stamp into the data frame. For example, in the slave station, the data value of the data M may be stored in the 40001 register, the 40002 register may be defined as the time stamp of 40001, that is, the time stamp of the data M may be stored in 40002, the data M is stored at the same time by 40001 and 40002, and the data M is requested at the same time when the data is requested.
In the above data backfilling method of the embodiment of the application, after storing the record in a preset cache pool, the method may further include: and responding to a cache clearing instruction from a master station, clearing part of or all of the records in the cache pool, wherein the cache clearing instruction is sent by the master station after reading the part of or all of the records in the cache pool. Therefore, the slave station deletes the data in the cache pool after receiving the confirmation of the master station, and the accurate uploading of the data can be further ensured.
Specifically, the cache pool may be emptied according to a flag bit of the cache pool, and after the master station confirms that a part of records or all records in the cache pool are received, the master station may send an empty cache instruction to the slave station, and after the slave station receives the empty cache instruction, the slave station may clear the part of records or all records in the cache pool, and reset the instruction. Other technical details may refer to the description of this section on the master station side below.
Fig. 2 shows a flow of a data backfilling method at the master station side in the embodiment of the present application. As shown in fig. 2, the data backfill method at the master station side may include:
step S201, after communication between a master station and a slave station is recovered, reading a record from a cache pool preset in the slave station, wherein the record is generated by the slave station aiming at changed data during communication interruption and is stored in the cache pool;
specifically, the master station may read a record from a buffer pool preset in the slave station while transmitting the polling packet to the slave station.
After step S201, after the master station confirms that the record or records are successfully acquired, it may send a flush buffer instruction to the slave station, so that the slave station can flush the record or records in the buffer pool in time. For example, the cache pool has 6 records, after the master station successfully reads all the 6 records in the cache pool, the master station sends a cache clearing instruction to the slave station, and the slave station clears the cache pool according to the cache clearing instruction. For another example, the cache pool has 6 records in total, and after the master station successfully reads the record R in the 6 records in the cache pool, the master station may send a cache clearing instruction for the record R to the slave station, and the slave station may clear the record R in the cache pool according to the cache clearing instruction for the record R.
In practical application, the master station may obtain all records in the cache pool at one time by means of a periodic request, and notify the slave station to clear the part of the cache after the obtaining is successful. If the acquisition fails, the data can be acquired by repeating the request until the data is successfully acquired.
Step S202, analyzing the record to obtain data changed during the communication interruption period;
step S203, insert the data changed during the communication interruption into the database.
Specifically, the data address of the data changed during the communication interruption, the changed data value and the time stamp can be obtained by analyzing the record. And when the data is inserted, inserting the data changed during the communication interruption period into the database according to the data address and the time mark.
The data backfill method can cope with data loss caused by long-term network interruption. Under the condition of long-time interruption of communication, the slave station can record the data change process according to the data change condition of the slave station, generate a record and store the record in the cache pool, and once the communication is recovered to be normal, the master station can acquire the data which is changed in the network interruption period from the cache pool, so that the backfill of the lost data is realized.
The embodiment of the application also provides a data backfilling device from the station side, and an exemplary structure of the data backfilling device from the station side is shown in fig. 3. As shown in fig. 3, the data backfilling device from the station side may include: the monitoring module 31 is configured to perform local data monitoring when communication with the master station is interrupted; the recording module 32 is configured to generate a record for changed data when it is monitored that the local data is changed, and store the record in a preset cache pool, so that the master station can acquire the changed data from the cache pool after communication is resumed; a cache pool 33 configured to cache the records.
In some examples, the data backfilling device at the slave station side may further include: and the clearing module is configured to respond to a cache clearing instruction from a master station, clear part of or all of the records in the cache pool, wherein the cache clearing instruction is sent by the master station after the part of or all of the records in the cache pool are read.
For other technical details of the slave station data backfilling device, reference may be made to the above data backfilling method section of the slave station side, and details are not described again.
The embodiment of the application also provides a data backfilling device on the main station side, and fig. 4 shows an exemplary structure of the data backfilling device on the main station side. As shown in fig. 4, the data backfilling device at the master station side may include: a reading module 41, configured to read a record from a cache pool preset in a slave station after communication between the master station and the slave station is resumed, where the record is generated by the slave station for changed data during communication interruption and is stored in the cache pool; an analysis module 42 configured to analyze the record to obtain data that changes during the communication interruption; an insertion module 43 configured to insert data that changes during the communication interruption into the database.
In some examples, the data backfilling device at the master station side may further include: and the clearing indication module is configured to send a cache clearing instruction to the slave station after confirming that one or some or all of the records are successfully acquired, so that the slave station can clear the corresponding record in the cache pool in time.
Other technical details of the data backfilling device at the master station side can refer to the data backfilling method part at the master station side, and are not described in detail.
As shown in fig. 5, the embodiment of the present application further provides a data backfilling system, which may include a master station 51 and a slave station 52, and the slave station 52 is preset with a buffer pool 33. The master station 51 and the slave station 52 in the data backfilling system can implement the data backfilling method described above.
As shown in fig. 5, the slave station 52 may include three parts, namely a buffer pool 33, a control logic through which the monitoring module 31 and the recording module 32 may be implemented, and an IO interface which may receive data collected by field instruments and devices. The master 51 may communicate with the slave 52 and the master 51 may communicate with the buffer pool 33 of the slave 52 to read data therefrom. Specifically, the slave station 52 may be configured to perform local data monitoring when communication with the master station 51 is interrupted; when the local data change is monitored, generating a record aiming at the changed data, and storing the record into a preset cache pool; the master 51 may be configured to read the record from the buffer pool 33 after communication between the master 51 and the slave 52 is resumed, parse the record to obtain data that changes during the communication interruption, and insert the data that changes during the communication interruption into the database.
Still taking the master-slave station adopting the Modbus TCP protocol as an example, during normal communication, the Modbus TCP performs data polling, once communication is interrupted, the slave station 52 performs local data monitoring, once it is monitored that data change exceeds a specified threshold, the data change record is recorded and stored in the cache pool 33, the cache pool 52 also performs data storage in the form of Modbus data, and the storage content includes the register position, the changed data value, and the time of data change. After the communication is resumed, the master station 51 will read the data change record from the buffer pool in addition to the normal polling data, and extract and insert the data into the database, thereby implementing data backfill.
Therefore, the buffer pool is arranged on the slave station, once the slave station data changes, the change record is marked with a time mark and is buffered in a record form, and the master station can access the required data every time and can obtain the data which change during the communication interruption. Therefore, the embodiment of the application can ensure that the data collected by the SCADA can be backfilled, and data discontinuity caused by data loss and the like is avoided.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. A data backfill method comprising:
when the communication with the master station is interrupted, local data monitoring is carried out;
when the fact that local data change is monitored, a record is generated according to the changed data, and the record is stored in a preset cache pool, so that the master station can obtain the changed data from the cache pool after communication is recovered.
2. The data backfilling method of claim 1 wherein generating a record for the changed data comprises: and aiming at one change of one data, generating a record, wherein the content of the record comprises the data address of the data, the data value after the change and a time mark as the acquisition time of the data.
3. The data backfilling method of claim 1 further comprising: before monitoring local data, verifying whether the timing duration of the last polling message transmitted by the master station reaches a preset duration threshold, and determining that the communication between the current slave station and the master station is interrupted when the timing duration reaches the duration threshold.
4. The data backfilling method of claim 1 or 3 further comprising:
and when the communication with the master station is normal, responding to a polling message from the master station, packaging a current data value of local data and a time mark for indicating the local data acquisition time into a data frame, and uploading the data frame to the master station.
5. The data backfilling method of claim 1 wherein the performing local data monitoring comprises: detecting whether the data value change amplitude of data local to the slave station reaches a preset amplitude threshold value, and determining the data as changed data when the data value change amplitude of the data reaches the amplitude threshold value.
6. The data backfilling method of claim 1 further comprising, after storing the record in a pre-set buffer pool: and responding to a cache clearing instruction from the master station, clearing partial records or all records in the cache pool, wherein the cache clearing instruction is sent by the master station after reading the partial records or all records in the cache pool.
7. A data backfill device applied to a slave station comprises:
the monitoring module is configured to monitor local data when the communication with the master station is interrupted;
the recording module is configured to generate a record aiming at changed data when the local data are monitored to be changed, and store the record into a preset cache pool so that the master station can acquire the changed data from the cache pool after communication is recovered;
a cache pool configured to cache the record.
8. A data backfill method comprising:
after communication between a master station and a slave station is recovered, reading a record from a preset cache pool in the slave station, wherein the record is generated by the slave station aiming at changed data during communication interruption and is stored in the cache pool;
analyzing the record to obtain data changed during the communication interruption period;
data that changes during the communication interruption is inserted into the database.
9. A data backfill apparatus applied to a master station, comprising:
the reading module is configured to read a record from a cache pool preset in the slave station after communication between the master station and the slave station is recovered, wherein the record is generated by the slave station aiming at changed data during communication interruption and is stored in the cache pool;
the analysis module is configured to analyze the record so as to obtain data changed during the communication interruption period;
and the inserting module is configured to insert the data changed during the communication interruption into the database.
10. A data backfill system comprises a master station and a slave station, wherein a cache pool is preset in the slave station; wherein the content of the first and second substances,
the slave station is configured to monitor local data when communication with the master station is interrupted; when the local data change is monitored, generating a record aiming at the changed data, and storing the record into a preset cache pool;
and the master station is configured to read the record from the cache pool after the communication between the master station and the slave station is recovered, analyze the record to obtain the data changed during the communication interruption period, and insert the data changed during the communication interruption period into the database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010620522.7A CN111723116A (en) | 2020-06-30 | 2020-06-30 | Data backfill method, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010620522.7A CN111723116A (en) | 2020-06-30 | 2020-06-30 | Data backfill method, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111723116A true CN111723116A (en) | 2020-09-29 |
Family
ID=72570747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010620522.7A Pending CN111723116A (en) | 2020-06-30 | 2020-06-30 | Data backfill method, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111723116A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114069847A (en) * | 2021-10-29 | 2022-02-18 | 国电南瑞科技股份有限公司 | Distributed photovoltaic data backfill system and method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101374144A (en) * | 2007-08-21 | 2009-02-25 | 华为技术有限公司 | Method, apparatus and system for controlling conversational data synchronization |
| CN102402562A (en) * | 2010-09-14 | 2012-04-04 | 中兴通讯股份有限公司 | Database remote disaster tolerant method and system |
| CN102546779A (en) * | 2011-12-27 | 2012-07-04 | Tcl集团股份有限公司 | Synchronized method and device of application data across devices |
| CN107386746A (en) * | 2017-07-26 | 2017-11-24 | 合肥巨步机械科技有限公司 | A kind of burglary-resisting system for battery truck parking systems |
| CN107678994A (en) * | 2017-09-15 | 2018-02-09 | 华为技术有限公司 | PCIe device hot drawing method and device |
-
2020
- 2020-06-30 CN CN202010620522.7A patent/CN111723116A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101374144A (en) * | 2007-08-21 | 2009-02-25 | 华为技术有限公司 | Method, apparatus and system for controlling conversational data synchronization |
| CN102402562A (en) * | 2010-09-14 | 2012-04-04 | 中兴通讯股份有限公司 | Database remote disaster tolerant method and system |
| CN102546779A (en) * | 2011-12-27 | 2012-07-04 | Tcl集团股份有限公司 | Synchronized method and device of application data across devices |
| CN107386746A (en) * | 2017-07-26 | 2017-11-24 | 合肥巨步机械科技有限公司 | A kind of burglary-resisting system for battery truck parking systems |
| CN107678994A (en) * | 2017-09-15 | 2018-02-09 | 华为技术有限公司 | PCIe device hot drawing method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114069847A (en) * | 2021-10-29 | 2022-02-18 | 国电南瑞科技股份有限公司 | Distributed photovoltaic data backfill system and method |
| CN114069847B (en) * | 2021-10-29 | 2024-03-26 | 国电南瑞科技股份有限公司 | Distributed photovoltaic data backfill system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1311603C (en) | Fault data synchronization via peer-to peer communications network | |
| CN101331736B (en) | Using travel-time as means for improving the accuracy of simple network time protocol | |
| US9344347B2 (en) | Delay time measuring apparatus, computer readable record medium on which delay time measuring program is recorded, and delay time measuring method | |
| CN102412999B (en) | Packet capturing based remote fault location method, system and device | |
| CN1998186A (en) | Method and apparatus for characterizing an end-to-end path of a packet-based network | |
| CN108632108B (en) | Monitoring method and device based on polling protocol and readable storage medium | |
| CN109120067B (en) | Event sequence record reporting method, device, equipment and readable storage medium | |
| CN113259886B (en) | Industrial edge data transmission method and device | |
| JP6574031B2 (en) | Mobile video call quality measurement method and system | |
| CN111723116A (en) | Data backfill method, device and system | |
| CN112333758B (en) | Stability monitoring management method, system and application of television gateway system | |
| CN111565132B (en) | Overtime detection method and system for bidirectional forwarding detection message | |
| JP2019176582A (en) | Monitoring system for power receiving and distributing facility | |
| CN112486009B (en) | Time service signal abnormal waveform recording analyzer and method | |
| CN104579848A (en) | Data analyzing and network monitoring integration method for OBS instrument | |
| CN104580507A (en) | Field station data transmission and integration method | |
| CN108225311B (en) | Track data acquisition system method and system for automatic track measuring vehicle | |
| JP2004289748A (en) | Quality monitoring system and its quality monitoring method for multimedia communications | |
| CN117478561B (en) | Ethernet industrial bus time delay analysis method and system | |
| CN112887354B (en) | Performance information acquisition method and device | |
| CN100593293C (en) | Method for synchronizing memory areas in a transmitter apparatus and a receiver apparatus, and corresponding receiver apparatus | |
| CN104580508A (en) | Field station data transmission and integration system | |
| CN113364646B (en) | Method, device and system for determining round-trip delay, storage medium and electronic equipment | |
| CN113064869B (en) | Log processing method, device, transmitting end, receiving end equipment and storage medium | |
| CN113626279B (en) | I2C communication monitoring method, device, CPLD and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 8050, 8 / F, 1033 Changning Road, Changning District, Shanghai 200050 Applicant after: Aopu (Shanghai) new energy Co., Ltd Address before: Room 501, No. 358 Ledu Road, Songjiang District, Shanghai, 201103 Applicant before: SHANGHAI YUYUAN POWER TECHNOLOGY Co.,Ltd. |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200929 |