CN109150596B - SCADA system real-time data dump method and device - Google Patents
SCADA system real-time data dump method and device Download PDFInfo
- Publication number
- CN109150596B CN109150596B CN201810897123.8A CN201810897123A CN109150596B CN 109150596 B CN109150596 B CN 109150596B CN 201810897123 A CN201810897123 A CN 201810897123A CN 109150596 B CN109150596 B CN 109150596B
- Authority
- CN
- China
- Prior art keywords
- task
- data
- dump
- dumping
- real
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the invention discloses a method and a device for dumping real-time data of an SCADA system, wherein the method comprises the following steps: establishing a SCADA system real-time data dump task queue, wherein the queue comprises a first dump task T1 and a second dump task T2; respectively collecting real-time data through a first dumping task T1 and a second dumping task T2 and storing the real-time data into corresponding buffer data files; applying for acquiring data dump right and executing normal data dump by a first dump task T1; when the first dump task T1 breaks down, the second dump task T2 carries out breakpoint transmission; when the second dump task T2 fails and the failure of the first dump task T1 is eliminated, the first dump task T1 performs breakpoint resume to realize automatic failure recovery. The scheme of the embodiment solves the problem of data interruption caused by software fault in the real-time data dump process of the SCADA system, and improves the stability of the real-time data dump of the SCADA system.
Description
Technical Field
The embodiment of the invention relates to a data dump technology, in particular to a real-time data dump method and a real-time data dump device for an SCADA (supervisory control and data acquisition) system.
Background
In the field of industrial automation Control, SCADA (Supervisory Control And Data Acquisition) is commonly used for acquiring Data And monitoring field operation equipment, various instruments And communication equipment are installed on the field, a SCADA system acquires Data through a communication protocol, And the SCADA system has a Data storage function, so that historical Data trend of a Data item to be monitored can be checked And analyzed through a SCADA system software module, And the upper Data analysis module needs to calculate And analyze bottom Data to obtain various summarized Data such as metering Data And operation indexes, And needs to dump basic Data to a relational database in real time through the SCADA system, And the upper application software accesses the Data, calculates And then performs business analysis.
The SCADA system real-time data dump operating environment generally comprises: the SCADA server runs an SCADA software system and a real-time data dump program, the database server runs a relational database, the application server runs upper-layer industry application software and the like.
The SCADA software system collects real-time data from field equipment, and a real-time data dump program accesses a real-time database through an interface or a conventional communication protocol to obtain the data and provides the data for a relational database accessible by application software in the upper industry.
At present, the commonly used data dump measures of the SCADA system include:
(1) ADO data processing script
And embedding an ADO data processing script in the SCADA timing script, and directly writing the real-time SCADA variable data into a historical library.
ADO (ActiveX Data objects): a program object for representing data structures and contained data in a user database, using ADO objects to create or modify tables and queries, check the database, or access external data sources.
(2) ODBC data dump component
The ODBC data dump component acquires data by accessing an internal interface through an SDK of an SCADA manufacturer and dumping real-time data into a relational database in a certain format at regular time through an ODBC mode.
ODBC (Open Database Connectivity Open Database interconnection) is a component of microsoft corporation Open Services Architecture (WOSA), which builds a set of specifications and provides a set of standard APIs (application programming interfaces) for Database access.
SDK (Software Development Kit Software Development tool) is a collection of Development tools used by Software engineers to create application Software for a specific Software package, Software framework, hardware platform, operating system, etc., and the patent uses an application program interface API provided for SCADA Software vendors to access SCADA real-time databases through the interface.
(3) OPC data acquisition and storage software
And deploying OPC data acquisition and storage software, acquiring SCADA real-time data by the software in an OPC mode, acquiring the real-time data by the software at regular time and writing the real-time data into a relational database in an ODBC mode.
OPC (object connection and embedding for Process Control) is a specification designed to solve a communication standard between a field management layer and a Process Control layer based on Microsoft COM/DCOM (Distributed component object model) technology and DNA (Distributed internet Application) framework, and provides a standard mechanism for data communication between a Process management Application and a field automation Control system and devices.
However, the current SCADA system data dump has the following disadvantages:
(1) ADO data processing script
Because the ADO script is complex and inconvenient to maintain, and the operation process can affect the operation efficiency of the SCADA system software, the use is less.
(2) Data loss often occurs during the process of using ODBC data dump component or OPC data acquisition and storage software:
when the ODBC component dumps abnormally, the data is temporarily interrupted;
the database access is abnormal, and the data is temporarily interrupted;
and the OPC acquisition and storage software has abnormal data storage (database writing failure, network abnormality and the like), and the data is temporarily interrupted.
In order to solve the above problems, the conventional solutions are as follows:
the traditional solution is to restart and activate the software function by monitoring whether the process is abnormal or not, detecting whether the data stops dumping or not, and manually maintaining and completing the data (inquiring the data record of the SCADA historical database and manually completing the data). However, the following problems still exist with these current solutions:
(1) data are interrupted for a long time, and the workload of filling up is large;
(2) partial data are incomplete, so that the result of timing data calculation is deviated;
(3) the data maintenance cost is high;
(4) affecting upper layer data calculation and analysis processing application functions.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present invention provide a method and an apparatus for data collection and monitoring control of real-time data dump of an SCADA system, which can prevent data interruption after an abnormal data dump program, avoid fault recovery and completion of historical data caused by a fault, and ensure sequential write-in of data dump, thereby improving stability of real-time data dump of the SCADA system.
In one embodiment of the present invention, there is provided:
(1) a SCADA system real-time data dump method can comprise the following steps:
establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2;
starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file;
applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database data table;
when the first dumping task T1 fails in the process of data dumping, the second dumping task T2 performs breakpoint resuming according to the real-time data stored in the second buffer data file, so as to realize automatic failure recovery; when the second dumping task T2 fails during the data dumping process and the failure of the first dumping task T1 is eliminated, the first dumping task T1 performs breakpoint resuming according to the real-time data stored in the first buffered data file, thereby achieving automatic failure recovery.
In various other embodiments of the invention:
(2) the method for real-time data dump of SCADA system according to (1), wherein the establishing a task queue for real-time data dump of SCADA system may include:
setting a dump task configuration table;
configuring the first dumping task T1 and the second dumping task T2 in the dumping task configuration table;
wherein, the dump task configuration table comprises: task numbering and collecting and dumping configuration information; the task numbers of the first dumping task T1 and the second dumping task T2 are different, and the acquisition and dumping configuration information are the same.
(3) The SCADA system real-time data dump method according to (2), wherein the acquisition and dump configuration information may include one or more of the following: task name, task type, data collection and dump program path, data collection and dump program name, collection task number and dump destination data table name.
(4) The SCADA system real-time data dump method in (1), the method may further include: after a SCADA system real-time data dump task queue is established, setting the main-standby relationship between the first dump task T1 and the second dump task T2;
the first dump task T1 is a main task, and the second dump task T2 is a standby task.
(5) The SCADA system real-time data dump method according to (1), wherein the starting the first dumping task T1 and the second dumping task T2 may include:
respectively starting a first data acquisition and dump program A1 corresponding to the first dumping task T1 and a second data acquisition and dump program A2 corresponding to the second dumping task T2;
the operation information is accessed and recorded to the preset task synchronization table Y1 through the first data collecting and dumping program A1 and the second data collecting and dumping program A2.
(6) The SCADA system real-time data dump method according to (5), wherein the task synchronization table Y1 may include: task coding, task state, state change time and data dump time point;
wherein the task encoding comprises: a first code M1 representing the main task and a second code M2 representing the standby task;
the task states include: an operation state X1, a fault state X2, an application state X3, a continuous transmission state X4 and a working state X5.
(7) The method for real-time data dump of SCADA system according to (6), wherein the applying for obtaining a data dump right by the first dump task T1 and executing a normal data dump to write the real-time data into a preset relational database data table may include:
setting the task state of the first dumping task T1 in the task synchronization table Y1 to be an application state X3 through the first data collecting and dumping program A1 so as to apply for the data dumping right;
after the data dump right is acquired, executing a normal data dump task process through the first data acquisition and dump program A1, and writing acquired real-time data into a preset first relational database data table W1; and setting the task state in the task synchronization table Y1 to a working state X5, and updating the data dump time point and the state change time.
(8) The SCADA system real-time data dump method according to (6), wherein, when the first dumping task T1 fails during data dumping, the performing of breakpoint resuming by the second dumping task T2 according to the real-time data stored in the second buffered data file may include:
the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right;
acquiring a first data breakpoint B1 of the first dumping task T1 from a data dumping time point and a state change time of the task synchronization table Y1;
extracting a continuous data queue comprising the first data breakpoint B1 from the second buffered data file and performing a resume process for breakpoint resume;
after the breakpoint continuous transmission is completed, changing the recovery state of a preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1;
when the second dumping task T2 fails during the data dumping process and the failure of the first dumping task T1 is eliminated, the performing, by the first dumping task T1, a breakpoint resume according to the real-time data stored in the first buffered data file may include:
the first dumping task T1 judges the fault of the second dumping task T2 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right;
acquiring a second data breakpoint B2 of the second dump task T2 from the data dump time point and the state change time of the task synchronization table Y1;
extracting a continuous data queue comprising the second data breakpoint B2 from the first buffered data file and performing a resume process for breakpoint resume;
and after the breakpoint continuous transmission is completed, changing the recovery state of the preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1.
(9) The SCADA system real-time data dump method according to any one of (1) - (8), wherein the fault in the data dump process may include: the dump program itself fails.
In another embodiment of the present invention, there is provided:
(10) a real-time data dump device of a data acquisition and monitoring control SCADA system comprises: the system comprises a processor and a computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions are executed by the processor, the SCADA system real-time data dump method is realized.
The embodiment of the invention comprises the following steps: establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2; starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file; applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database data table; when the first dumping task T1 fails in the process of data dumping, the second dumping task T2 performs breakpoint resuming according to the real-time data stored in the second buffer data file, so as to realize automatic failure recovery; when the second dumping task T2 fails during the data dumping process and the failure of the first dumping task T1 is eliminated, the first dumping task T1 performs breakpoint resuming according to the real-time data stored in the first buffered data file, thereby achieving automatic failure recovery. Through the scheme of the embodiment, the problem of data interruption caused by software fault in the real-time data dump process of the SCADA system is solved, the data interruption caused by abnormal data dump program is prevented, the fault recovery and historical data completion behaviors caused by faults are avoided, the sequential write-in of data dump is ensured, and the stability of the real-time data dump of the SCADA system is improved.
Additional features and advantages of embodiments of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the examples of the application do not constitute a limitation of the embodiments of the invention.
FIG. 1 is a flow chart of a method for real-time data dump of a SCADA system according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a SCADA system real-time data dump method according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for initiating a first dump task T1 and a second dump task T2 according to an embodiment of the present invention;
FIG. 4 is a flowchart of a method for applying for acquiring data dump right and executing normal data dump by the first dump task T1 according to an embodiment of the present invention;
FIG. 5 is a flowchart of a method for performing breakpoint resuming by the second dumping task T2 according to the real-time data stored in the second buffered data file when the first dumping task T1 fails during the data dumping process according to an embodiment of the present invention;
fig. 6 is a flowchart of a method for performing breakpoint resuming by the first dumping task T1 according to the real-time data stored in the first buffered data file when the second dumping task T2 has a fault during data dumping and the fault of the first dumping task T1 is eliminated according to the embodiment of the present invention;
FIG. 7 is a block diagram of a SCADA system real-time data dump device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The steps illustrated in the flow charts of the figures may be performed in a computer system such as a set of computer-executable instructions. Also, while a logical order is shown in the flow diagrams, in some cases, the steps shown or described may be performed in an order different than here.
Example one
A SCADA system real-time data dump method, as shown in FIG. 1 and FIG. 2, may include S101-S104:
s101, establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2;
s102, starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file;
s103, applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database data table;
s104, when the first dumping task T1 fails in the process of data dumping, the second dumping task T2 continues to transfer the data at the break point according to the real-time data stored in the second buffer data file, so as to realize automatic fault recovery; when the second dumping task T2 fails during the data dumping process and the failure of the first dumping task T1 is eliminated, the first dumping task T1 performs breakpoint resuming according to the real-time data stored in the first buffered data file, thereby achieving automatic failure recovery.
In the embodiment of the invention, in order to solve the problems provided in the background art, a task queue of real-time data dump of the SCADA system is established, a plurality of data acquisition and dump programs are operated in parallel, a software synchronous operation mechanism is established, an online operation backup function is provided, when data acquisition or dump is abnormal, fault recording and redundancy processing are realized, and the stability of real-time data dump of the SCADA system can be greatly improved.
Example two
The embodiment provides a specific implementation way for establishing a real-time data dump task queue of the SCADA system on the basis of the first embodiment.
In this embodiment of the present invention, the establishing a real-time data dump task queue of the SCADA system may include:
setting a dump task configuration table;
configuring the first dumping task T1 and the second dumping task T2 in the dumping task configuration table;
wherein, the dump task configuration table comprises: task numbering and collecting and dumping configuration information; the task numbers of the first dumping task T1 and the second dumping task T2 are different, and the acquisition and dumping configuration information are the same.
In this embodiment of the present invention, a dump task configuration table C2 may be provided, where the two dump tasks in the dump task configuration table C2 are configured as a first dump task T1 and a second dump task T2, respectively. The task numbers of the first dumping task T1 and the second dumping task T2 are different, and the collection and dumping configuration information is the same.
In an embodiment of the present invention, the collecting and dumping configuration information may include one or more of the following: task name, task type, data collection and dump program path, data collection and dump program name, collection task number and dump destination data table name.
In the embodiment of the present invention, the dump task configuration table C2 may include one or more of task number, task name, task type, data collection and dump program path, data collection and dump program name, collection task number, dump destination data table name, and other information items.
In the embodiment of the present invention, the method may further include: after a SCADA system real-time data dump task queue is established, setting the main-standby relationship between the first dump task T1 and the second dump task T2; wherein, the first dump task T1 is a main task (M1), and the second dump task T2 is a standby task (M2).
In the embodiment of the present invention, in the dump task configuration table C2, the task type of the first dump task T1 may be set as the main task (M1), and the task type of the second dump task T2 may be set as the standby task (M2).
In the embodiment of the present invention, before the SCADA system real-time data dump task queue is established, the collection data configuration table C1 may be preset, so as to execute uninterrupted real-time data collection tasks according to the collection data configuration table C1.
In the embodiment of the present invention, the collected data configuration table C1 may include information of data items to be acquired and dumped from the SCADA system, and specifically may include any one or more of the following information: collecting task numbers, collected input/output IO point names, data item names, collection frequency, value taking modes and the like.
EXAMPLE III
This embodiment is based on the first embodiment or the second embodiment, and a specific implementation manner of starting the first dumping task T1 and the second dumping task T2 is given.
In the embodiment of the present invention, as shown in fig. 3, the starting the first dumping task T1 and the second dumping task T2 may include S201 to S202:
s201, respectively starting a first data acquisition and dump program A1 corresponding to the first dumping task T1 and a second data acquisition and dump program A2 corresponding to the second dumping task T2;
s202, accessing and recording the running information to a preset task synchronization table Y1 through the first data collecting and dumping program A1 and the second data collecting and dumping program A2.
In the embodiment of the present invention, a first data collecting and dumping program a1 corresponding to the first dumping task T1 and a second data collecting and dumping program a2 corresponding to the second dumping task T2 may be written in advance; when the first dumping task T1 and the second dumping task T2 are restarted, the first data collecting and dumping program a1 and the second data collecting and dumping program a2 may be started, respectively, the program a1 and the program a2 access and record the running information to the task synchronization table Y1, and record the task state in the task synchronization table Y1 as the X1 (running) state.
In this embodiment of the present invention, the task synchronization table Y1 may include: task coding, task state, state change time and data dump time point;
wherein the task encoding comprises: a first code M1 representing the main task and a second code M2 representing the standby task;
the task states include: an operation state X1, a fault state X2, an application state X3, a continuous transmission state X4 and a working state X5.
In the embodiment of the present invention, task coding (or task type coding) describes: m1 (main): a main task; m2 (stock): and (5) preparing tasks. That is, when the corresponding task code column is set to M1, the corresponding dump task is the main task; when the corresponding task code column is set to M2, the corresponding dump task is referred to as a standby task.
In the embodiment of the present invention, the task code may include, but is not limited to, the letters M1 and M2, and any number, letter, character, etc. other than M1 and M2 may be used.
In the embodiment of the invention, the task state description comprises the following steps:
x1 (run): after the software is started, the task state item of the task synchronization table Y1 is written into the state;
x2 (fault): the software dump is abnormal, and the task state item in the task synchronization table Y1 is written into the state;
x3 (application): applying for executing a data dump task, and when the task state is not dumped (a non-continuous transfer state and a non-working state) or the update time is invalid (such as a record with the update time less than 60 seconds), updating the task state into an application state;
x4 (continuous transmission): executing breakpoint continuous transmission work;
x5 (working): and executing real-time dump work.
In the embodiment of the present invention, the representation manner of the task state may include, but is not limited to, letters X1-X5, and any number, letter, character, etc. other than X1-X5 may be used.
In the embodiment of the invention, the implementation mode of the data collection and dump task program can select an OPC mode or an interface calling mode.
Example four
On the basis of any of the above embodiments, this embodiment provides a specific implementation manner that the real-time data are collected continuously and stored in the corresponding first buffered data file and the corresponding second buffered data file by the first dumping task T1 and the second dumping task T2, respectively.
In the embodiment of the invention, real-time data can be acquired from the SCADA system according to the preset information of the acquisition data configuration table C1, and after the data acquisition and dump program is started, the program can execute uninterrupted real-time data acquisition tasks.
In an embodiment of the present invention, the uninterrupted collection of real-time data by the first dumping task T1 and the second dumping task T2, and the storing of the corresponding first buffered data file and the second buffered data file may specifically include: continuously executing a collecting process K1_1 and a writing buffer data process K3_1 of the first transstorage task T1; the collection process K1_2 and the write buffer data process K3_2 of the second dump task T2 are executed without interruption.
In an embodiment of the present invention, the uninterrupted execution of the collection process K1_1 and the write buffer data process K3_1 of the first dumping task T1 may include: a first data acquisition and dump program A1 of a first dumping task T1 executes an acquisition process K1_1 according to an acquisition data configuration table C1 list, and acquires real-time data from an SCADA real-time base; the K3_1 process is performed to write the real-time data to the first buffered data file F1.
In an embodiment of the present invention, the uninterrupted execution of the collection process K1_2 and the write buffered data process K3_2 of the second dump task T2 may include: a second data acquisition and dump program A2 of a second dump task T2 executes an acquisition process K1_2 according to an acquisition data configuration table C1 list, and acquires real-time data from an SCADA real-time base; the K3_2 process is performed to write the real-time data to the second buffered data file F2.
EXAMPLE five
The embodiment provides a specific implementation manner of the first dumping task T1 applying for acquiring the data dumping right on the basis of any of the above embodiments.
In the embodiment of the present invention, as shown in fig. 4, applying for obtaining a data dump right by the first dumping task T1, and performing a normal data dump to write the real-time data into a preset relational database data table may include S301 to S302:
s301, setting the task state of the first dumping task T1 in the task synchronization table Y1 to be an application state X3 through the first data collecting and dumping program A1 so as to apply for the data dumping right;
s302, after the data dump right is obtained, executing a normal data dump task process through the first data acquisition and dump program A1, and writing acquired real-time data into a preset first relational database data table W1; and setting the task state in the task synchronization table Y1 to a working state X5, and updating the data dump time point and the state change time.
In the embodiment of the present invention, the data dump right application process K2_1 may be executed by the first data collection and dump program a1, so that the first dump task T1, which is marked as a main task, applies for acquiring data dump right. Specifically, the corresponding task state of the first dumping task T1 in the task synchronization table Y1 may be set to the X3 (application) state by the first data collection and dumping program a1 to apply for data dumping rights.
In the embodiment of the present invention, the data dump right application process K2_2 may also be executed first by the second data acquisition and dump program a2, so that the second dump task T2, which is marked as a secondary task, applies for acquiring the data dump right. The specific definition may be self-defined according to an actual application scenario, and is not particularly limited herein.
In this embodiment of the present invention, after acquiring the data dump right, the first dumping task T1 executes a normal data dump, and writes the real-time data into the relational database data table, which may specifically include: the first data collecting and dumping program a1 of the first dumping task T1 executes a normal data dumping task process K5_1, writes the collected real-time data into a data table W1, sets the task state of the first dumping task T1 in a task synchronization table Y1 to an X5 (working) state, and updates the data dumping time point and the state change time. The first dumping task T1 continues to execute the dumping task process K5_1 without interruption.
In the embodiment of the present invention, the data table W1 may include, but is not limited to, any one or more of the following items: IO (input output) point code, data item name, value, and acquisition time, etc.
EXAMPLE six
This embodiment is based on any of the above embodiments, and a specific implementation of breakpoint resuming by the second dump task T2 after the first dump task T1 fails is given.
In this embodiment of the present invention, as shown in fig. 5, when the first dumping task T1 fails during the data dumping process, the performing, by the second dumping task T2, breakpoint resuming according to the real-time data stored in the second buffered data file may include S401-S404:
s401, the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring data dumping right;
s402, acquiring a first data breakpoint B1 of the first dumping task T1 from a data dump time point and a state change time of the task synchronization table Y1;
s403, extracting a continuous data queue including the first data breakpoint B1 from the second buffer data file and executing a continuous transmission process to perform breakpoint continuous transmission;
s404, after the breakpoint resume is completed, the preset recovery state of the task fault table Y2 is changed to be recovered, and the data dump time point and the state change time of the task synchronization table Y1 are updated.
In the embodiment of the present invention, after the first data collecting and dumping program a1 of the first dumping task T1 performs the data dumping task process K5_1 and has a fault, the fault recording process K4_1 of the first data collecting and dumping program a1 may record the time point of the fault data into the task fault table Y2, the recovery state entry in the task fault table Y2 is marked as not recovered, the task state corresponding to the first dumping task T1 in the task synchronization table Y1 is set to be the X2 (fault) state, and the state change time is updated.
In the embodiment of the present invention, the failure of the first dumping task T1 and the failure recovery process implemented by the second dumping task T2 may specifically include: 1. the second dumping task T2 judges the failure of the first dumping task T1 and applies for acquiring the data dumping right; that is, the second dump task T2 accesses the task synchronization table Y1 to determine that the first dump task T1 has a fault, the second dump task T2 applies for and acquires a data dump right, and a task state corresponding to the second dump task T2 in the task synchronization table Y1 is assigned as an X3 (application) state; 2. acquiring a data breakpoint of the first dump task T1 from the task state table Y1, and marking the data breakpoint as a first data breakpoint B1; 3. breakpoint resume is carried out on the first data breakpoint B1, and fault recovery is achieved; that is, the second data collecting and dumping program a2 executes the process of extracting buffered data K3_2, extracting buffered data from the second buffered data file F2, obtaining a continuous data queue including the first data breakpoint B1, and executing the resume process K5_2 (the resume process K5_2 is also a normal data dumping task process of the second dumping task T2); and after the breakpoint resume is completed, changing the recovery state of the task fault table Y2 into recovered state, and updating the data dump time point and the state change time.
In the embodiment of the present invention, during the process of executing data dump task K5_2 continuously by the second data collection and dump program a2, the collected real-time data is written into the data table W1, and the task state of the second dump task T2 in the task synchronization table Y1 is set to X5 (working).
In the embodiment of the present invention, if during the execution of the normal data dump task process K5_2 by the second dump task T2, the fault of the first dump task T1 is cleared, and the normal data dump task can be continuously executed, the first dump task T1 may reapply to acquire the data dump right; to continue writing real-time data into the data table W1 by the first dump task T1.
In the embodiment of the present invention, the recovery status in the task fault table Y2 may be represented by different status flags, which may include, but are not limited to, values 01 and 02, where 01 may represent not recovered and 02 may represent recovered; in other embodiments, any number, letter, character, etc. other than 01 and 02 may be used.
EXAMPLE seven
This embodiment is based on any of the above embodiments, and a specific implementation of breakpoint resuming by the first dumping task T1 after the second dumping task T2 fails is given.
In this embodiment of the present invention, as shown in fig. 6, when the second dumping task T2 fails during the data dumping process and the failure of the first dumping task T1 is eliminated, the performing of the breakpoint resuming by the first dumping task T1 according to the real-time data stored in the first buffered data file may include S501-S504:
s501, the first dumping task T1 judges the fault of the second dumping task T2 through the task state of the task synchronization table Y1 and applies for acquiring data dumping right;
s502, acquiring a second data breakpoint B2 of the second dump task T2 from the data dump time point and the state change time of the task synchronization table Y1;
s503, extracting a continuous data queue comprising the second data breakpoint B2 from the first buffer data file and executing a continuous transmission process to perform breakpoint continuous transmission;
s504, after the breakpoint resume is completed, the preset recovery state of the task fault table Y2 is changed to be recovered, and the data dump time point and the state change time of the task synchronization table Y1 are updated.
In the embodiment of the present invention, after the second data collecting and dumping program a2 of the second dumping task T2 executes the normal data dumping task process K5_2 and has a fault, the fault recording process K4_2 of the second data collecting and dumping program a2 may record the time point of the fault data into the task fault table Y2, mark the recovery state entry in the task fault table Y2 as not recovered, set the task state corresponding to the second dumping task T2 in the task synchronization table Y1 as the X2 (fault) state, and update the state change time.
In the embodiment of the present invention, when the second dumping task T2 fails, and the first dumping task T1 implements a failure recovery process, specifically, the process may include: 1. the first dumping task T1 judges the fault of the second dumping task T2 and applies for acquiring the data dumping right; that is, the first dumping task T1 accesses the task synchronization table Y1 to determine that the second dumping task T2 has a fault, the first dumping task T1 applies for and acquires a data dumping right, and a task state corresponding to the first dumping task T1 in the task synchronization table Y1 is assigned as an X3 (application) state; 2. acquiring a data breakpoint of a second dump task T2 from the task state table Y1, and marking the data breakpoint as a second data breakpoint B2; 3. breakpoint resume is carried out on the second data breakpoint B2, and fault recovery is achieved; that is, the first data collecting and dumping program a1 executes the process of extracting buffered data K3_1, extracting buffered data from the first buffered data file F1, obtaining a continuous data queue including the second data breakpoint B2, and executing the resume process K5_1 (the resume process K5_1 is also a normal data dumping task process of the first resume task T1); and after the breakpoint resume is completed, changing the recovery state of the task fault table Y2 into recovered state, and updating the data dump time point and the state change time.
In the embodiment of the present invention, during the first data collecting and dumping program a1 continuously executing the data dumping task process K5_1, the collected real-time data is written into the data table W1, and the task state of the first dumping task T1 in the task synchronization table Y1 is set to X5 (working).
Example eight
The embodiment provides an embodiment of fault types in the data dump process on the basis of any embodiment.
In the embodiment of the present invention, the failure in the data dump process may include, but is not limited to: the dump program itself fails.
In the embodiment of the present invention, the reasons for the dump task failure scenario (the dump program itself fails) may include: program exception, data dump write exception. In this scenario, the failed dump task cannot continue to perform data dump, and then the data dump can be continued by performing breakpoint resuming through another normal dump task. Through the scheme of the embodiment, the problem of data interruption caused by software fault in the real-time data dump process of the SCADA system is solved, the data interruption caused by abnormal data dump program is prevented, the fault recovery and historical data completion behaviors caused by faults are avoided, the sequential write-in of data dump is ensured, and the stability of the real-time data dump of the SCADA system is improved.
Example nine
A real-time data dump device 1 of a SCADA system for data acquisition and monitoring control, as shown in FIG. 7, may include: a processor 11 and a computer readable storage medium 12, wherein the computer readable storage medium 12 stores instructions, and when the instructions are executed by the processor 11, the method for real-time data dump of the SCADA system is implemented.
The embodiment of the invention comprises the following steps: establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2; starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file; applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database data table; when the first dumping task T1 fails in the process of data dumping, the second dumping task T2 performs breakpoint resuming according to the real-time data stored in the second buffer data file, so as to realize automatic failure recovery; when the second dumping task T2 fails during the data dumping process and the failure of the first dumping task T1 is eliminated, the first dumping task T1 performs breakpoint resuming according to the real-time data stored in the first buffered data file, thereby achieving automatic failure recovery. Through the scheme of the embodiment, the problem of data interruption caused by software fault in the real-time data dump process of the SCADA system is solved, the data interruption caused by abnormal data dump program is prevented, the fault recovery and historical data completion behaviors caused by faults are avoided, the sequential write-in of data dump is ensured, and the stability of the real-time data dump of the SCADA system is improved.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Claims (10)
1. A real-time data dump method of a data acquisition and monitoring control SCADA system comprises the following steps:
establishing a real-time data dump task queue of the SCADA system; the real-time data dump task queue at least comprises: a first dumping task T1 and a second dumping task T2;
starting the first dumping task T1 and the second dumping task T2, continuously collecting real-time data through the first dumping task T1 and the second dumping task T2 respectively, and storing the real-time data into a corresponding first buffer data file and a corresponding second buffer data file;
applying for acquiring a data dump right by the first dump task T1, and executing normal data dump so as to write the real-time data into a preset relational database data table;
when the first dumping task T1 fails in the process of data dumping, the second dumping task T2 performs breakpoint resuming according to the real-time data stored in the second buffer data file, so as to realize automatic failure recovery; when the second dumping task T2 fails during the data dumping process and the failure of the first dumping task T1 is eliminated, the first dumping task T1 performs breakpoint resuming according to the real-time data stored in the first buffered data file, thereby achieving automatic failure recovery.
2. The SCADA system real-time data dump method of claim 1, wherein the establishing a SCADA system real-time data dump task queue comprises:
setting a dump task configuration table;
configuring the first dumping task T1 and the second dumping task T2 in the dumping task configuration table;
wherein, the dump task configuration table comprises: task numbering and collecting and dumping configuration information; the task numbers of the first dumping task T1 and the second dumping task T2 are different, and the acquisition and dumping configuration information are the same.
3. A SCADA system real-time data dump method according to claim 2, characterized in that the acquisition and dump configuration information comprises one or more of the following: task name, task type, data collection and dump program path, data collection and dump program name, collection task number and dump destination data table name.
4. The SCADA system real-time data dump method of claim 1, wherein the method further comprises: after a SCADA system real-time data dump task queue is established, setting the main-standby relationship between the first dump task T1 and the second dump task T2;
the first dump task T1 is a main task, and the second dump task T2 is a standby task.
5. A SCADA system real-time data dump method according to claim 1, wherein the starting the first dump task T1 and the second dump task T2 comprises:
respectively starting a first data acquisition and dump program A1 corresponding to the first dumping task T1 and a second data acquisition and dump program A2 corresponding to the second dumping task T2;
the operation information is accessed and recorded to the preset task synchronization table Y1 through the first data collecting and dumping program A1 and the second data collecting and dumping program A2.
6. A SCADA system real-time data dump method according to claim 5, wherein the task synchronization table Y1 comprises: task coding, task state, state change time and data dump time point;
wherein the task encoding comprises: a first code M1 representing the main task and a second code M2 representing the standby task;
the task states include: an operation state X1, a fault state X2, an application state X3, a continuous transmission state X4 and a working state X5.
7. A SCADA system real-time data dump method according to claim 6, wherein the applying for acquiring a data dump right by the first dump task T1 and executing a normal data dump to write the real-time data into a preset relational database data table comprises:
setting the task state of the first dumping task T1 in the task synchronization table Y1 to be an application state X3 through the first data collecting and dumping program A1 so as to apply for the data dumping right;
after the data dump right is acquired, executing a normal data dump task process through the first data acquisition and dump program A1, and writing acquired real-time data into a preset first relational database data table W1; and setting the task state in the task synchronization table Y1 to a working state X5, and updating the data dump time point and the state change time.
8. A SCADA system real-time data dump method according to claim 6, wherein the performing breakpoint resume by the second dump task T2 according to the real-time data stored in the second buffered data file when the first dump task T1 fails in the process of performing data dump comprises:
the second dumping task T2 judges the fault of the first dumping task T1 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right;
acquiring a first data breakpoint B1 of the first dumping task T1 from a data dumping time point and a state change time of the task synchronization table Y1;
extracting a continuous data queue comprising the first data breakpoint B1 from the second buffered data file and performing a resume process for breakpoint resume;
after the breakpoint continuous transmission is completed, changing the recovery state of a preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1;
when the second dumping task T2 fails during data dumping and the failure of the first dumping task T1 is eliminated, the performing, by the first dumping task T1, breakpoint resuming according to the real-time data stored in the first buffered data file includes:
the first dumping task T1 judges the fault of the second dumping task T2 through the task state of the task synchronization table Y1 and applies for acquiring the data dumping right;
acquiring a second data breakpoint B2 of the second dump task T2 from the data dump time point and the state change time of the task synchronization table Y1;
extracting a continuous data queue comprising the second data breakpoint B2 from the first buffered data file and performing a resume process for breakpoint resume;
and after the breakpoint continuous transmission is completed, changing the recovery state of the preset task fault table Y2 into recovered state, and updating the data dump time point and the state change time of the task synchronization table Y1.
9. A SCADA system real-time data dump method according to any one of claims 1-8, characterized in that the fault in the data dump process comprises: the dump program itself fails.
10. A real-time data dump device of a data acquisition and monitoring control SCADA system comprises: processor and computer readable storage medium, wherein instructions are stored, wherein the instructions, when executed by the processor, implement the SCADA system real-time data dump method according to any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810897123.8A CN109150596B (en) | 2018-08-08 | 2018-08-08 | SCADA system real-time data dump method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810897123.8A CN109150596B (en) | 2018-08-08 | 2018-08-08 | SCADA system real-time data dump method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109150596A CN109150596A (en) | 2019-01-04 |
| CN109150596B true CN109150596B (en) | 2021-08-03 |
Family
ID=64792160
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810897123.8A Active CN109150596B (en) | 2018-08-08 | 2018-08-08 | SCADA system real-time data dump method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109150596B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113515516A (en) * | 2021-08-09 | 2021-10-19 | 北京字节跳动网络技术有限公司 | Data processing method and device, computer equipment and computer storage medium |
| CN116302646B (en) * | 2023-02-24 | 2024-03-29 | 荣耀终端有限公司 | Fault positioning method, system, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102594874A (en) * | 2008-06-20 | 2012-07-18 | 华为技术有限公司 | Synchronization processing method and device |
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
| CN107018554A (en) * | 2017-03-07 | 2017-08-04 | 广东欧珀移动通信有限公司 | A kind of data transmission method, device and intelligent terminal |
| CN107423303A (en) * | 2016-05-24 | 2017-12-01 | 北京京东尚科信息技术有限公司 | The method and system of data syn-chronization |
| CN107967118A (en) * | 2016-10-19 | 2018-04-27 | 南京拓界信息技术有限公司 | A kind of method and apparatus realized hard disc data high speed dump and checked |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8908531B2 (en) * | 2011-08-25 | 2014-12-09 | At&T Mobility Ii Llc | Communication gateway for facilitating communications with a supervisory control and data aquisition system |
| CN104391764B (en) * | 2014-10-22 | 2018-02-16 | 上海海得控制系统股份有限公司 | A kind of MATLAB software method and system |
-
2018
- 2018-08-08 CN CN201810897123.8A patent/CN109150596B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102594874A (en) * | 2008-06-20 | 2012-07-18 | 华为技术有限公司 | Synchronization processing method and device |
| CN104699541A (en) * | 2015-03-30 | 2015-06-10 | 北京奇虎科技有限公司 | Method, device, data transmission assembly and system for synchronizing data |
| CN107423303A (en) * | 2016-05-24 | 2017-12-01 | 北京京东尚科信息技术有限公司 | The method and system of data syn-chronization |
| CN107967118A (en) * | 2016-10-19 | 2018-04-27 | 南京拓界信息技术有限公司 | A kind of method and apparatus realized hard disc data high speed dump and checked |
| CN107018554A (en) * | 2017-03-07 | 2017-08-04 | 广东欧珀移动通信有限公司 | A kind of data transmission method, device and intelligent terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109150596A (en) | 2019-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110088744B (en) | Database maintenance method and system | |
| CN114584539B (en) | Cloud upgrading method and system for industrial field device | |
| CN106789741A (en) | The consuming method and device of message queue | |
| CN113946499A (en) | Micro-service link tracking and performance analysis method, system, equipment and application | |
| CN109150596B (en) | SCADA system real-time data dump method and device | |
| CN112416957B (en) | Data increment updating method and device based on data model layer and computer equipment | |
| WO2019061364A1 (en) | Failure analyzing method and related device | |
| CN109116818B (en) | Real-time data dump method and device during SCADA system upgrade | |
| CN114816820A (en) | Method, device, equipment and storage medium for repairing chproxy cluster fault | |
| CN112491047A (en) | Fault positioning method, device and equipment for secondary equipment of transformer substation | |
| CN109241029B (en) | Method and device for realizing smooth migration of SCADA system database | |
| CN113672306A (en) | Server component self-checking abnormity recovery method, device, system and medium | |
| WO2019005294A1 (en) | Recovery of application from error | |
| CN109189599B (en) | Method and device for improving real-time data dump stability of SCADA system | |
| CN114841678B (en) | Post data exchange method, data exchange system, server and storage medium | |
| CN108804239B (en) | Platform integration method and device, computer equipment and storage medium | |
| CN110968456A (en) | Method and device for processing fault disk in distributed storage system | |
| CN115687019A (en) | Database cluster fault processing method, intelligent monitoring platform, equipment and medium | |
| CN114398334A (en) | Prometheus remote storage method and system based on ZNBase cluster | |
| CN113836223A (en) | Heterogeneous database data synchronization method | |
| CN116149707B (en) | Method and device for detecting and avoiding upgrading risk of distributed system | |
| CN115134213B (en) | Disaster recovery method, device, equipment and storage medium | |
| CN113535528B (en) | Log management system, method and medium for distributed graph iterative computation job | |
| CN117632559A (en) | Fault disk repairing method and device, storage medium and electronic equipment | |
| CN116775361A (en) | Virtual cloud platform fault repairing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |