CN111726252A - Abnormal data reporting method and tool for industrial equipment - Google Patents
Abnormal data reporting method and tool for industrial equipment Download PDFInfo
- Publication number
- CN111726252A CN111726252A CN202010553021.1A CN202010553021A CN111726252A CN 111726252 A CN111726252 A CN 111726252A CN 202010553021 A CN202010553021 A CN 202010553021A CN 111726252 A CN111726252 A CN 111726252A
- Authority
- CN
- China
- Prior art keywords
- data
- log
- industrial equipment
- module
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0604—Management of faults, events, alarms or notifications using filtering, e.g. reduction of information by using priority, element types, position or time
- H04L41/0627—Management of faults, events, alarms or notifications using filtering, e.g. reduction of information by using priority, element types, position or time by acting on the notification or alarm source
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses an abnormal data reporting method for industrial equipment, and relates to the technical field of data monitoring. The invention also discloses an abnormal data reporting tool for the industrial equipment, which is realized by the lightweight log acquisition module, the data conversion module and the data analysis module, monitors some monitoring fields in the log file, and reports the monitoring fields to the industrial Internet of things platform or the Internet of things platform when the monitoring fields exceed the early warning range, so that the condition of untimely reporting is avoided.
Description
Technical Field
The invention relates to the technical field of data monitoring, in particular to an abnormal data reporting method and tool for industrial equipment.
Background
With the increasing number of devices accessing the industrial internet and the internet of things, the storage scale of the data of the devices is increased day by day. In order to store massive device data, one method is to use distributed storage to increase the storage scale of a storage server, and the other method is to preprocess the device data and perform related screening and cleaning, so as to reduce the storage capacity of the data.
For some equipment data, all the equipment data are not required to be put in storage in a normal working range generally, and only when a certain item of data exceeds a certain range, the data need to be reported in time.
Disclosure of Invention
Aiming at the requirements and the defects of the prior art development, the invention provides an abnormal data reporting method and tool for industrial equipment.
Firstly, the invention discloses an abnormal data reporting method aiming at industrial equipment, and the technical scheme adopted for solving the technical problems is as follows:
the method needs to monitor a data log of the industrial equipment in real time, analyzes fields of the data log, sets an early warning range for the monitoring fields of the data log, does not report when the monitoring fields are within the early warning range, and timely sends the monitoring fields to an industrial Internet of things platform or an Internet of things platform for reporting when the monitoring fields exceed the early warning range.
Specifically, the abnormal data reporting method dynamically monitors the data log of the industrial equipment through the lightweight log acquisition module, senses the update change of the data log through the lightweight log acquisition module, and pushes the acquisition result to the log data collection engine.
More specifically, the related lightweight log collection module is deployed in the target industrial equipment, and the lightweight log collection module dynamically monitors a data log of the target industrial equipment and generates a plurality of log files.
More specifically, the abnormal data reporting method filters, analyzes, converts and structures a plurality of log files from the target industrial equipment through a data conversion module, the data conversion module is deployed in a gateway or an edge computing platform of the target industrial equipment, and the data conversion module is used for converting data of the plurality of log files into a uniform format and outputting the uniform format.
More specifically, the abnormal data reporting method analyzes data output by the data conversion module through the data analysis module, the data analysis module is deployed in a gateway or an edge computing platform of the target industrial equipment, a monitoring field of a data log and an early warning range corresponding to the monitoring field can be set through the data analysis module, the abnormal data reporting method does not report when the monitoring field is within the early warning range, and the abnormal data reporting method timely sends the abnormal data to the industrial internet of things platform or the internet of things platform for reporting when the monitoring field exceeds the early warning range.
Secondly, the invention also discloses an abnormal data reporting tool for the industrial equipment, and the technical scheme adopted for solving the technical problems is as follows:
an abnormal data reporting tool for industrial equipment, which structurally comprises:
the system comprises a lightweight log acquisition module, a data acquisition module and a data processing module, wherein the lightweight log acquisition module is deployed in target industrial equipment and is used for dynamically monitoring data logs of the industrial equipment and generating a plurality of log files;
the data conversion module is deployed on a gateway or an edge computing platform of the target industrial equipment and is used for filtering, analyzing, converting and structuring a plurality of log files from the lightweight log acquisition module, and converting the data of the log files into a uniform format and outputting the uniform format;
and the data analysis module is deployed in a gateway or an edge computing platform of the target industrial equipment, is used for analyzing the data output by the data conversion module, and is also used for setting a monitoring field of the data log and an early warning range corresponding to the monitoring field, wherein the monitoring field is not reported when being in the early warning range, and the monitoring field is timely sent to the industrial Internet of things platform or the Internet of things platform for reporting when exceeding the early warning range.
Specifically, the related lightweight log collection module comprises a searching unit and a collection unit, wherein the searching unit is used for searching a log file path to be collected and starting a collection unit in each searched file, and the collection unit is used for reading a single log file and sending log data to the data conversion module.
Preferably, the plurality of log files generated by the lightweight log collection module includes, but is not limited to, logs from a system and a server.
More specifically, the data conversion module comprises an input unit, a filtering unit and an output unit,
the input unit is used for setting data acquisition sources, including mutual authentication with the acquisition unit,
the filtering unit is used for processing the format of the log data, including data type conversion, data filtering, field analysis, field addition and field modification,
the output unit is used for outputting the data processed by the filtering unit to the data analysis module.
Preferably, the filtering unit converts the received log files into a uniform format, and transmits the uniform format to the data analysis module through the output unit.
Compared with the prior art, the abnormal data reporting method and the tool for the industrial equipment have the beneficial effects that:
according to the invention, the log file in the data reporting process is dynamically monitored and analyzed, so that monitoring of some monitoring fields in the log file is realized, and the monitoring fields are reported to the industrial Internet of things platform or the Internet of things platform when exceeding the early warning range, so that the condition of untimely reporting is avoided.
Drawings
FIG. 1 is a block diagram of the framework of the present invention.
Detailed Description
In order to make the technical scheme, the technical problems to be solved and the technical effects of the present invention more clearly apparent, the following technical scheme of the present invention is clearly and completely described with reference to the specific embodiments.
The first embodiment is as follows:
the embodiment provides an abnormal data reporting method for industrial equipment, which needs to monitor a data log of the industrial equipment in real time, analyze fields of the data log, set an early warning range for monitoring fields of the data log, do not report when the monitoring fields are within the early warning range, and send the monitoring fields to an industrial internet of things platform or an internet of things platform in time for reporting when the monitoring fields exceed the early warning range.
With reference to fig. 1, in this embodiment, a lightweight log collection module is used to dynamically monitor a data log of an industrial device. The lightweight log collection module is deployed on target industrial equipment, senses the updating change of the data log and generates a plurality of log files.
With reference to fig. 1, in this embodiment, a data conversion module is used to filter, analyze, convert and structure a plurality of log files from a target industrial device. The data conversion module is deployed in a gateway or an edge computing platform of the target industrial equipment and used for converting data of the plurality of log files into a uniform format and outputting the uniform format.
With reference to fig. 1, in this embodiment, the data output by the data conversion module is analyzed by the data analysis module. The data analysis module is deployed in a gateway or an edge computing platform of the target industrial equipment, a monitoring field of the data log and an early warning range corresponding to the monitoring field can be set through the data analysis module, the monitoring field is not reported when being in the early warning range, and the monitoring field is timely sent to the industrial Internet of things platform or the Internet of things platform to be reported when exceeding the early warning range.
Example two:
with reference to fig. 1, the present embodiment provides an abnormal data reporting tool for an industrial device, which includes: the system comprises a lightweight log acquisition module, a data conversion module and a data analysis module.
The lightweight log acquisition module is deployed on target industrial equipment and used for dynamically monitoring data logs of the industrial equipment and generating a plurality of log files. In this embodiment, the lightweight log collection module includes a search unit and a collection unit, the search unit is configured to search a log file path to be collected, and start a collection unit in each searched file, and the collection unit is configured to read a single log file and send log data to the data conversion module. It is to be added that the plurality of log files generated by the lightweight log collection module include, and are not limited to, logs from systems and server logs.
The data conversion module is deployed in a gateway or an edge computing platform of the target industrial equipment and used for filtering, analyzing, converting and structuring a plurality of log files from the lightweight log acquisition module, and converting the data of the log files into a uniform format and outputting the uniform format. In this embodiment, the data conversion module includes an input unit, a filtering unit, and an output unit, where the input unit is configured to set a data acquisition source, and includes mutual authentication with the acquisition unit, the filtering unit is configured to perform format processing on log data, including data type conversion, data filtering, field parsing, field adding, and field modifying, so as to convert a plurality of received log files into a unified format, and the output unit is configured to output data processed by the filtering unit to the data analysis module.
The data analysis module is deployed in a gateway or an edge computing platform of the target industrial equipment, and is used for analyzing data output by the data conversion module, setting a monitoring field of the data log and an early warning range corresponding to the monitoring field, not reporting when the monitoring field is within the early warning range, and timely sending the monitoring field to the industrial Internet of things platform or the Internet of things platform for reporting when the monitoring field exceeds the early warning range.
In summary, by using the abnormal data reporting method and tool for the industrial equipment, the log file in the data reporting process is dynamically monitored and analyzed, so that monitoring of some monitoring fields in the log file is realized, and the monitoring fields are reported to the industrial internet of things platform or the internet of things platform when exceeding the early warning range, thereby avoiding the situation of untimely reporting.
The principles and embodiments of the present invention have been described in detail using specific examples, which are provided only to aid in understanding the core technical content of the present invention. Based on the above embodiments of the present invention, those skilled in the art should make any improvements and modifications to the present invention without departing from the principle of the present invention, and therefore, the present invention should fall into the protection scope of the present invention.
Claims (10)
1. The method is characterized in that a data log of the industrial equipment needs to be monitored in real time, fields of the data log are analyzed, an early warning range is set for monitoring fields of the data log, the monitoring fields are not reported when being within the early warning range, and the monitoring fields are timely sent to an industrial Internet of things platform or an Internet of things platform to be reported when exceeding the early warning range.
2. The abnormal data reporting method for the industrial equipment according to claim 1, wherein the method dynamically monitors the data log of the industrial equipment through a lightweight log collection module, and the lightweight log collection module senses the update change of the data log and pushes the collection result to a log data collection engine.
3. The abnormal data reporting method for the industrial equipment according to claim 2, wherein the lightweight log collection module is deployed in the target industrial equipment, and the lightweight log collection module dynamically monitors the data log of the target industrial equipment and generates a plurality of log files.
4. The method as claimed in claim 3, wherein the method filters, parses, converts and structures the plurality of log files from the target industrial device through a data conversion module, the data conversion module is deployed in a gateway or an edge computing platform of the target industrial device, and the data conversion module is configured to convert the data of the plurality of log files into a uniform format and output the uniform format.
5. The method as claimed in claim 4, wherein the data analysis module analyzes the data output by the data conversion module, the data analysis module is deployed in a gateway or an edge computing platform of a target industrial device, the data analysis module can set a monitoring field of a data log and an early warning range corresponding to the monitoring field, the monitoring field is not reported when the monitoring field is within the early warning range, and the monitoring field is timely sent to an industrial internet of things platform or an internet of things platform to be reported when the monitoring field exceeds the early warning range.
6. An abnormal data reporting tool for an industrial device, the abnormal data reporting tool being structurally characterized by comprising:
the system comprises a lightweight log acquisition module, a data acquisition module and a data processing module, wherein the lightweight log acquisition module is deployed in target industrial equipment and is used for dynamically monitoring data logs of the industrial equipment and generating a plurality of log files;
the data conversion module is deployed on a gateway or an edge computing platform of the target industrial equipment and is used for filtering, analyzing, converting and structuring a plurality of log files from the lightweight log acquisition module, and converting the data of the log files into a uniform format and outputting the uniform format;
and the data analysis module is deployed in a gateway or an edge computing platform of the target industrial equipment, is used for analyzing the data output by the data conversion module, and is also used for setting a monitoring field of the data log and an early warning range corresponding to the monitoring field, wherein the monitoring field is not reported when being in the early warning range, and the monitoring field is timely sent to the industrial Internet of things platform or the Internet of things platform for reporting when exceeding the early warning range.
7. The abnormal data reporting tool for industrial equipment as claimed in claim 6, wherein the lightweight log collection module comprises a search unit and a collection unit, the search unit is configured to search for a log file path to be collected and start a collection unit for each searched file, and the collection unit is configured to read a single log file and send log data to the data conversion module.
8. The abnormal data reporting tool for industrial equipment of claim 7, wherein the plurality of log files generated by the lightweight log collection module include, but are not limited to, system logs and server logs.
9. The abnormal data reporting tool for industrial equipment as claimed in claim 7, wherein the data conversion module comprises an input unit, a filtering unit and an output unit,
the input unit is used for setting a data acquisition source and comprises mutual authentication with the acquisition unit,
the filtering unit is used for processing the format of the log data, including data type conversion, data filtering, field analysis, field addition and field modification,
the output unit is used for outputting the data processed by the filtering unit to the data analysis module.
10. The abnormal data reporting tool for industrial equipment as claimed in claim 9, wherein the filtering unit converts the received log files into a unified format and transmits the unified format to the data analysis module through the output unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010553021.1A CN111726252A (en) | 2020-06-17 | 2020-06-17 | Abnormal data reporting method and tool for industrial equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010553021.1A CN111726252A (en) | 2020-06-17 | 2020-06-17 | Abnormal data reporting method and tool for industrial equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111726252A true CN111726252A (en) | 2020-09-29 |
Family
ID=72567176
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010553021.1A Pending CN111726252A (en) | 2020-06-17 | 2020-06-17 | Abnormal data reporting method and tool for industrial equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111726252A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112462724A (en) * | 2020-12-11 | 2021-03-09 | 四川川能智网实业有限公司 | Data monitoring system based on industrial internet |
| CN112488877A (en) * | 2020-12-11 | 2021-03-12 | 四川川能智网实业有限公司 | Industrial energy consumption visualization system for energy service platform |
| CN112488558A (en) * | 2020-12-11 | 2021-03-12 | 四川川能智网实业有限公司 | Energy consumption monitoring and analyzing system based on industrial internet |
| CN114726631A (en) * | 2022-04-12 | 2022-07-08 | 中国电信股份有限公司 | Security protection method of identifier resolution system architecture and related equipment |
| CN115017014A (en) * | 2022-07-11 | 2022-09-06 | 广东利通科技投资有限公司 | Highway electromechanical monitoring system and method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881354A (en) * | 2015-06-30 | 2015-09-02 | 北京奇虎科技有限公司 | Cloud disk monitoring method and device |
| CN109144817A (en) * | 2018-08-03 | 2019-01-04 | 江苏满运软件科技有限公司 | A kind of daily record data monitoring system and method |
| CN109617737A (en) * | 2018-12-27 | 2019-04-12 | 携程计算机技术（上海）有限公司 | The monitoring method and system of the log platform of internet |
| CN111092946A (en) * | 2019-12-18 | 2020-05-01 | 博依特(广州)工业互联网有限公司 | Data processing method and system applied to edge computing gateway |
| CN111124839A (en) * | 2019-12-31 | 2020-05-08 | 中国银行股份有限公司 | Distributed log data monitoring method and device |
| CN111143286A (en) * | 2019-12-13 | 2020-05-12 | 无锡华云数据技术服务有限公司 | Cloud platform log management method and system |
-
2020
- 2020-06-17 CN CN202010553021.1A patent/CN111726252A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104881354A (en) * | 2015-06-30 | 2015-09-02 | 北京奇虎科技有限公司 | Cloud disk monitoring method and device |
| CN109144817A (en) * | 2018-08-03 | 2019-01-04 | 江苏满运软件科技有限公司 | A kind of daily record data monitoring system and method |
| CN109617737A (en) * | 2018-12-27 | 2019-04-12 | 携程计算机技术（上海）有限公司 | The monitoring method and system of the log platform of internet |
| CN111143286A (en) * | 2019-12-13 | 2020-05-12 | 无锡华云数据技术服务有限公司 | Cloud platform log management method and system |
| CN111092946A (en) * | 2019-12-18 | 2020-05-01 | 博依特(广州)工业互联网有限公司 | Data processing method and system applied to edge computing gateway |
| CN111124839A (en) * | 2019-12-31 | 2020-05-08 | 中国银行股份有限公司 | Distributed log data monitoring method and device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112462724A (en) * | 2020-12-11 | 2021-03-09 | 四川川能智网实业有限公司 | Data monitoring system based on industrial internet |
| CN112488877A (en) * | 2020-12-11 | 2021-03-12 | 四川川能智网实业有限公司 | Industrial energy consumption visualization system for energy service platform |
| CN112488558A (en) * | 2020-12-11 | 2021-03-12 | 四川川能智网实业有限公司 | Energy consumption monitoring and analyzing system based on industrial internet |
| CN114726631A (en) * | 2022-04-12 | 2022-07-08 | 中国电信股份有限公司 | Security protection method of identifier resolution system architecture and related equipment |
| CN114726631B (en) * | 2022-04-12 | 2023-10-03 | 中国电信股份有限公司 | Safety protection method of identification analysis system architecture and related equipment |
| CN115017014A (en) * | 2022-07-11 | 2022-09-06 | 广东利通科技投资有限公司 | Highway electromechanical monitoring system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111726252A (en) | Abnormal data reporting method and tool for industrial equipment | |
| CN110740141A (en) | integration network security situation perception method, device and computer equipment | |
| CN107577588B (en) | Intelligent operation and maintenance system for mass log data | |
| CN107943668B (en) | Computer server cluster log monitoring method and monitor supervision platform | |
| CN108039959B (en) | Data situation perception method, system and related device | |
| CN105608144B (en) | A kind of big data analysis stage apparatus and method based on multilayered model iteration | |
| US20190108112A1 (en) | System and method for generating a log analysis report from a set of data sources | |
| CN112433998B (en) | Multisource heterogeneous data acquisition and convergence system and method based on power system | |
| CN113360554B (en) | Method and equipment for extracting, converting and loading ETL (extract transform load) data | |
| CN108521339B (en) | Feedback type node fault processing method and system based on cluster log | |
| CN110690984A (en) | Spark-based big data weblog acquisition, analysis and early warning method and system | |
| CN102780726A (en) | Log analysis method and log analysis system based on WEB platform | |
| CN102918534A (en) | Query pipeline | |
| CN111274095A (en) | Log data processing method, device, equipment and computer readable storage medium | |
| CN103248625A (en) | Monitoring method and system for abnormal operation of web crawler | |
| CN112948436A (en) | Alarm work order merging method, device, equipment and storage medium | |
| CN112711520A (en) | Method, device and equipment for processing abnormal log information and storage medium | |
| CN112463543A (en) | Business data monitoring method, rule data generating method, device and system | |
| Ferry et al. | Towards a big data platform for managing machine generated data in the cloud | |
| CN111506672B (en) | Method, device, equipment and storage medium for analyzing environment-friendly monitoring data in real time | |
| CN110609761B (en) | Method and device for determining fault source, storage medium and electronic equipment | |
| Hou et al. | Alarms association rules based on sequential pattern mining algorithm | |
| CN111130882A (en) | Monitoring system and method of network equipment | |
| US20150066947A1 (en) | Indexing apparatus and method for search of security monitoring data | |
| CN115664992A (en) | Network operation data processing method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200929 |
|
| RJ01 | Rejection of invention patent application after publication |