CN112134860A - Processing method and device of environmental monitoring data, air micro-station and storage medium - Google Patents
Processing method and device of environmental monitoring data, air micro-station and storage medium Download PDFInfo
- Publication number
- CN112134860A CN112134860A CN202010942790.0A CN202010942790A CN112134860A CN 112134860 A CN112134860 A CN 112134860A CN 202010942790 A CN202010942790 A CN 202010942790A CN 112134860 A CN112134860 A CN 112134860A
- Authority
- CN
- China
- Prior art keywords
- data
- alarm
- environmental monitoring
- processing
- national standard
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/562—Brokering proxy services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/08—Protocols for interworking; Protocol conversion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The embodiment of the invention provides a processing method and device of environmental monitoring data, an air micro-station and a storage medium. The processing method of the environment monitoring data comprises the following steps: acquiring a data packet of environment monitoring data, and converting the data packet into national standard protocol data; acquiring national standard protocol data through a data acquisition component; analyzing the national standard protocol data through a data analysis component; performing alarm calculation on the analyzed national standard protocol data through a data calculation component to determine alarm data; and the alarm data is stored in a database through the data processing assembly, and alarm information is output. According to the method for processing the environmental monitoring data, provided by the embodiment of the invention, the time interval from the time when the environmental monitoring equipment monitors the environmental monitoring data to the time when the relevant working personnel processes the environmental monitoring data according to the alarm information is very short, the calculation process is simple, the error probability is extremely low, and the problems of untimely and inaccurate alarm are effectively solved.
Description
Technical Field
The embodiment of the invention relates to the technical field of environmental monitoring, in particular to a processing method of environmental monitoring data, a processing device of the environmental monitoring data, an air micro-station and a computer readable storage medium.
Background
The air micro-station mainly carries out 24-hour continuous online monitoring on conventional pollution factors and meteorological parameters in the air, provides analyzed data for an environmental protection bureau to serve as a reference for the quality of the air and assists environmental protection decision. Typically, the environmental monitoring factors include: pollution of very fine particulate matter (PM2.5, PM10), ozone, sulfur dioxide, carbon monoxide, hydrogen sulfide, nitrogen oxides, volatile organic pollutants, total suspended particulate matter, lead, benzene, meteorological parameters, visibility, and the like.
In order to monitor air, a plurality of air micro-stations are generally set around a national standard station in an environmental key city, environment monitoring equipment is installed in the stations for continuous automatic monitoring, and a client analyzes and stores environment monitoring data in real time and provides the environment monitoring data for an environmental protection bureau. However, the processing of the environment monitoring data by the client is not timely and accurate, so that the alarm cannot be timely given out when the environment monitoring data exceeds the standard, and the staff cannot timely process the environment, thereby influencing the surrounding environment.
Disclosure of Invention
Embodiments of the present invention are directed to solving at least one of the technical problems occurring in the prior art.
To this end, a first aspect of an embodiment of the present invention provides a method for processing environmental monitoring data.
A second aspect of embodiments of the present invention provides an apparatus for processing environmental monitoring data.
A third aspect of embodiments of the present invention provides an air micro station.
A fourth aspect of embodiments of the present invention provides a computer-readable storage medium.
In view of this, according to a first aspect of the embodiments of the present invention, there is provided a processing method of environment monitoring data, the processing method including: acquiring a data packet of environment monitoring data, and converting the data packet into national standard protocol data; acquiring national standard protocol data through a data acquisition component; analyzing the national standard protocol data through a data analysis component; performing alarm calculation on the analyzed national standard protocol data through a data calculation component to determine alarm data; the alarm data is stored in a database through a data processing component, and alarm information is output; the data acquisition component, the data analysis component, the data calculation component and the data processing component are components of a streaming big data processing framework.
The method for processing the environmental monitoring data provided by the embodiment of the invention is used for the client. Firstly, a data packet of environment monitoring data is obtained from environment monitoring equipment, and then the data packet is converted into national standard protocol data, namely the binary or hexadecimal data packet is analyzed into plaintext national standard protocol data, so that the environment monitoring data can be used universally. Specifically, the national standard protocol data is the HJ/T212 national standard protocol data, which is a data transmission standard protocol specified by the national environmental protection industry. Secondly, performing alarm calculation on the national standard protocol data through a streaming big data processing framework to determine whether the national standard protocol data converted by the data packet includes alarm data, and when the national standard protocol data is determined to be the alarm data, storing the alarm data into an alarm table of a database and outputting alarm information so that related workers can check the alarm data in time. Specifically, the streaming big data processing framework is a strom streaming big data processing framework. The data can be processed in large data quantity and large batch through the strom streaming big data processing framework, so that the data is accurate, rapid and is not lost. According to the method for processing the environmental monitoring data, provided by the embodiment of the invention, the time interval from the time when the environmental monitoring equipment monitors the environmental monitoring data to the time when the relevant working personnel processes the environmental monitoring data according to the alarm information is very short, the calculation process is simple, the error probability is extremely low, and the problems of untimely and inaccurate alarm are effectively solved.
In addition, the processing method of the environmental monitoring data provided by the technical scheme of the invention also has the following additional technical characteristics:
in one possible design, before performing alarm calculation on the national standard protocol data through the streaming big data processing framework, the method further comprises the following steps: and transmitting the national standard protocol data to the streaming big data processing framework through the message middleware.
In the design, the national standard protocol data is transmitted to the streaming big data processing framework through the message middleware, and particularly, the message middleware is kafka message middleware, namely, the national standard protocol data is continuously transmitted to the streaming big data processing framework through the kafka message middleware. The kafka message middleware can transmit data in large data volume and large batch, thereby ensuring that the data is accurate, rapid and is not lost.
In one possible design, before obtaining the data packet of the environmental monitoring data, the method further includes: and establishing connection with the environment monitoring equipment through a transmission control protocol.
In the design, the source of the data packet of the environment monitoring data is the environment monitoring device, and the client-side establishes connection with the environment monitoring device through three-way handshake of a transmission control protocol (namely, a TCP protocol), so as to obtain the data packet of the environment monitoring data. The environment monitoring equipment adopts a high-sensitivity sensor, and can quickly identify and collect monitoring conditions in the atmosphere. The data is transmitted through the transmission control protocol, so that the data can be ensured to be accurate, rapid and not lost.
In one possible design, parsing the national standard protocol data through the data parsing component specifically includes: and converting the national standard protocol data into data in a data exchange format through a data analysis component.
In the design, the national standard protocol data is converted into data in a data exchange format through the data analysis component, and the alarm calculation is performed by adopting the data in the data exchange format, so that the calculation process is simple, and the error probability is reduced. Specifically, the data exchange format is a json format, which is a lightweight data exchange format, is relatively simple, easy to read and write, compressed, small in occupied bandwidth, easy to analyze, and capable of supporting multiple languages.
In one possible design, the alarm calculation of the parsed national standard protocol data by the data calculation component to determine the alarm data specifically includes: determining data exchange format data corresponding to the environmental monitoring factor through a data calculation component; and calculating the data exchange format data corresponding to the environment monitoring factors according to an alarm rule to determine alarm data.
In this design, the environmental monitoring factors typically include: pollution of very fine particulate matter (PM2.5, PM10), ozone, sulfur dioxide, carbon monoxide, hydrogen sulfide, nitrogen oxides, volatile organic pollutants, total suspended particulate matter, lead, benzene, meteorological parameters, visibility, and the like. As will be readily understood by those skilled in the art, different environmental monitoring factors have different standard data, and thus different environmental monitoring factors correspond to different alarm rules. The data exchange format data corresponding to the environment monitoring factors are determined through the data calculation component, namely the environment monitoring factors are matched with the corresponding data exchange format data, the data exchange format data corresponding to each environment monitoring factor are calculated according to the corresponding alarm rule, so that the alarm data are determined, and the problem of inaccurate alarm is effectively solved.
In one possible design, the calculating the data exchange format data corresponding to the environment monitoring factor according to the alarm rule to determine the alarm data specifically includes: comparing the data exchange format data corresponding to the environment monitoring factor with the standard data; when the data value of the data exchange format data is greater than that of the standard data, determining the data exchange format data as alarm data; and when the data value of the data exchange format data is less than or equal to the data value of the standard data, determining the data exchange format data as normal data.
In the design, comparing data exchange format data corresponding to each environment monitoring factor with standard data corresponding to each environment monitoring factor, and determining the data exchange format data as alarm data when the data value of the data exchange format data is greater than that of the standard data; and when the data value of the data exchange format data is less than or equal to the data value of the standard data, determining the data exchange format data as normal data. That is to say, when the data value of the data in the data exchange format exceeds the standard, the data in the data exchange format is determined as alarm data, the alarm data is stored in the database, and the data in the data exchange format determined as normal data can also be stored in the database as a retention record for subsequent viewing and verification. Meanwhile, an alarm message is output according to the alarm data to prompt relevant workers to process in time.
According to a second aspect of the embodiments of the present invention, there is provided an apparatus for processing environmental monitoring data, including: the storage stores a computer program, and the processor executes the steps of the processing method of the environmental monitoring data provided by any one of the above technical solutions when executing the computer program, so that the method has all the beneficial technical effects of the processing method of the environmental monitoring data, and is not described herein again.
According to a third aspect of embodiments of the present invention, there is provided an air micro station comprising: the processing device for environment monitoring data provided by the above technical solution has all the beneficial technical effects of the processing device for environment monitoring data, and is not described herein again.
In one possible design, the air micro-station further comprises: the environment monitoring equipment is connected with the processing device of the environment monitoring data through a transmission control protocol; and the alarm device is connected with the processing device of the environmental monitoring data and used for prompting alarm information.
In this design, environmental monitoring data is gathered to environmental monitoring equipment, and environmental monitoring data's processing apparatus passes through transmission control protocol and is connected with environmental monitoring equipment establishment to acquire environmental monitoring data's data package, handle the data package, output alarm information. The alarm device is used for prompting according to alarm information output by the processing device of the environment monitoring data, so that untimely alarm is avoided, related workers can rapidly process according to the prompted alarm information, and the environment is effectively protected.
According to a fourth aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, on which a computer program is stored, where the computer program, when being executed by a processor, performs the steps of the method for processing environmental monitoring data according to any of the above technical solutions, so that all the beneficial technical effects of the method for processing environmental monitoring data are achieved, and details are not repeated herein.
Additional aspects and advantages in accordance with the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 shows a flow diagram of a method of processing environmental monitoring data according to one embodiment of the invention;
FIG. 2 is a flow diagram illustrating a method of processing environmental monitoring data according to another embodiment of the invention;
fig. 3 is a flow chart illustrating a method for processing environmental monitoring data according to still another embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced otherwise than as specifically described herein, and thus the scope of the present invention is not limited by the specific embodiments disclosed below.
Methods, apparatus, air micro-stations and storage media for processing environmental monitoring data provided according to some embodiments of the present invention are described below with reference to fig. 1-3.
The first embodiment is as follows:
as shown in fig. 1, a first embodiment provides a processing method of environmental monitoring data, where the processing method includes:
102, acquiring a data packet of the environmental monitoring data, and converting the data packet into national standard protocol data;
104, acquiring national standard protocol data through a data acquisition component;
106, analyzing the national standard protocol data through a data analysis component;
and step 110, storing the alarm data to a database through a data processing component, and outputting alarm information.
The method for processing the environmental monitoring data provided by the embodiment of the invention is used for the client.
Firstly, a data packet of environment monitoring data is obtained from environment monitoring equipment, and then the data packet is converted into national standard protocol data, namely the binary or hexadecimal data packet is analyzed into plaintext national standard protocol data, so that the environment monitoring data can be used universally. Specifically, the national standard protocol data is the HJ/T212 national standard protocol data, which is a data transmission standard protocol specified by the national environmental protection industry.
Secondly, the data acquisition component, the data parsing component, the data computation component, and the data processing component are components of a streaming big data processing framework. And performing alarm calculation on the national standard protocol data through a streaming big data processing framework to determine whether the national standard protocol data converted by the data packet comprises alarm data, storing the alarm data to a database when the national standard protocol data is determined to be the alarm data, and outputting the alarm information so that related workers can check the alarm data in time. Specifically, the streaming big data processing framework is a strom streaming big data processing framework. The data can be processed in large data quantity and large batch through the strom streaming big data processing framework, so that the data is accurate, rapid and is not lost.
The strom streaming big data processing framework comprises an spout component and a bolt component, wherein the data acquisition component is the spout component, and the data analysis component, the data calculation component and the data processing component are different bolt components. And receiving the national standard protocol data continuously transmitted from the kafka message middleware through the spout component, transmitting the national standard protocol data to the bolt component for processing, performing general analysis on the national standard protocol data by using the first bolt component, performing alarm calculation on the analyzed national standard protocol data by using the second bolt component to determine alarm data, and storing the alarm data into a database by using the third bolt component to output alarm information. The data can be processed in time through the components of the strom streaming big data processing framework, the calculation process is simple, and the error probability is extremely low.
In the related art, once a large amount of data is processed, a program may be crashed, so that the environment monitoring data processing is not timely. According to the method for processing the environmental monitoring data, which is provided by the embodiment of the invention, the time interval is very short, the flow is simple, the data flow direction is clear, and the problems of untimely and inaccurate alarming are effectively solved from the time when the environmental monitoring equipment monitors the environmental monitoring data to the time when relevant workers process the environmental monitoring data according to the alarming information.
Example two:
as shown in fig. 2, on the basis of the first embodiment, a processing method of the environmental monitoring data is further defined, and the processing method includes:
and 212, storing the alarm data to a database through the data processing assembly, and outputting alarm information.
In this embodiment, the national standard protocol data is transmitted to the streaming big data processing framework through the message middleware, specifically, the message middleware is kafka message middleware, that is, the national standard protocol data is continuously transmitted to the streaming big data processing framework through the kafka message middleware. The kafka message middleware can transmit data in large data volume and large batch, thereby ensuring that the data is accurate, rapid and is not lost.
Example three:
as shown in fig. 3, on the basis of any of the above embodiments, a processing method of the environmental monitoring data is further defined, the processing method including:
308, acquiring national standard protocol data through a data acquisition component;
and step 314, storing the alarm data to a database through the data processing assembly, and outputting alarm information.
In this embodiment, the source of the data packet of the environment monitoring data is the environment monitoring device, and the client establishes a connection with the environment monitoring device through three handshakes of a transmission control protocol (i.e., a TCP protocol), so as to obtain the data packet of the environment monitoring data. The environment monitoring equipment adopts a high-sensitivity sensor, and can quickly identify and collect monitoring conditions in the atmosphere. The data is transmitted through the transmission control protocol, so that the data can be ensured to be accurate, rapid and not lost.
Example four:
on the basis of the first embodiment, further limiting the parsing of the national standard protocol data by the data parsing component in step 106, specifically including: and converting the national standard protocol data into data in a data exchange format through a data analysis component.
In the embodiment, the national standard protocol data is converted into the data in the data exchange format through the data analysis component, and the alarm calculation is performed by adopting the data in the data exchange format, so that the calculation process is simple, and the error probability is reduced. Specifically, the data exchange format is a json format, which is a lightweight data exchange format, is relatively simple, easy to read and write, compressed, small in occupied bandwidth, easy to analyze, and capable of supporting multiple languages.
Further, the alarm calculation of the analyzed national standard protocol data by the data calculation component in step 108 to determine alarm data specifically includes: determining data exchange format data corresponding to the environmental monitoring factor through a data calculation component; and calculating the data exchange format data corresponding to the environment monitoring factors according to an alarm rule to determine alarm data.
In this embodiment, the environmental monitoring factors typically include: pollution of very fine particulate matter (PM2.5, PM10), ozone, sulfur dioxide, carbon monoxide, hydrogen sulfide, nitrogen oxides, volatile organic pollutants, total suspended particulate matter, lead, benzene, meteorological parameters, visibility, and the like. As will be readily understood by those skilled in the art, different environmental monitoring factors have different standard data, and thus different environmental monitoring factors correspond to different alarm rules. The data exchange format data corresponding to the environment monitoring factors are determined through the data calculation component, namely the environment monitoring factors are matched with the corresponding data exchange format data, the data exchange format data corresponding to each environment monitoring factor are calculated according to the corresponding alarm rule, so that the alarm data are determined, and the problem of inaccurate alarm is effectively solved.
Further, calculating data exchange format data corresponding to the environment monitoring factor according to an alarm rule to determine alarm data, specifically comprising: comparing the data exchange format data corresponding to the environment monitoring factor with the standard data; when the data value of the data exchange format data is greater than that of the standard data, determining the data exchange format data as alarm data; and when the data value of the data exchange format data is less than or equal to the data value of the standard data, determining the data exchange format data as normal data.
In the embodiment, the data exchange format data corresponding to each environment monitoring factor is compared with the standard data corresponding to each environment monitoring factor, and when the data value of the data exchange format data is greater than that of the standard data, the data exchange format data is determined to be alarm data; and when the data value of the data exchange format data is less than or equal to the data value of the standard data, determining the data exchange format data as normal data. That is to say, when the data value of the data in the data exchange format exceeds the standard, the data in the data exchange format is determined as alarm data, the alarm data is stored in the database, and the data in the data exchange format determined as normal data can also be stored in the database as a retention record for subsequent viewing and verification. Meanwhile, an alarm message is output according to the alarm data to prompt relevant workers to process in time.
Example five:
an embodiment five provides a processing apparatus of environmental monitoring data, including: the storage stores a computer program, and the processor executes the computer program to perform the steps of the method for processing environmental monitoring data according to any one of the embodiments.
The processing apparatus for environmental monitoring data provided in the embodiments of the present invention has all the beneficial technical effects of the processing method for environmental monitoring data in any of the embodiments, and details are not repeated herein.
Example six:
an embodiment provides an air micro-station, comprising: the processing device for environment monitoring data according to the fifth embodiment.
The processing system of environmental monitoring data provided in the embodiment of the present invention has all the beneficial technical effects of the processing device of environmental monitoring data in the fifth embodiment, that is, all the beneficial technical effects of the processing method of environmental monitoring data in any one of the embodiments, and details are not described herein again.
Further, the air micro-station also comprises: the environment monitoring equipment is connected with the processing device of the environment monitoring data through a transmission control protocol; and the alarm device is connected with the processing device of the environmental monitoring data and used for prompting alarm information.
The environment monitoring equipment collects environment monitoring data, and the processing device of the environment monitoring data is connected with the environment monitoring equipment through a transmission control protocol, so that a data packet of the environment monitoring data is obtained, the data packet is processed, and alarm information is output. The alarm device is used for prompting according to alarm information output by the processing device of the environment monitoring data, so that untimely alarm is avoided, related workers can rapidly process according to the prompted alarm information, and the environment is effectively protected.
Example seven:
embodiment seven provides a computer readable storage medium, on which a computer program is stored, the computer program, when executed by a processor, performs the steps of the method for processing environmental monitoring data according to any one of the above embodiments.
The computer-readable storage medium provided in the embodiments of the present invention has all the beneficial technical effects of the method for processing environmental monitoring data in any of the above embodiments, and details are not repeated herein.
The method for processing the environmental monitoring data provided by the embodiment of the invention mainly comprises the following specific implementation steps:
TCP Environment building and client connection TCP
And building a TCP environment based on the springboot, and performing three-way handshake with the environment monitoring equipment of the air micro station through a TCP client so as to establish contact. The environment monitoring equipment sends a data packet of environment monitoring data to the client, and then the client analyzes the transmitted binary or hexadecimal data packet and converts the binary or hexadecimal data packet into national standard protocol data.
Environmental monitoring data set up and transmission in kafka environment
Kafka advantages include: high throughput, low latency, kafka can handle hundreds of thousands of messages per second, with a minimum latency of only a few milliseconds; the scalability and the kafka cluster support the hot expansion; persistence, reliability, messages are persisted to local disks and support data backup to prevent data loss; fault tolerance, allowing a node in the cluster to fail (if the number of copies is n, allowing n-1 nodes to fail); high concurrency, and support thousands of clients to read and write simultaneously.
According to the method and the device, the environment monitoring equipment basically uploads the second-level data and the data volume is large, and the kafka message middleware is selected, so that the data can be rapidly transmitted to the next process node.
The kafka environment building comprises the following steps: starting a zookeeper, configuring kafka related attributes and configuring a cluster, wherein the kafka cluster refers to a cluster formed by a plurality of brookers and is managed by the zookeeper. The cluster formed by the three brokers is deployed and can be started respectively, and the starting condition of the cluster formed by the three brokers can be checked.
After the kafka message middleware is integrated, the TCP sends the national standard protocol data after the data packet analysis to a kafka queue, and further enables the data to flow to the storm cluster.
Storm environment installation and integration
storm advantages include: the storm is a distributed streaming processing framework, has high instantaneity and high speed, can be used for processing messages which are continuously streamed from a computing source, and writes the results into a certain storage after processing.
storm's workflow includes: environment installation and storm cluster startup. Specifically, the storm can be divided into two components, namely, a spout component and a bolt component, wherein a data source starts from the spout component, data is sent to the bolt in a tuple manner, a plurality of bolts can be connected in series, and one bolt can also be connected into a plurality of spots/bolts.
topology: name of a real-time application running in storm. And integrating the spout and the bolt into a topological graph. The combination relationship, concurrency number, configuration, etc. of spout and bolt are defined.
spout: components of a source data stream are obtained in a topology. Typically, the spout reads data from an external data source and then converts to source data within the topology.
bolt: a component that accepts data and then performs processing. In which the user can perform the operation he wants.
tuple: the basic unit of a message delivery, understood as a group of messages, is a tuple.
stream: tuple set represents the flow of data.
4. Calculation process
The storm integrates kafka, receives continuous data transmitted from kafka, distributes different bolts in the storm, respectively processes and analyzes transmitted national standard protocol data, compares standard data for calculation, stores confirmed alarm data in a database, and outputs alarm information.
In the description herein, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance unless explicitly stated or limited otherwise; the terms "connected," "mounted," "secured," and the like are to be construed broadly and include, for example, fixed connections, removable connections, or integral connections; may be directly connected or indirectly connected through an intermediate. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A processing method of environmental monitoring data is characterized by comprising the following steps:
acquiring a data packet of the environmental monitoring data, and converting the data packet into national standard protocol data;
acquiring the national standard protocol data through a data acquisition component;
analyzing the national standard protocol data through a data analysis component;
performing alarm calculation on the analyzed national standard protocol data through a data calculation component to determine alarm data;
storing the alarm data to a database through a data processing component, and outputting alarm information;
the data acquisition component, the data analysis component, the data calculation component and the data processing component are components of a streaming big data processing framework.
2. The method for processing environmental monitoring data according to claim 1, further comprising, before performing alarm calculation on the national standard protocol data through a streaming big data processing framework:
and transmitting the national standard protocol data to the streaming big data processing framework through message middleware.
3. The method for processing environmental monitoring data according to claim 1 or 2, further comprising, before the obtaining the data packet of the environmental monitoring data:
and establishing connection with the environment monitoring equipment through a transmission control protocol.
4. The method for processing environmental monitoring data according to claim 1, wherein the parsing the national standard protocol data by the data parsing component specifically includes:
and converting the national standard protocol data into data in a data exchange format through a data analysis component.
5. The method for processing environmental monitoring data according to claim 4, wherein the alarm calculation is performed on the analyzed national standard protocol data through a data calculation component to determine the alarm data, specifically comprising:
determining, by a data computation component, the data exchange format data corresponding to an environmental monitoring factor;
and calculating the data exchange format data corresponding to the environment monitoring factors according to an alarm rule to determine alarm data.
6. The method for processing environmental monitoring data according to claim 5, wherein the calculating the data exchange format data corresponding to the environmental monitoring factor according to an alarm rule to determine alarm data specifically includes:
comparing the data exchange format data corresponding to the environmental monitoring factor with standard data;
when the data value of the data exchange format data is greater than that of the standard data, determining the data exchange format data as alarm data;
and when the data value of the data exchange format data is less than or equal to the data value of the standard data, determining that the data exchange format data is normal data.
7. An apparatus for processing environmental monitoring data, comprising: a memory storing a computer program and a processor executing the computer program to perform the steps of the method of processing environmental monitoring data according to any one of claims 1 to 6.
8. An air micro-station, comprising: the environmental monitoring data processing apparatus of claim 7.
9. The air micro station of claim 8, further comprising:
the environment monitoring equipment is connected with the processing device of the environment monitoring data through a transmission control protocol;
and the alarm device is connected with the processing device of the environment monitoring data and used for prompting the alarm information.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of processing environmental monitoring data according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010942790.0A CN112134860A (en) | 2020-09-09 | 2020-09-09 | Processing method and device of environmental monitoring data, air micro-station and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010942790.0A CN112134860A (en) | 2020-09-09 | 2020-09-09 | Processing method and device of environmental monitoring data, air micro-station and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112134860A true CN112134860A (en) | 2020-12-25 |
Family
ID=73846468
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010942790.0A Withdrawn CN112134860A (en) | 2020-09-09 | 2020-09-09 | Processing method and device of environmental monitoring data, air micro-station and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112134860A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150022351A1 (en) * | 2013-07-16 | 2015-01-22 | Leeo, Inc. | Electronic device with environmental monitoring |
| CN106951464A (en) * | 2017-02-27 | 2017-07-14 | 江苏徐工信息技术股份有限公司 | Based on the personalized early warning mechanism big data computational methods of storm user orienteds |
-
2020
- 2020-09-09 CN CN202010942790.0A patent/CN112134860A/en not_active Withdrawn
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150022351A1 (en) * | 2013-07-16 | 2015-01-22 | Leeo, Inc. | Electronic device with environmental monitoring |
| CN106951464A (en) * | 2017-02-27 | 2017-07-14 | 江苏徐工信息技术股份有限公司 | Based on the personalized early warning mechanism big data computational methods of storm user orienteds |
Non-Patent Citations (2)
| Title |
|---|
| 杨素素: "基于Storm的城市消防联网远程监控系统的实时数据处理应用", 《计算机测量与控制》 * |
| 马鸣: "Storm处理框架在地球物理数据处理中的应用", 《 中国地球物理学会信息技术专业委员会地球物理信息前沿技术研讨会论文摘要集》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109144014B (en) | System and method for detecting operation condition of industrial equipment | |
| JP6085550B2 (en) | Log analysis apparatus and method | |
| CN106940677A (en) | One kind application daily record data alarm method and device | |
| CN109462490B (en) | Video monitoring system and fault analysis method | |
| CN110231998B (en) | Detection method and device for distributed timing task and storage medium | |
| CN108234161A (en) | For the access detection method and system of on-line off-line multitiered network framework | |
| CN114356499A (en) | Kubernetes cluster alarm root cause analysis method and device | |
| CN114090395A (en) | Sensor data processing method and device | |
| CN113112038B (en) | Intelligent monitoring and diagnostic analysis system, device, electronic equipment and storage medium | |
| CN115220995A (en) | Agent probe-based micro-service full-link analysis method | |
| CN112134860A (en) | Processing method and device of environmental monitoring data, air micro-station and storage medium | |
| CN116204386B (en) | Method, system, medium and equipment for automatically identifying and monitoring application service relationship | |
| CN110609761B (en) | Method and device for determining fault source, storage medium and electronic equipment | |
| CN113760634A (en) | Data processing method and device | |
| CN116259165A (en) | Monitoring data processing method and device, electronic equipment and storage medium | |
| CN114363435B (en) | Environment data monitoring and processing method | |
| CN117714262A (en) | Equipment fault detection method, device, equipment and storage medium | |
| CN114328093A (en) | Hadoop-based monitoring method, system, storage medium and equipment | |
| CN113347045A (en) | Alarm message processing method and device | |
| EP2533153B1 (en) | Unit for managing messages indicating event situations of monitored objects | |
| CN109586968A (en) | The log processing method of big data platform and the log processing system of big data platform | |
| CN115909532A (en) | Method and device for monitoring terminal polling, electronic equipment and medium | |
| US20230334340A1 (en) | Automated positive train control event data extraction and analysis engine for performing root cause analysis of unstructured data | |
| CN117692686A (en) | Live broadcast quality abnormity automatic diagnosis method and system | |
| CN115988198A (en) | Big data-based inspection method and system for front-end sensing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201225 |
|
| WW01 | Invention patent application withdrawn after publication |