CN114338746A - Analysis early warning method and system for data collection of Internet of things equipment - Google Patents
Analysis early warning method and system for data collection of Internet of things equipment Download PDFInfo
- Publication number
- CN114338746A CN114338746A CN202111661049.8A CN202111661049A CN114338746A CN 114338746 A CN114338746 A CN 114338746A CN 202111661049 A CN202111661049 A CN 202111661049A CN 114338746 A CN114338746 A CN 114338746A
- Authority
- CN
- China
- Prior art keywords
- data
- internet
- equipment
- things
- early warning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013480 data collection Methods 0.000 title claims abstract description 30
- 238000004458 analytical method Methods 0.000 title claims abstract description 26
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000005065 mining Methods 0.000 claims abstract description 10
- 238000004590 computer program Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 14
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 2
- 238000000513 principal component analysis Methods 0.000 claims description 2
- 230000009467 reduction Effects 0.000 claims description 2
- 238000012549 training Methods 0.000 claims description 2
- 238000011084 recovery Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 8
- 238000007726 management method Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000005477 standard model Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013523 data management Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003032 molecular docking Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000003339 best practice Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000030279 gene silencing Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Abstract
The invention provides an analysis early warning method and system for data collection of Internet of things equipment, which comprises the steps of obtaining real-time reported information of the Internet of things equipment and carrying out unified definition; classifying the real-time reported information based on the Internet of things equipment; carrying out data preprocessing on the classified data; performing association mining on the preprocessed data in corresponding different scenes to determine association information of the Internet of things equipment data; and performing real-time early warning according to the association information among the Internet of things equipment data and the set equipment reported information triggering condition. The invention supports various data sources to acquire data and flow direction, and has extremely high safety reliability and data recovery capability.
Description
Technical Field
The invention belongs to the technical field of Internet of things, and particularly relates to an analysis early warning method and system for data collection of Internet of things equipment.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Through the development of more than 20 years, the industry of the internet of things gradually matures, and with the fusion and application of new technologies such as cloud computing, AI, 5G and the like, the internet of things gradually transits to the AIoT direction: IoT and AI are combined, mass data are generated and collected through the Internet of things, are stored in an equipment terminal, an edge terminal or a cloud end, and are intelligently analyzed through machine learning so as to realize the intelligent association of everything.
At present, the data architecture of most companies is Lambda architecture, which solves the requirements of batch offline processing and real-time data processing of large data of one company. Data enters a big data platform from a data source at the bottom layer through various formats, is collected in the big data platform through data components such as Kafka and Flume, and is then divided into two lines for calculation. One line is entered into a Streaming computing platform (e.g., Storm, Flink, or Spark Streaming) to compute some metrics in real time; another line goes to an offline computation platform for batch data processing (e.g., Mapreduce, Hive, Spark SQL) to compute T +1 related business indicators that need to be seen every other day. The Lambda architecture has been stable over the years, but has some fatal disadvantages, and is not suitable for the requirement of data analysis business in the big data 3.0 era.
The disadvantages are as follows:
1. the problem of data aperture caused by inconsistency of real-time and batch calculation results;
2. batch calculation cannot be completed in a calculation window;
3. the data source change needs to be redeveloped, the development period is long, and the service response is not rapid enough;
4. the requirement on the storage of the server is high, and the storage pressure of the server is increased.
Disclosure of Invention
In order to solve the problems, the invention provides an analysis and early warning method and system for data collection of equipment of the internet of things.
According to some embodiments, a first aspect of the present invention provides an analysis and early warning method for data collection of an internet of things device, which adopts the following technical solutions:
an analysis early warning method for data collection of Internet of things equipment comprises the following steps:
acquiring real-time reported information of the Internet of things equipment, and performing unified definition;
classifying the real-time reported information based on the Internet of things equipment;
carrying out data preprocessing on the classified data;
performing association mining on the preprocessed data in corresponding different scenes to determine association information of the Internet of things equipment data;
and performing real-time early warning according to the association information among the Internet of things equipment data and the set equipment reported information triggering condition.
According to some embodiments, a second aspect of the present invention provides an analysis and early warning system for data collection of an internet of things device, which adopts the following technical solutions:
an analysis and early warning system for internet of things device data collection, comprising:
the data collection module is configured to acquire real-time reported information of the Internet of things equipment and perform unified definition;
the data classification module is configured to classify the real-time reported information based on the Internet of things equipment;
the data preprocessing module is configured to perform data preprocessing on the classified data;
the data association module is configured to perform association mining on the preprocessed data in corresponding different scenes, and determine association information of the Internet of things equipment data;
and the equipment early warning module is configured to perform real-time early warning according to the association information among the Internet of things equipment data and the set equipment reporting information triggering condition.
According to some embodiments, a third aspect of the invention provides a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in an analysis and early warning method for internet of things device data collection as described in the first aspect above.
According to some embodiments, a fourth aspect of the invention provides a computer apparatus.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the analysis and early warning method for data collection of devices of the internet of things as described in the first aspect.
Compared with the prior art, the invention has the beneficial effects that:
1. the intelligent gas and water integrated system adopts a micro-service architecture, has open capability and flexible design, can integrate multiple service scenes such as intelligent gas, intelligent water service, intelligent kitchens, intelligent communities and the like, can adapt to the constantly changing service scenes of various enterprises, and embraces the ecology of the industry.
2. The platform supports rapid mixed adaptation of multiple devices, multiple protocols, multiple networks and the like, eliminates single-point faults through load balancing and distributed cluster deployment technologies such as Docker and the like, ensures high availability of services, supports real-time access of hundred million-level devices, efficiently processes tens of millions of data in a concurrent mode, and has second-level communication capacity and millisecond-level response time.
3. According to the invention, the data are subjected to time-efficient layering treatment, and the maximum comprehensive treatment efficiency is obtained.
4. In order to overcome the defects, the automatic alarm of multiple devices in real time is realized through data collection analysis and association mining.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention.
Fig. 1 is a flowchart of an analysis and early warning method for data collection of an internet of things device according to a first embodiment of the present invention;
fig. 2 is a device access flow chart of an analysis and early warning system for collecting device data of the internet of things according to a first embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following figures and examples.
It is to be understood that the following detailed description is exemplary and is intended to provide further explanation of the invention as claimed. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
Example one
As shown in fig. 1-2, the present embodiment provides an analysis and early warning method for data collection of an internet of things device, and in the present embodiment, the method includes the following steps:
acquiring real-time reported information of the Internet of things equipment, and performing unified definition;
classifying the real-time reported information based on the Internet of things equipment;
carrying out data preprocessing on the classified data;
performing association mining on the preprocessed data in corresponding different scenes to determine association information of the Internet of things equipment data;
and performing real-time early warning according to the association information among the Internet of things equipment data and the set equipment reported information triggering condition.
Specifically, the acquiring of the real-time reported information of the internet of things device and performing unified definition specifically include:
analyzing the message of the real reported information of different Internet of things equipment to obtain unified messages with unified definitions;
the unified message mainly comprises deviceId, messageId, headers and timestamp;
deviceId is the only identification of the equipment, messageId is the only identification of the message, headers are the message header, and timestamp is the timestamp of the message; the headers are message headers and are used for performing actions of processing the self-defined message, such as whether the message is asynchronous or not, whether the message is fragmented or not and the like.
Wherein the device accesses the best practice;
the core of the device access is a protocol packet, and the protocol packet can be processed in a user-defined protocol packet theoretically no matter the device is directly connected or cloud docking is performed.
Object model definition
The project delivery of the Internet of things in the industry needs a standard model, and for equipment manufacturers, if the standard model exists, the equipment manufacturers do not need to select a data model of one manufacturer; for ISV application manufacturers, the development stage cannot exhaust all applications/devices to be pre-integrated; for an SI manufacturer, the subsystems are too many, and the docking integration takes time; for the customer, the lead time is too long, the standard model can be used between object models, between objects and applications, and between applications to improve efficiency, and the standard model can be compatible with various large manufacturers to define the standard model, various devices, various manufacturers and unified management.
When accessing a device, firstly, the attribute, function and event of the device object model are designed according to the device and the device access document (message description).
In the normal case: for the inherent invariable information of the equipment, the equipment label is recommended to be used for management; attributes are used to define some metric data, such as: voltage, temperature, etc.
Attributes should be simple data types such as: int, float, string, etc., avoiding the use of complex types such as structures, etc.
A function is used to define some executable actions that a device has. For example: and (4) silencing, turning off the lamp, controlling the holder, and designing input parameters and output parameters according to conditions.
Events are used to define actions that occur when a device is in a particular condition, such as: the fire alarm detects a human face, usually in a structural body type, and is used for storing more complex data.
Specifically, protocol packet development:
it is proposed to use a policy schema to define the function codes and the corresponding parsing rules for the different function codes. For example, enumeration is used to define function codes, array copy is avoided, and offset should be used to directly process the packet.
In particular, storage policy selection
The platform supports configuring data storage policies, different policies using different data storage approaches to store device data,
1. default-lined storage
The storage scheme used by the system by default uses the elastic search storage device data. Each attribute value is stored as an index record. Typical application scenarios: the device only reports a part of attributes each time and supports reading part of attribute data.
2. Default-columnar storage
The method uses the elastic search storage device data and one attribute as a column, uses one attribute message as an index record for storage, and is suitable for the scene that the device reports all attribute values each time.
Access device using MQTT service gateway
The following describes the access of third-party software to an internet of things platform by using an MQTT protocol by taking MQTTX as an example. MQTTX is an MQTT client tool written in Java language based on Eclipse Paho. Supporting subscription and publication of messages through Topic
1. Establishing a message protocol according to the message protocol definition;
2. creating a product according to the equipment configuration information and releasing the product;
3. defining an object model for the product device;
4. importing equipment, wherein after the equipment is imported, the equipment can be subjected to operations such as alarm setting, visualization and the like;
5. creating gateway configuration, creating MQTT service network component and MQTT service component configuration;
6. the device is connected with the platform and performs some basic operations of event transceiving and attribute reading.
In a specific embodiment, the classification is performed based on real-time reported information of the internet of things device, and specifically includes:
setting classification marks according to different manufacturers of different equipment;
storing the set classification identification into a database in a message queue form;
and adopting shunting and database-based processing according to the size of the data of each category.
And performing data preprocessing on the classified data, including missing value filling, data normalization, data dimension reduction and data principal component analysis.
Performing association mining on the preprocessed data in corresponding different scenes to determine association information of the internet of things equipment data, specifically:
training through an FP-tree frequency set algorithm based on the preprocessed data set to determine the associated information of the Internet of things equipment in the corresponding scene, specifically comprising the following steps:
the method comprises the steps of taking the Internet of things equipment association information as a transaction, and setting attribute characteristics as elements;
traversing the data set based on the correlation information of each Internet of things device, and counting the occurrence frequency of each element item;
traversing the data set again, and constructing an FP tree based on elements and the occurrence times of the elements;
for each element item in the FP tree, acquiring a corresponding conditional mode base;
based on the FP tree, according to the sequence from bottom to top and according to an FP-tree frequency set algorithm, finally obtaining a frequent pattern;
and determining the association information of each Internet of things device based on the frequent items.
Unified device connection management, multi-protocol adaptation (TCP, MQTT, UDP, CoAP, HTTP, etc.), the platform encapsulates the multi-protocol network communication interface, but the specific data is parsed by the message protocol.
The protocol support (protocolinport) mainly comprises an Authenticator (Authenticator), a message codec (DeviceMessageCodec), a message sending interceptor (devicemessagesendinterceptor) and configuration metadata (ConfigMetadata), shields the complexity of network programming, and flexibly accesses devices of different manufacturers and different protocols.
The early warning delay of the equipment is reduced, and the platform supports the setting of triggering conditions, triggers at fixed time and can also trigger in real time about early warning.
1. Triggering in real time: when the alarm threshold value of the data reported by the equipment is likely to be jittered, multiple alarms may be triggered, and at this time, the data can be processed by configuring anti-jitter rules.
Such as: alarm is given for multiple times within 1 minute, and only 1 time is processed. When alarm is configured in the equipment product, the anti-shake processing is carried out on the information reported by each equipment, the trigger condition is set,
2. timing triggering:
triggering is timed in a way of filling in cron expressions. Trigger conditions based on the object model, such as events, attributes, etc., may be selected.
Example two
The embodiment provides an analysis early warning system for internet of things device data collection, including:
the data collection module is configured to acquire real-time reported information of the Internet of things equipment and perform unified definition;
the data classification module is configured to classify the real-time reported information based on the Internet of things equipment;
the data preprocessing module is configured to perform data preprocessing on the classified data;
the data association module is configured to perform association mining on the preprocessed data in corresponding different scenes, and determine association information of the Internet of things equipment data;
and the equipment early warning module is configured to perform real-time early warning according to the association information among the Internet of things equipment data and the set equipment reporting information triggering condition.
The modules are the same as the corresponding steps in the implementation example and application scenarios, but are not limited to the disclosure of the first embodiment. It should be noted that the modules described above as part of a system may be implemented in a computer system such as a set of computer-executable instructions.
In the foregoing embodiments, the descriptions of the embodiments have different emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
A micro-service set of an analysis early warning system for collecting equipment data of the Internet of things is composed of 4 micro-service layers and two enhanced basic system services, and provides a series of services from physical domain data acquisition to information domain data processing and the like. The specific technical scheme of the four-layer architecture is as follows:
a driving layer: the SDK is used for providing standard or proprietary protocol connection physical equipment, is responsible for data acquisition and instruction control of southbound equipment, and can realize rapid development of drive based on the SDK
And (3) a data layer: the data management system is responsible for collecting and warehousing equipment data and providing data management interface service;
and (3) a management layer: the data management center is used for providing a micro-service registration center, an equipment instruction interface, equipment registration and association pairing and a data management center, is a core part of all micro-service interaction, is responsible for the management of various configuration data and provides interface service to the outside;
an application layer: the method is used for providing a rule engine, data opening, task scheduling, alarming and message notification, log management and the like, and has the capability of being connected with a third-party platform.
Platform unified device message definition:
the platform analyzes the message reported by the equipment into a platform unified message by using a self-defined protocol packet to carry out unified management.
The platform unified message is basically defined in the object model, and mainly comprises a property (property), a function (function) and an event (event).
The message composition is as follows:
the message mainly comprises deviceId, messageId, heads and timestamp. deviceId is the unique identifier of the device, messageI is the unique identifier of the message, headers are the message header, and are generally used for actions of processing custom messages, such as whether asynchronous messages exist or not, whether messages are fragmented or not, and the like.
Attribute-related messages
1. And acquiring a message ReadPropertyMessagereply corresponding to the equipment reply by the equipment attribute (ReadPropertyMessage).
2. The device property (WritePropertyMessage) is modified to correspond to the message WritePropertyMessageReply that the device replies.
3. The device report attribute (ReportPropertyMessage) is reported by the device.
Event message
The event message EventMessage is sent to the platform by the equipment terminal;
the device message corresponds to an event bus topic;
after the protocol packet analyzes the message reported by the equipment into equipment messages with unified platform, the messages are converted into corresponding topic and are sent to an event bus, the messages can be processed by subscribing the messages from the event bus, and prefixes of the topic of all the equipment messages need to be according to a specified rule;
for example: device/{ productId }/{ deviceId }. thus, all device specific messages can be subscribed to via wildcards, such as: device/' online, or subscribe to all messages: device/' star
The proposed system can be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the above-described modules is merely a logical division, and in actual implementation, there may be other divisions, for example, multiple modules may be combined or integrated into another system, or some features may be omitted, or not executed.
EXAMPLE III
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the analysis and early warning method for data collection of devices in the internet of things according to the first embodiment of the present invention.
Example four
The embodiment provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps in the analysis and early warning method for data collection of devices in the internet of things according to the first embodiment.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.
Claims (10)
1. An analysis early warning method for data collection of Internet of things equipment is characterized by comprising the following steps:
acquiring real-time reported information of the Internet of things equipment, and performing unified definition;
classifying the real-time reported information based on the Internet of things equipment;
carrying out data preprocessing on the classified data;
performing association mining on the preprocessed data in corresponding different scenes to determine association information of the Internet of things equipment data;
and performing real-time early warning according to the association information among the Internet of things equipment data and the set equipment reported information triggering condition.
2. The analysis and early warning method for data collection of the internet of things equipment according to claim 1, wherein the real-time reported information of the internet of things equipment is obtained and is defined uniformly, specifically:
analyzing the message of the real reported information of different Internet of things equipment to obtain unified messages with unified definitions;
the unified message mainly comprises deviceId, messageId, headers and timestamp;
deviceId is the only identification of the equipment, messageId is the only identification of the message, headers are the message header, and timestamp is the timestamp of the message;
the headers are message headers and are used for performing actions of processing the self-defined message, such as whether the message is asynchronous or not and whether the message is fragmented or not.
3. The analytic warning method for internet of things device data collection of claim 2, wherein the definition of the unified message mainly consists of attributes, functions and events.
4. The analysis and early warning method for the data collection of the equipment of the internet of things as claimed in claim 2, wherein when one piece of equipment is accessed, firstly, the attribute, the function and the event of the equipment object model are designed according to the equipment and the equipment access document;
wherein, for the inherent invariable information of the apparatus, use the apparatus label to manage; the attributes are used for defining some index data;
the function is used for defining some executable actions of the equipment, and designing input parameters and output parameters according to conditions;
events are used to define actions that occur when a device is in a particular condition.
5. The analysis and early warning method for the data collection of the equipment of the internet of things according to claim 1, wherein the classification is performed based on real-time reported information of the equipment of the internet of things, and specifically comprises the following steps:
setting classification marks according to different manufacturers of different equipment;
storing the set classification identification into a database in a message queue form;
and adopting shunting and database-based processing according to the size of the data of each category.
6. The analysis and early warning method for the data collection of the equipment of the internet of things as claimed in claim 1, wherein the data preprocessing for the classified data comprises missing value filling, data normalization, data dimension reduction and data principal component analysis.
7. The analysis and early warning method for the data collection of the equipment of the internet of things according to claim 1, wherein the pre-processed data is subjected to association mining under corresponding different scenes to determine the association information of the equipment data of the internet of things, and specifically comprises the following steps:
training through an FP-tree frequency set algorithm based on the preprocessed data set to determine the associated information of the Internet of things equipment in the corresponding scene, specifically comprising the following steps:
the method comprises the steps of taking the Internet of things equipment association information as a transaction, and setting attribute characteristics as elements;
traversing the data set based on the correlation information of each Internet of things device, and counting the occurrence frequency of each element item;
traversing the data set again, and constructing an FP tree based on elements and the occurrence times of the elements;
for each element item in the FP tree, acquiring a corresponding conditional mode base;
based on the FP tree, according to the sequence from bottom to top and according to an FP-tree frequency set algorithm, finally obtaining a frequent pattern;
and determining the association information of each Internet of things device based on the frequent items.
8. An analysis and early warning system for data collection of internet of things equipment, comprising:
the data collection module is configured to acquire real-time reported information of the Internet of things equipment and perform unified definition;
the data classification module is configured to classify the real-time reported information based on the Internet of things equipment;
the data preprocessing module is configured to perform data preprocessing on the classified data;
the data association module is configured to perform association mining on the preprocessed data in corresponding different scenes, and determine association information of the Internet of things equipment data;
and the equipment early warning module is configured to perform real-time early warning according to the association information among the Internet of things equipment data and the set equipment reporting information triggering condition.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for analytic pre-warning of internet of things device data collection according to any one of claims 1 to 7.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the method for analytic pre-warning of internet of things device data collection as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111661049.8A CN114338746A (en) | 2021-12-30 | 2021-12-30 | Analysis early warning method and system for data collection of Internet of things equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111661049.8A CN114338746A (en) | 2021-12-30 | 2021-12-30 | Analysis early warning method and system for data collection of Internet of things equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114338746A true CN114338746A (en) | 2022-04-12 |
Family
ID=81019559
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111661049.8A Pending CN114338746A (en) | 2021-12-30 | 2021-12-30 | Analysis early warning method and system for data collection of Internet of things equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114338746A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115242622A (en) * | 2022-07-25 | 2022-10-25 | 京东方科技集团股份有限公司 | Alarm processing system and method and Internet of things system |
| CN115277421A (en) * | 2022-07-21 | 2022-11-01 | 南京数睿数据科技有限公司 | Configuration information pushing method and device, electronic equipment and computer readable medium |
| CN116246445A (en) * | 2023-03-14 | 2023-06-09 | 国网湖北省电力有限公司孝感供电公司 | Knowledge-graph-based warehouse safety multi-source Internet-of-things data early warning method |
| CN116668493A (en) * | 2023-07-25 | 2023-08-29 | 森智明信息科技南京有限公司 | Message processing system of mqtt Internet of things equipment under distributed scene |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102098175A (en) * | 2011-01-26 | 2011-06-15 | 浪潮通信信息系统有限公司 | Alarm association rule obtaining method of mobile internet |
| CN104239437A (en) * | 2014-08-28 | 2014-12-24 | 国家电网公司 | Power-network-dispatching-oriented intelligent warning analysis method |
| US20180020021A1 (en) * | 2016-07-13 | 2018-01-18 | Hill Top Security, Inc. | Computerized system and method for providing cybersecurity detection and response functionality |
| US20180107695A1 (en) * | 2016-10-19 | 2018-04-19 | Futurewei Technologies, Inc. | Distributed fp-growth with node table for large-scale association rule mining |
| CN110855461A (en) * | 2018-08-20 | 2020-02-28 | 北京航天长峰科技工业集团有限公司 | Log analysis method based on association analysis and rule base |
| CN112528519A (en) * | 2020-05-14 | 2021-03-19 | 北京赛宝工业技术研究院有限公司 | Method, system, readable medium and electronic device for engine quality early warning service |
| CN112785108A (en) * | 2019-11-11 | 2021-05-11 | 国网天津市电力公司 | Power grid operation data correlation analysis method and system based on regulation cloud |
| CN113783931A (en) * | 2021-08-02 | 2021-12-10 | 中企云链（北京）金融信息服务有限公司 | Internet of things data aggregation and analysis method |
-
2021
- 2021-12-30 CN CN202111661049.8A patent/CN114338746A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102098175A (en) * | 2011-01-26 | 2011-06-15 | 浪潮通信信息系统有限公司 | Alarm association rule obtaining method of mobile internet |
| CN104239437A (en) * | 2014-08-28 | 2014-12-24 | 国家电网公司 | Power-network-dispatching-oriented intelligent warning analysis method |
| WO2016029570A1 (en) * | 2014-08-28 | 2016-03-03 | 北京科东电力控制系统有限责任公司 | Intelligent alert analysis method for power grid scheduling |
| US20180020021A1 (en) * | 2016-07-13 | 2018-01-18 | Hill Top Security, Inc. | Computerized system and method for providing cybersecurity detection and response functionality |
| US20180107695A1 (en) * | 2016-10-19 | 2018-04-19 | Futurewei Technologies, Inc. | Distributed fp-growth with node table for large-scale association rule mining |
| CN110855461A (en) * | 2018-08-20 | 2020-02-28 | 北京航天长峰科技工业集团有限公司 | Log analysis method based on association analysis and rule base |
| CN112785108A (en) * | 2019-11-11 | 2021-05-11 | 国网天津市电力公司 | Power grid operation data correlation analysis method and system based on regulation cloud |
| CN112528519A (en) * | 2020-05-14 | 2021-03-19 | 北京赛宝工业技术研究院有限公司 | Method, system, readable medium and electronic device for engine quality early warning service |
| CN113783931A (en) * | 2021-08-02 | 2021-12-10 | 中企云链（北京）金融信息服务有限公司 | Internet of things data aggregation and analysis method |
Non-Patent Citations (1)
| Title |
|---|
| 橙大仙: "物联网平台：一文读懂什么是物模型", Retrieved from the Internet <URL:https://zhuanlan.zhihu.com/p/338940995> * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115277421A (en) * | 2022-07-21 | 2022-11-01 | 南京数睿数据科技有限公司 | Configuration information pushing method and device, electronic equipment and computer readable medium |
| CN115277421B (en) * | 2022-07-21 | 2023-11-24 | 南京数睿数据科技有限公司 | Configuration information pushing method, device, electronic equipment and computer readable medium |
| CN115242622A (en) * | 2022-07-25 | 2022-10-25 | 京东方科技集团股份有限公司 | Alarm processing system and method and Internet of things system |
| CN116246445A (en) * | 2023-03-14 | 2023-06-09 | 国网湖北省电力有限公司孝感供电公司 | Knowledge-graph-based warehouse safety multi-source Internet-of-things data early warning method |
| CN116668493A (en) * | 2023-07-25 | 2023-08-29 | 森智明信息科技南京有限公司 | Message processing system of mqtt Internet of things equipment under distributed scene |
| CN116668493B (en) * | 2023-07-25 | 2023-10-24 | 森智明信息科技南京有限公司 | Message processing system of mqtt Internet of things equipment under distributed scene |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111526060B (en) | Method and system for processing service log | |
| US11386127B1 (en) | Low-latency streaming analytics | |
| US10855712B2 (en) | Detection of anomalies in a time series using values of a different time series | |
| CN114338746A (en) | Analysis early warning method and system for data collection of Internet of things equipment | |
| US11586972B2 (en) | Tool-specific alerting rules based on abnormal and normal patterns obtained from history logs | |
| US20180129579A1 (en) | Systems and Methods with a Realtime Log Analysis Framework | |
| CN105095048B (en) | A kind of monitoring system alarm association processing method based on business rule | |
| CN111712813A (en) | Intelligent preprocessing of multi-dimensional time series data | |
| CN109587125B (en) | Network security big data analysis method, system and related device | |
| EP3341845A1 (en) | Identifying and monitoring normal user and user group interactions | |
| US11042525B2 (en) | Extracting and labeling custom information from log messages | |
| CN106293891B (en) | Multidimensional investment index monitoring method | |
| CN111858251B (en) | Data security audit method and system based on big data computing technology | |
| CN113448812A (en) | Monitoring alarm method and device under micro-service scene | |
| CN116166505B (en) | Monitoring platform, method, storage medium and equipment for dual-state IT architecture in financial industry | |
| CN111258798A (en) | Fault positioning method and device for monitoring data, computer equipment and storage medium | |
| CN112600719A (en) | Alarm clustering method, device and storage medium | |
| Solmaz et al. | ALACA: A platform for dynamic alarm collection and alert notification in network management systems | |
| CN114090529A (en) | Log management method, device, system and storage medium | |
| US11734297B1 (en) | Monitoring platform job integration in computer analytics system | |
| CN116910144A (en) | Computing power network resource center, computing power service system and data processing method | |
| CN116743790A (en) | Device data acquisition, device data analysis method and device and computer device | |
| WO2023136847A1 (en) | Correlation engine and policy manager (cpe), method and computer program product | |
| WO2014184263A1 (en) | Integration platform monitoring | |
| CN116976984A (en) | Advertisement report acquisition method and device based on MetaAPI and related medium thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |