CN116488985A - Processing method of multi-source heterogeneous Internet of things data - Google Patents
Processing method of multi-source heterogeneous Internet of things data Download PDFInfo
- Publication number
- CN116488985A CN116488985A CN202310485879.2A CN202310485879A CN116488985A CN 116488985 A CN116488985 A CN 116488985A CN 202310485879 A CN202310485879 A CN 202310485879A CN 116488985 A CN116488985 A CN 116488985A
- Authority
- CN
- China
- Prior art keywords
- equipment
- platform
- processing
- data
- message
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 36
- 238000004891 communication Methods 0.000 claims abstract description 23
- 238000012360 testing method Methods 0.000 claims abstract description 9
- 238000000034 method Methods 0.000 claims description 33
- 238000012544 monitoring process Methods 0.000 claims description 28
- 238000007726 management method Methods 0.000 claims description 25
- 230000007246 mechanism Effects 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 6
- 238000007405 data analysis Methods 0.000 claims description 6
- 238000013079 data visualisation Methods 0.000 claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/022—Multivendor or multi-standard integration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/0226—Mapping or translating multiple network management protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computer And Data Communications (AREA)
Abstract
The processing method of the multi-source heterogeneous Internet of things data comprises the following steps: defining an object model in a platform, wherein the object model describes equipment types, equipment source information and equipment communication protocols; defining product information, including names, object models and manufacturer information of products; defining and writing a data format of a device source, and converting the data format into a general data format script of a system; connecting test equipment, inputting IP and authentication of the equipment; the test equipment and the platform send messages to each other and test the connection condition; configuring a rule engine, defining a processing mode of a message sent by equipment, processing the message, and sending the message to a corresponding message queue; and analyzing the data sent by the equipment, drawing into a report, and generating a use report.
Description
Technical Field
The invention relates to a data processing technology, in particular to an internet of things data processing technology, and specifically relates to a processing method of multi-source heterogeneous internet of things data.
Background
With the rapid development of the internet of things technology, more and more devices are added to the internet of things. The internet of things has been integrated into aspects of life production, provides monitoring information, remotely controls various functions, and provides great convenience for people.
The protocols, access methods and platforms of all the Internet of things devices are different, so that the data acquisition and transmission of the devices become more complex, the devices are extremely troublesome in interactive use, and unified management and mutual cooperation are not facilitated.
The communication protocols and data formats used by different devices and systems are different, so that the devices and the systems cannot directly communicate with each other and are realized through a gateway or middleware; this not only increases the complexity and cost of the system, but also limits the development of the internet of things.
Therefore, it is necessary to provide a method for processing multi-source heterogeneous internet of things data to solve the above problems.
Disclosure of Invention
The invention aims to provide a processing method of multi-source heterogeneous Internet of things data, which supports various protocols and devices, and comprises HTTP and MQTT which can be conveniently accessed into different devices and systems; supporting a custom protocol and equipment types to adapt to the requirements of different users; the platform uses a unified data format and a communication protocol, so that different devices and systems can communicate directly without middleware or gateway; the complexity and the cost of the system are reduced, and the reliability and the performance of the system are also improved; the device supports real-time monitoring and remote control, helps a user to better manage the device and data, and improves working efficiency and production efficiency.
The technical proposal is as follows:
a processing method of multi-source heterogeneous Internet of things data comprises the following steps:
s1) defining an object model in a platform, wherein the object model describes equipment types, equipment source information and equipment communication protocols;
s2) defining product information, including names, object models and manufacturer information of products;
s3) defining and writing a data format of a device source, and converting the data format into a general data format script of a system;
s4) connecting test equipment, inputting IP and authentication of the equipment;
s5) the testing equipment and the platform send messages to each other, and the connection condition is tested;
s6) configuring a rule engine, defining a processing mode of a message sent by equipment, processing the message and sending the message to a corresponding message queue;
s7) analyzing the data sent by the equipment, drawing the data into a report, and generating a use report.
Further, the platform comprises:
and the equipment management module: the system is responsible for registering, maintaining and managing equipment, provides functions of real-time monitoring of equipment connection state and inquiring, modifying and deleting equipment information, supports various equipment types and communication protocols, and can help users to quickly access and manage a large number of equipment;
and the connection management module is used for: the method is responsible for connection management and control between equipment and a platform, provides a safe and stable connection mechanism, supports various connection modes and protocols such as MQTT and HTTP, provides a monitoring and control function of connection states, and helps a user to guarantee the communication quality of the equipment and the platform.
Further, the platform further comprises:
a user interface module: the interactive interface of the user is used for controlling and managing the equipment by the user; the basic functions of user registration, login and account management are supported, and meanwhile, the functions of equipment management and data visualization are provided, so that a user is helped to better master equipment state and data information;
protocol management module: is responsible for managing and resolving communication protocols between devices and platforms; the method supports the analysis and conversion of various communication protocols, such as JSON, XML and binary, and provides the support of custom protocols, thereby helping users to more flexibly cope with various communication scenes.
Further, the platform further comprises:
the rule engine module: the method is responsible for analyzing and processing equipment data and triggering corresponding operations according to preset rules; complex data analysis and processing are supported, and the user can be helped to realize automatic control and intelligent decision;
script management module: the script writing and management functions are provided, and a user can realize more complex functions and business logic by writing a custom script; support multiple scripting languages and operating environments, such as JavaScript, python, and provide script debugging and monitoring functions that help users develop and debug applications more efficiently.
Further, the platform further comprises:
message issuing module: responsible for delivering and distributing messages between the device and the platform; supporting various message formats and transmission modes, such as asynchronous messages and synchronous messages, providing message queues and a publish-subscribe mechanism, and helping users to realize efficient message transmission and processing;
and an analysis module: the method is responsible for analyzing and processing data generated by equipment, and extracting useful information and insight business trends; support multiple data analysis and processing algorithms and can help users to realize data visualization and report generation functions.
Further, the platform further comprises:
and the operation and maintenance monitoring module is used for: the system is responsible for monitoring and managing the running state and performance of the platform; providing various monitoring and alarming mechanisms, such as log monitoring and performance monitoring, and helping users to realize automatic operation and fault handling;
and a log module: the system is responsible for recording log information generated by a platform and equipment; the system supports various log formats and storage modes, provides log inquiry and statistics functions, and helps users track problems and troubleshoot faults.
Compared with the prior art, the invention supports a plurality of protocols and devices, including HTTP and MQTT, and can be conveniently accessed into different devices and systems; supporting a custom protocol and equipment types to adapt to the requirements of different users; the platform uses a unified data format and a communication protocol, so that different devices and systems can communicate directly without middleware or gateway; the complexity and the cost of the system are reduced, and the reliability and the performance of the system are also improved; the device supports real-time monitoring and remote control, helps a user to better manage the device and data, and improves working efficiency and production efficiency.
Drawings
Fig. 1 is a schematic view of the construction of the platform of the present invention.
Fig. 2 is a schematic flow chart of the present invention.
Detailed Description
Examples:
referring to fig. 1, the present embodiment shows a platform for processing multi-source heterogeneous internet of things data, where the platform includes:
device management module 2: the system is responsible for registering, maintaining and managing equipment, provides functions of real-time monitoring of equipment connection state and inquiring, modifying and deleting equipment information, supports various equipment types and communication protocols, and can help users to quickly access and manage a large number of equipment;
connection management module 3: the method is responsible for connection management and control between equipment and a platform, provides a safe and stable connection mechanism, supports various connection modes and protocols such as MQTT and HTTP, provides a monitoring and control function of connection states, and helps a user to guarantee the communication quality of the equipment and the platform.
User interface module 1: the interactive interface of the user is used for controlling and managing the equipment by the user; the basic functions of user registration, login and account management are supported, and meanwhile, the functions of equipment management and data visualization are provided, so that a user is helped to better master equipment state and data information;
protocol management module 4: is responsible for managing and resolving communication protocols between devices and platforms; the method supports the analysis and conversion of various communication protocols, such as JSON, XML and binary, and provides the support of custom protocols, thereby helping users to more flexibly cope with various communication scenes.
Rule engine module 5: the method is responsible for analyzing and processing equipment data and triggering corresponding operations according to preset rules; complex data analysis and processing are supported, and the user can be helped to realize automatic control and intelligent decision;
script management module 6: the script writing and management functions are provided, and a user can realize more complex functions and business logic by writing a custom script; support multiple scripting languages and operating environments, such as JavaScript, python, and provide script debugging and monitoring functions that help users develop and debug applications more efficiently.
Message publishing module 7: responsible for delivering and distributing messages between the device and the platform; supporting various message formats and transmission modes, such as asynchronous messages and synchronous messages, providing message queues and a publish-subscribe mechanism, and helping users to realize efficient message transmission and processing;
analysis module 8: the method is responsible for analyzing and processing data generated by equipment, and extracting useful information and insight business trends; support multiple data analysis and processing algorithms and can help users to realize data visualization and report generation functions.
Operation and maintenance monitoring module 9: the system is responsible for monitoring and managing the running state and performance of the platform; providing various monitoring and alarming mechanisms, such as log monitoring and performance monitoring, and helping users to realize automatic operation and fault handling;
log module 10: the system is responsible for recording log information generated by a platform and equipment; the system supports various log formats and storage modes, provides log inquiry and statistics functions, and helps users track problems and troubleshoot faults.
The processing method of the multi-source heterogeneous Internet of things data comprises the following steps:
s1) defining an object model in a platform, wherein the object model describes equipment types, equipment source information and equipment communication protocols;
s2) defining product information, including names, object models and manufacturer information of products;
s3) defining and writing a data format of a device source, and converting the data format into a general data format script of a system;
s4) connecting test equipment, inputting IP and authentication of the equipment;
s5) the testing equipment and the platform send messages to each other, and the connection condition is tested;
s6) configuring a rule engine, defining a processing mode of a message sent by equipment, processing the message and sending the message to a corresponding message queue;
s7) analyzing the data sent by the equipment, drawing the data into a report, and generating a use report.
Compared with the prior art, the embodiment supports various protocols and devices, including HTTP and MQTT, and can be conveniently accessed into different devices and systems; supporting a custom protocol and equipment types to adapt to the requirements of different users; the platform uses a unified data format and a communication protocol, so that different devices and systems can communicate directly without middleware or gateway; the complexity and the cost of the system are reduced, and the reliability and the performance of the system are also improved; the device supports real-time monitoring and remote control, helps a user to better manage the device and data, and improves working efficiency and production efficiency.
What has been described above is merely some embodiments of the present invention. It will be apparent to those skilled in the art that various modifications and improvements can be made without departing from the spirit of the invention.
Claims (10)
1. A processing method of multi-source heterogeneous Internet of things data is characterized by comprising the following steps: the method comprises the following steps:
s1) defining an object model in a platform, wherein the object model describes equipment types, equipment source information and equipment communication protocols;
s2) defining product information, including names, object models and manufacturer information of products;
s3) defining and writing a data format of a device source, and converting the data format into a general data format script of a system;
s4) connecting test equipment, inputting IP and authentication of the equipment;
s5) the testing equipment and the platform send messages to each other, and the connection condition is tested;
s6) configuring a rule engine, defining a processing mode of a message sent by equipment, processing the message and sending the message to a corresponding message queue;
s7) analyzing the data sent by the equipment, drawing the data into a report, and generating a use report.
2. The method for processing multi-source heterogeneous internet of things data according to claim 1, wherein the method comprises the following steps: the platform includes:
and the equipment management module: the system is responsible for registering, maintaining and managing equipment, provides functions of real-time monitoring of equipment connection state and inquiring, modifying and deleting equipment information, supports various equipment types and communication protocols, and can help users to quickly access and manage a large number of equipment.
3. The method for processing multi-source heterogeneous internet of things data according to claim 1, wherein the method comprises the following steps: the platform further comprises:
and the connection management module is used for: the method is responsible for connection management and control between equipment and a platform, provides a safe and stable connection mechanism, supports various connection modes and protocols such as MQTT and HTTP, provides a monitoring and control function of connection states, and helps a user to guarantee the communication quality of the equipment and the platform.
4. The method for processing multi-source heterogeneous internet of things data according to claim 1, wherein the method comprises the following steps: the platform further comprises:
a user interface module: the interactive interface of the user is used for controlling and managing the equipment by the user; the system supports the basic functions of user registration, login and account management, simultaneously provides the functions of equipment management and data visualization, and helps the user to better master the equipment state and data information.
5. The method for processing multi-source heterogeneous internet of things data according to claim 4, wherein the method comprises the following steps: the platform further comprises:
protocol management module: is responsible for managing and resolving communication protocols between devices and platforms; the method supports the analysis and conversion of various communication protocols, such as JSON, XML and binary, and provides the support of custom protocols, thereby helping users to more flexibly cope with various communication scenes.
6. The method for processing multi-source heterogeneous internet of things data according to claim 5, wherein the method comprises the following steps: the platform further comprises:
the rule engine module: the method is responsible for analyzing and processing equipment data and triggering corresponding operations according to preset rules; support complex data analysis and processing, and can help users to realize automated control and intelligent decision.
7. The method for processing multi-source heterogeneous internet of things data according to claim 6, wherein the method comprises the following steps: the platform further comprises:
script management module: the script writing and management functions are provided, and a user can realize more complex functions and business logic by writing a custom script; support multiple scripting languages and operating environments, such as JavaScript, python, and provide script debugging and monitoring functions that help users develop and debug applications more efficiently.
8. The method for processing multi-source heterogeneous internet of things data according to claim 1, wherein the method comprises the following steps: the platform further comprises:
message issuing module: responsible for delivering and distributing messages between the device and the platform; support multiple message formats and transmission modes, such as asynchronous messages and synchronous messages, and provide message queues and a publish-subscribe mechanism to help users achieve efficient message delivery and processing.
9. The method for processing multi-source heterogeneous internet of things data according to claim 1, wherein the method comprises the following steps: the platform further comprises:
and an analysis module: the method is responsible for analyzing and processing data generated by equipment, and extracting useful information and insight business trends; support multiple data analysis and processing algorithms and can help users to realize data visualization and report generation functions.
10. The method for processing multi-source heterogeneous internet of things data according to claim 1, wherein the method comprises the following steps: the platform further comprises:
and the operation and maintenance monitoring module is used for: the system is responsible for monitoring and managing the running state and performance of the platform; providing various monitoring and alarming mechanisms, such as log monitoring and performance monitoring, and helping users to realize automatic operation and fault handling;
and a log module: the system is responsible for recording log information generated by a platform and equipment; the system supports various log formats and storage modes, provides log inquiry and statistics functions, and helps users track problems and troubleshoot faults.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310485879.2A CN116488985A (en) | 2023-04-28 | 2023-04-28 | Processing method of multi-source heterogeneous Internet of things data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310485879.2A CN116488985A (en) | 2023-04-28 | 2023-04-28 | Processing method of multi-source heterogeneous Internet of things data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116488985A true CN116488985A (en) | 2023-07-25 |
Family
ID=87223033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310485879.2A Pending CN116488985A (en) | 2023-04-28 | 2023-04-28 | Processing method of multi-source heterogeneous Internet of things data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116488985A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117857608A (en) * | 2024-03-07 | 2024-04-09 | 安徽慕京信息技术有限公司 | Method and system for collecting equipment data based on Internet of things platform |
-
2023
- 2023-04-28 CN CN202310485879.2A patent/CN116488985A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117857608A (en) * | 2024-03-07 | 2024-04-09 | 安徽慕京信息技术有限公司 | Method and system for collecting equipment data based on Internet of things platform |
| CN117857608B (en) * | 2024-03-07 | 2024-05-03 | 安徽慕京信息技术有限公司 | Method and system for collecting equipment data based on Internet of things platform |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107438095B (en) | Session interface proxy for manufacturing operational information | |
| CN108847979B (en) | Self-adaptive configuration system and method based on SCADA | |
| RU2541911C2 (en) | Intelligent system kernel | |
| US8478714B2 (en) | Integrated distribution management system channel adapter | |
| CN103296755A (en) | Network online monitoring system for transformer substation | |
| CN1783068B (en) | Method for implementing fault diagnosis and monitoring database service | |
| CN110995859A (en) | Intelligent transformer substation supporting platform system based on ubiquitous Internet of things | |
| CN116488985A (en) | Processing method of multi-source heterogeneous Internet of things data | |
| CN111970363B (en) | Data processing method and device based on Internet of things | |
| CN109921986A (en) | A kind of multi-protocol data cut-in method, device, intelligent gateway and storage medium | |
| CN113467771B (en) | Model-based industrial edge cloud collaboration system and method | |
| CN114827140A (en) | Real-time data centralized management and control system for wind tunnel site | |
| CN110008267B (en) | Data processing system and method | |
| CN112486706B (en) | Internet of things local equipment linkage method based on MQTT message driving mechanism | |
| CN113630463B (en) | Intelligent classroom recording and broadcasting terminal equipment management system and method based on kubeedge | |
| CN114785830A (en) | Performance smart cloud platform | |
| CN115002146A (en) | Application method of object model in industrial Internet of things | |
| CN104270432B (en) | Based on drilling well industry Real-time Data Service system and data interactive method | |
| CN111711695B (en) | Distributed equipment management system based on cloud platform | |
| CN113848838A (en) | Data communication system and method for industrial Internet of things | |
| CN114567568B (en) | Electric power Internet of things data processing method and device based on edge calculation | |
| CN106330622B (en) | Data capturing method and device and communication gateway machine | |
| CN112769624B (en) | Point table-based industrial Internet of things gateway design method | |
| CN116743886A (en) | Industrial control equipment data acquisition system based on Internet of things | |
| CN104639893A (en) | Unified power transmission and transformation video monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |