CN117614981A - Data processing system based on Internet of things - Google Patents
Data processing system based on Internet of things Download PDFInfo
- Publication number
- CN117614981A CN117614981A CN202311574009.9A CN202311574009A CN117614981A CN 117614981 A CN117614981 A CN 117614981A CN 202311574009 A CN202311574009 A CN 202311574009A CN 117614981 A CN117614981 A CN 117614981A
- Authority
- CN
- China
- Prior art keywords
- management device
- data
- target
- gateway
- communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 26
- 230000003993 interaction Effects 0.000 claims abstract description 4
- 238000004891 communication Methods 0.000 claims description 96
- 230000008859 change Effects 0.000 claims description 16
- 238000003860 storage Methods 0.000 claims description 15
- 238000012544 monitoring process Methods 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 5
- 230000000295 complement effect Effects 0.000 claims description 4
- 208000033748 Device issues Diseases 0.000 claims description 3
- 238000007726 management method Methods 0.000 description 199
- 230000006870 function Effects 0.000 description 50
- 238000012423 maintenance Methods 0.000 description 16
- 230000002159 abnormal effect Effects 0.000 description 10
- 238000000034 method Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 7
- 238000013500 data storage Methods 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 238000013480 data collection Methods 0.000 description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007405 data analysis Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000011084 recovery Methods 0.000 description 2
- 230000010076 replication Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/66—Arrangements for connecting between networks having differing types of switching systems, e.g. gateways
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0894—Policy-based network configuration management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/20—Traffic policing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/18—Multiprotocol handlers, e.g. single devices capable of handling multiple protocols
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/40—Network security protocols
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention relates to the field of Internet of things and discloses a data processing system based on the Internet of things, which comprises an acquisition management device, a platform management device and an I/O management device, wherein the platform management device is connected with each target device; the acquisition management device performs information interaction with each target device and performs batch acquisition on the data of each target device; the platform management device acquires the data acquired by the acquisition management device, and the platform management device accesses and connects the target equipment to the platform management device or releases the link of the target equipment and the platform management device according to the access online notification and the replacement offline notification of the target equipment; the I/O management device stores the acquired data acquired by the platform management device into the middleware, and pushes the data corresponding to the pushing scheme to the user through the API interface according to the pushing scheme.
Description
Technical Field
The invention relates to the technical field of the Internet of things, in particular to a data processing system based on the Internet of things.
Background
With the continuous development of technologies such as sensors, wireless communication, cloud computing, big data, artificial intelligence and the like, the development of the technology of the internet of things brings new opportunities and challenges to various industries and fields. The data acquisition system of the equipment is generally composed of data acquisition, data storage, data processing, data display and other components so as to realize functions of real-time monitoring, data analysis, decision support and the like of the equipment of the Internet of things.
In the prior art, software or a platform is often adopted to collect and process the data of the devices of the internet of things, however, the intelligent park is often large in number of devices and variety, and the existing software platform cannot be integrated with various devices for management.
Disclosure of Invention
In view of the above, the present invention provides a data processing system based on the internet of things to solve the problem of integrated management by combining multiple devices.
In a first aspect, the present invention provides a data processing system based on the internet of things, the system comprising an acquisition management device, a platform management device and an I/O management device, wherein,
the platform management device is connected with each target device;
the acquisition management device performs information interaction with each target device through a communication protocol adapted to each target device, and performs batch acquisition on data of each target device according to a preset acquisition scheme corresponding to each target device;
the platform management device acquires the data acquired by the acquisition management device, when the platform management device receives an access online notification of the target equipment, the target equipment is connected to the platform management device in an access way, and when the platform management device receives a replacement offline notification of the target equipment, the platform management device releases the link of the target equipment and the platform management device;
the I/O management device stores the acquired data acquired by the platform management device into the middleware, the I/O management device calls the data from the middleware according to service requirements, and the I/O management device pushes the data corresponding to the pushing scheme to a user through an API interface according to the pushing scheme.
The invention connects the collection management device with the target equipment through the communication protocol to realize unified management of various equipment, collects the data of the target equipment through the collection management device to realize data collection, manages the access and the unlink of the target equipment through the platform management device to realize the management of the target equipment, and calls and pushes the collection data acquired by the platform management device through the I/O management device to realize data processing and unified management of the equipment so as to realize integrated management of various equipment.
In an alternative embodiment, the system further comprises rights management means, wherein,
the right management device is used for acquiring the identity authentication information of the user in the data request when the data request sent by the user is received;
and the authority management device performs identity authentication on the identity verification information through an API interface, returns a temporary access identifier if the identity authentication passes, and returns error information if the identity authentication does not pass.
The invention verifies the identity of the user in the data request through the authority management device so as to prevent the data from being illegally acquired, thereby improving the safety of the data.
In an alternative embodiment, the system further comprises a system gateway and a gateway management device, wherein,
the gateway management device is used for providing a system gateway list and configuring a gateway file;
the gateway management device is also used for monitoring the online state of the system gateway, and is used for carrying out communication setting on the online system gateway when the system gateway is online and inquiring the historical communication setting of the system gateway when the system gateway is not online.
The invention monitors the online state of the system gateway through the gateway management device so as to carry out corresponding communication setting according to the online state of the system gateway, thereby facilitating the user to know the online state of the gateway in real time.
In an alternative embodiment, the system further comprises a device management apparatus, wherein,
the device management device is used for judging whether the target device has a management function, if the target device has the management function, the device management device determines the target device as a management device, and if the target device does not have the management function, the device management device determines the target device as a common device;
when receiving the equipment access request of the user, the equipment management device issues files to the system gateway according to the control authority of the user.
The device is classified by the device management device so as to manage different types of devices, and files are issued according to the control authority of the user, so that unified management of the system gateway and the devices is realized.
In an alternative embodiment, the system further comprises a status management device, wherein,
the state management device is used for monitoring the online state of the target equipment in real time;
the state management device is also used for recording the historical online state of the target equipment and the online state change of the target equipment.
The invention monitors the online state of the target equipment so as to monitor and analyze the online state of the equipment, and records the historical online state and state change so as to inquire the online state.
In an alternative embodiment, the system further comprises protocol management means, wherein,
the protocol management device manages a plurality of communication protocols in the protocol data item through a communication command library, and the plurality of communication protocols are used for realizing communication between the target equipment and the acquisition management device.
The invention manages the communication protocol through the protocol management device so as to ensure the normal communication between the target equipment and the acquisition management device.
In an alternative embodiment, the system further comprises a clock management device, wherein,
the clock management device carries out time setting on a clock of the target equipment according to a preset time setting scheme, and generates a time setting communication record which comprises a time setting success notification and a time setting failure notification;
the clock management device generates a time-to-time failure record according to the time-to-time failure notification.
According to the invention, the clock management device is used for carrying out time synchronization on the clock of the target equipment so as to reduce equipment time difference, and time synchronization communication and time synchronization failure are recorded so as to conveniently inquire time synchronization records.
In an alternative embodiment, the system further comprises a current limit management device, wherein,
the current limiting management device carries out current limiting filtration on the communication request of the user according to a preset current limiting scheme;
the flow limit management device generates a communication request queue according to the communication request and distributes the communication request to a plurality of servers.
The invention carries out current limiting and filtering on the communication request through the current limiting management device so as to ensure the stability and reliability of the system, avoid overload of the system, prevent the system from crashing and running slowly and improve the performance and usability of the system.
In an optional implementation manner, the system further comprises a log management device, the log management device collects and stores the system log according to a preset storage rule, the system log comprises a failure log, and the log management device generates the failure log according to storage failure data.
The invention collects and stores the system logs, so that a user can check and analyze the system logs conveniently, and the management of the system logs is realized.
In an alternative embodiment, the system further comprises policy management means, wherein,
the strategy management device manages and adjusts the system strategy according to the system change;
the policy management device is used for performing any one or more of complement copy scheme configuration, acquisition scheme configuration, push scheme configuration, time setting scheme configuration, current limiting scheme configuration or log scheme configuration.
The invention manages and adjusts the system strategy to effectively execute the corresponding functions of the system and adapt to the system change and the requirement.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a portion of a block diagram of a data processing system in accordance with an embodiment of the invention;
FIG. 2 is a schematic diagram of an overall block diagram of an Internet of things-based data processing system according to an embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the related art, software or a platform is generally adopted to collect and process the data of the internet of things equipment, privacy and confidential information of a user can be involved in the process of collecting the data, and strict security measures are required to be adopted to avoid data leakage and attack.
Because the quality of the data is unstable, noise and abnormal values may exist, and data cleaning and checking technology is required to improve the quality of the data. The data format and protocol are not uniform, and uniform standards and protocols are required to be formulated, so that data exchange and sharing are facilitated. To manage a large amount of device data, a perfect data management system needs to be established, including data backup, recovery, migration and the like.
The embodiment of the invention provides a data processing system based on the Internet of things, which comprises an acquisition management device, a platform management device and an I/O management device as shown in figure 1.
The platform management device is connected with each target device, the acquisition management device performs information interaction with each target device through a communication protocol matched with each target device, and batch acquisition is performed on data of each target device according to a preset acquisition scheme corresponding to each target device.
In the embodiment of the invention, the target equipment comprises various intelligent terminals including equipment such as an air conditioner, various sensors, an intelligent ammeter, a remote water meter, a flowmeter, an intelligent switch, a sensor, a circuit breaker, an intelligent street lamp, a Bluetooth gateway, an intelligent gateway and the like, various equipment protocols are accessed, protocol expansion is supported, data is provided for a service system, remote communication of the equipment is realized, and a unified Internet of things cloud platform is established.
The acquisition management device is used for acquiring real-time data, historical data and abnormal alarm information of the target equipment, and the main functions comprise real-time data acquisition, historical data acquisition, parameter issuing, batch acquisition, timing plan acquisition configuration and the like. The system implements multiple connection points and uses a time-series database for device data storage.
Specifically, a developer logs in the target equipment to the system through a communication protocol, and can realize data timing acquisition, support data pushing and push the data to a service system through configuration of equipment related information, acquisition strategies, data pushing and the like, and the service system does not need to manage access and equipment protocols of equipment.
The acquisition management device actively acquires the target equipment data at regular time according to a preset acquisition scheme, a pre-stored acquisition scheme can be issued through gateway parameters, the system gateway can actively report the target equipment data, the acquisition management device can actively analyze missing data, and the missing data is actively acquired through a missing data acquisition scheme, so that the integrity of the data is ensured.
The functions of the acquisition management device mainly comprise:
(1) Counting item collection rate, today's equipment accumulated data and target equipment average data (the item consists of a plurality of target equipment);
(2) Checking the historical collection rate of the project, the accumulated data of the historical equipment and the average data of the historical equipment through time screening;
(3) Analyzing and counting a change chart of target equipment, and checking various common data;
(4) The batch collection of the data of a plurality of target devices is supported, the real-time data collection can be executed, and the collection of the missing reading devices is automatically screened;
(5) Supporting parameter issuing of a system gateway, configuring a system gateway scheme and parameters, and acquiring data through active reporting of target equipment;
(6) And the batch control, parameter issuing and valve control of a plurality of target devices are supported.
The platform management device is connected with each target device through an API interface, acquires data acquired by the acquisition management device, acquires temporary access rights through authentication, and connects the target device to the platform management device in an access manner when the platform management device receives an access online notification of the target device, and releases the link of the target device and the platform management device when the platform management device receives a replacement offline notification of the target device.
In the embodiment of the invention, the platform management device realizes the docking of platform data and the management of binding equipment, supports a OC, AEP, oneNet platform, and has the main functions of: platform project information maintenance, platform equipment information maintenance, batch registration, batch unbinding, report importing, platform data synchronization, platform subscription management, instruction communication flow, historical data query and the like.
The functions of the platform management device mainly comprise:
(1) The operation entrance accessed by the platform is unified, and after the platform can be selected, the platform API interface is docked;
(2) Providing information and binding states of the view items and the target devices;
(3) The project provides key management, and temporary access rights are obtained through authentication;
(4) An excel importing function and a template downloading function are provided, so that information batch entry is facilitated;
(5) The target equipment is accessed to be on line, replaced to be off line, and the target equipment is accessed to or disconnected from the platform through the batch platform registration and unbinding function;
(6) If the data is manually injected or unbindd in the OC, AEP, oneNet platform, the data is required to be synchronized in the platform, so that file consistency is achieved;
(7) After the equipment is accessed to the OC, AEP, oneNet platform, a subscription push address is set, the subscription address is pre-stored, so that quick writing is convenient to select, and the platform pushes the active report data after the subscription push address is provided;
(8) The platform has active instruction issuing, supports the in OC, AEP, oneNet instant issuing and the buffer issuing;
(9) After the instruction is issued, storing an issuing process, managing the issuing progress, and recording the starting time, the ending time and detailed data;
(10) If the detection device acquisition is missing, historical data is automatically acquired from the OC, AEP, oneNet platform.
The I/O management device stores the acquired data acquired by the platform management device into the middleware, the I/O management device calls the data from the middleware according to service requirements, and the I/O management device pushes the data corresponding to the pushing scheme to a user through an API interface according to the pushing scheme.
In the embodiment of the invention, the I/O management device realizes the functions of data storage and pushing, equipment collects data and stores the data into the middleware, and the equipment calls an internal service system and externally uses Http pushing service, and the main functions comprise: storage failure maintenance, push failure maintenance, platform data maintenance, historical data maintenance and the like.
The platform supports data quality verification, analyzes and stores target equipment data, supports a data pushing function, meets corresponding pushing formats according to requirements of different service systems, supports failed maintenance and execution again, and ensures data integrity.
The functions of the I/O management apparatus mainly include:
(1) Storing target equipment data into a designated middleware, and acquiring the equipment data by a service system through the middleware;
(2) Storing the failed data, recording a failure log, and storing again through system maintenance;
(3) According to the pushing scheme, an API (application program interface) is used for pushing real-time data or notification to a client, the client does not need to poll a server, and bandwidth and server resources are saved;
(4) And pushing the failed data, recording a failure log, and carrying out pushing again through system maintenance.
According to the data processing system based on the Internet of things, the collection management device is connected with the target equipment through the communication protocol so as to realize unified management of various equipment, the collection management device is used for collecting data of the target equipment so as to realize data collection, the platform management device is used for managing access and disconnection of the target equipment so as to realize management of the target equipment, the I/O management device is used for calling and pushing the data so as to realize data processing and unified management of the equipment, and therefore integrated management of various equipment is realized.
As shown in fig. 2, the system includes a data analysis layer, a service logic layer, a transport layer, and a perception layer.
The data analysis layer realizes the functions of message analysis, data comparison, data statistics, data calculation and the like, and stores communication instructions and can execute timing tasks.
The business logic layer comprises an application layer, a service layer and a data layer, wherein a gateway management device, an acquisition management device, an I/O management device, a protocol management device, a current limiting management device, a clock management device, a log management device and a state management device are integrated in the application layer, the application layer realizes gateway management, acquisition management, I/O management, protocol management, current limiting management, clock management, log management and state management, the service layer realizes communication service, interface service, functional service and data service, and the data layer realizes data replication.
The transmission layer supports TCP/IP, 5G, 4G and NB-Iot, and the sensing layer comprises a terminal, an electric energy meter, a water meter, an intelligent switch, a sensor, an air conditioner, a flowmeter and an intelligent gateway.
In some optional embodiments, the system further comprises a rights management device, wherein the rights management device is used for acquiring authentication information of the user in the data request when the data request sent by the user is received; the authority management device performs identity authentication on the identity verification information through the API interface, if the identity authentication passes, the authority management device returns a temporary access identifier, and if the identity authentication does not pass, the authority management device returns error information.
In the embodiment of the invention, the rights management device realizes the safety management and control of the equipment control and the collected data, is used for verifying the identity of the user or the identity between systems, the user automatically generates the project ID and the secret key by creating the project, and the user needs to store the related information and authenticate the user when accessing the platform data.
The main functions of the rights management unit include:
(1) The user or the system sends a request for acquiring data to the server;
(2) After receiving the request, the server requests the user or the system to provide the identity verification information;
(3) Providing identity verification information, item id, key and other information by a user or a system, generating a signature token by a non-reversible algorithm md5, and carrying out identity authentication by an API identity authentication interface;
(4) The server verifies whether the identity information is consistent with the server or not to judge whether the identity information is correct or not;
(5) If the authentication is passed, the server returns a temporary access token;
(6) If the identity verification fails, the server returns error information, error code 01, inconsistent identity verification, error code 02 and no item;
(7) the token has a user-defined expiration time attribute, and is aged for 20 minutes, and then identity authentication is required to be re-performed, so that the risks of attack and imitation can be reduced from the time dimension;
(8) The user can access the acquired data only after acquiring the token.
According to the data processing system based on the Internet of things, the authority management device is used for verifying the identity of the user, so that data can be prevented from being illegally acquired, and the safety of the data is improved.
In some optional embodiments, the system further comprises a system gateway and a gateway management device, wherein the gateway management device is used for providing a system gateway list and configuring a gateway archive; the gateway management device is also used for monitoring the online state of the system gateway, and when the system gateway is online, the gateway management device carries out communication setting on the online system gateway, and when the system gateway is not online, the gateway management device inquires the historical communication setting of the system gateway.
In the embodiment of the invention, the gateway management device provides communication and management for the gateway and files, and the main functions comprise: gateway information maintenance, gateway archive maintenance, gateway online inquiry, offline rejection, archive communication synchronization, archive communication setting and the like.
The system gateway comprises terminals of various types and protocols, and a user configures gateway parameters, issues files and meter connection tests through the functions.
The main functions of the gateway management device include:
(1) The user can observe the real-time online state of the gateway;
(2) If online, the gateway can be set in communication;
(3) If offline, the historical settings can be queried;
(4) The gateway has the functions of logging in, heartbeat and off-line;
(5) The long-connection equipment has no heartbeat for a long time, and the broken line rejection of the client is executed;
(6) Providing a gateway list, and selecting a corresponding gateway to operate;
(7) Different parameter settings are provided according to different protocols, and parameters are issued to the gateway;
(8) Providing a meter file import template, and rapidly configuring a gateway file through importing;
(9) The meter file and the gateway actual file can be read to synchronously acquire the gateway internal actual file;
(10) After the file configuration is completed, the real-time data of the meter can be read through transparent transmission and used for detecting whether the connection between the gateway and the meter is normal or not.
According to the data processing system based on the Internet of things, the online state of the system gateway is monitored through the gateway management device, so that corresponding communication setting is carried out according to the online state of the system gateway, and a user can know the online state of the gateway in real time conveniently.
In some optional embodiments, the system further includes a device management apparatus, where the device management apparatus is configured to determine whether the target device meets the device profile feature, and if the target device meets the device profile feature, the device management apparatus determines the target device as a management device, and if the target device does not meet the device profile feature, the device management apparatus determines the target device as a normal device.
When receiving the equipment access request of the user, the equipment management device issues files to the system gateway according to the control authority of the user.
In the embodiment of the invention, the device management device divides the target device into the management device and the common device by judging whether the target device accords with the management function, if the target device has the management function, the device management device determines the target device as the management device, and if the target device does not have the management function, the device management device determines the target device as the common device, so that file management, parameter configuration, data collection scheme configuration, event collection scheme configuration, reporting scheme configuration and the like of the management device are realized, and parameter configuration, valve control, frozen data collection, abnormal event collection, real-time collection and the like of the common device are realized.
The system realizes 1 ten thousand connection points, uses a time sequence database aiming at equipment data storage, can realize 300 ten thousand equipment access quantity if 1 management equipment is accessed into 300 common equipment, and realizes load balancing among the equipment through a plurality of internal management, so that the equipment is balanced into a front-end processor. The time sequence database is used, so that the storage of mass data can be ensured, and the accuracy, the instantaneity and the traceability of the data are ensured.
The device management device provides unified management for the gateway and the devices under the platform, and the main functions comprise: project maintenance, device information maintenance, device key management, etc.
The platform can classify the devices to form a project, a plurality of devices form a project, an administrator can check all the projects, maintain the projects, and maintain the devices under the projects, wherein the maintenance comprises information such as device table addresses, imei, production dates, gateway addresses and the like.
The main functions of the device management apparatus include:
(1) An administrator can maintain information of the project and check the project information;
(2) Automatically generating a project key after the project is created, informing a user of data, acquiring a temporary certificate when an API accesses, and supporting a key forgetting reset function;
(3) The quick input of the equipment is supported, the input can be acquired through a mes system in the pair, and the input can be realized through a template in the pair;
(4) When the unique identifier of the equipment is the same as that in the gateway or the platform file, the unique identifier is used as a mapping relation;
(5) The user uses the secret key to access and acquire the equipment information through the API, the user acquires the control authority of the equipment, the gateway or the equipment can be communicated, and the gateway can be issued with files.
According to the data processing system based on the Internet of things, the equipment is classified through the equipment management device so as to manage different types of equipment, and files are issued according to the control authority of the user, so that unified management of the system gateway and the equipment is realized.
In some optional embodiments, the system further comprises a state management device for monitoring the online state of the target device in real time; the state management device is also used for recording the historical online state of the target device and the online state change of the target device.
In the embodiment of the invention, the state management device realizes real-time state monitoring and historical state statistics of the target equipment, and the main functions comprise: real-time online checking, historical online statistics, today acquisition conditions, historical acquisition rate statistics, historical anomaly statistics and the like.
The platform monitors the online state of the target equipment, records the offline and online conditions, captures equipment abnormality alarms, and analyzes the change of various conditions of the target equipment according to daily target equipment data.
The main functions of the state management device include:
(1) Counting the total number, online number, offline number, abnormal constant and online rate of the project equipment;
(2) Checking the total number, online number, offline number, abnormal constant and online rate of the project history equipment through time screening;
(3) Analyzing and counting daily change conditions of equipment, online change, offline change, abnormal change and changed equipment details;
(4) Checking the real-time state of a single device, inquiring the historical online and offline conditions and the historical online rate;
(5) And checking the abnormal state of the single device, and inquiring historical abnormal occurrence records, abnormal recovery records and device data when the abnormal state occurs.
According to the data processing system based on the Internet of things, the online state of the target equipment is monitored, so that the online state of the equipment is monitored and analyzed conveniently, and the historical online state and state change are recorded, so that the online state is queried conveniently.
In some alternative embodiments, the system further comprises a protocol management device, wherein the protocol management device manages a plurality of communication protocols in the protocol data item through the communication command library, and the plurality of communication protocols are used for realizing communication between the target device and the acquisition management device.
In the embodiment of the invention, the protocol management device adopts a communication command library to manage protocol data items, ensures protocol expansibility, supports protocols such as DL/T698.45, DL/T645.07, Q/GDW376.1, CJ/T188, NB-Iot, MQTT, self-grinding water meter and the like, and has the main functions of: control instruction maintenance, instruction detail maintenance, instruction replication, instruction conditional query and the like.
The main functions of the protocol management device include:
(1) Supporting the browsing and consulting of a protocol library, and inquiring according to conditions;
(2) Collecting a communication command library management protocol data item, and ensuring protocol expansibility;
(3) The functions of editing commands, copying instructions and the like are supported;
(4) Support power-related protocols such as DL/T698.45, DL/T645.07, Q/GDW376.1, etc.;
(5) Supporting CJ/T188, NB-Iot, self-grinding water meter and other water service related protocols;
(6) Subscription and release of the MQTT protocol are supported;
(7) Supporting communication protocols of various sensing devices;
(8) Protocol message analysis is supported for research personnel to analyze data.
According to the data processing system based on the Internet of things, the protocol management device is used for managing the communication protocol so as to ensure normal communication between the target equipment and the acquisition management device.
In some alternative embodiments, the system further comprises a clock management device that clocks the target device according to a preset clock schedule, generates a clock-to-clock communication record including a clock-to-clock success notification and a clock-to-clock failure notification, and generates the clock-to-clock failure record according to the clock-to-clock failure notification.
In the embodiment of the invention, the clock management device realizes the device clock time synchronization, and the device is timed by a time synchronization scheme to generate a time synchronization record and a time synchronization failure alarm record, and the main functions comprise: time-setting record inquiry, history time-setting comparison, failure record inquiry and the like.
The main functions of the clock management device include:
(1) The platform performs timing service on the equipment at regular time through a timing scheme;
(2) The time setting process generates time setting communication records, and the records succeed and fail;
(3) Comparing the platform execution timing detection equipment time with the platform time, and recording the time difference;
(4) The platform can inquire historical equipment time comparison data and communication time comparison data;
(5) And the platform records the time out-of-tolerance equipment condition.
According to the data processing system based on the Internet of things, the clock of the target device is clocked through the clock management device, so that the device time difference is reduced, clock clocked communication and clock clocked failure are recorded, and clock clocked recording is conveniently queried.
In some optional embodiments, the system further comprises a current limiting management device, and the current limiting management device performs current limiting filtering on the communication request of the user according to a preset current limiting scheme; the flow limit management device generates a communication request queue according to the communication request and distributes the communication request to a plurality of servers.
In the embodiment of the invention, the current limiting management device realizes the limitation of the client to the service call, allows the execution times in a certain time according to the current limiting scheme, forms a cache pool for the communication request, and orderly executes the communication requirement, wherein the main functions comprise: communication tasks, communication stops, communication data queries, real-time communications, cache communications, etc.
In the network communication or system operation process, the flow or request is limited, the orderly, timed and planned communication tasks of the equipment are ensured, so as to ensure the stability and reliability of the system, and the main purpose is to avoid overload of the system, prevent system breakdown or slow operation, and improve the performance and availability of the system.
The main functions of the current limiting management device include:
(1) Performing a current limiting filtering according to a current limiting scheme;
(2) Limiting the number of requests or the amount of data per unit time;
(3) Limiting the number of requests processed simultaneously;
(4) Placing the request into a queue, merging the communication commands, and limiting the length or waiting time of the queue;
(5) When the system load is too high, the service quality is reduced or partial functions are provided so as to ensure the usability of the system;
(6) Requests are distributed across multiple servers to balance system loads and improve system performance and availability.
In some optional embodiments, the system further comprises a log management device, the log management device collects and stores the system log according to a preset storage rule, the system log comprises a failure log, and the log management device generates the failure log according to storage failure data.
In the embodiment of the invention, the log management device realizes the management of the system log, the log is stored according to the storage rule, the storage days are automatically deleted, and the main functions comprise: log storage, log deletion, log downloading, log querying, log ranking, log browsing, and the like.
The log generated by the system is collected, stored, analyzed and displayed, so that a user is helped to quickly find out the system problem, locate the cause of the problem, and improve the reliability and stability of the system, and the main functions comprise: log storage, log deletion, log downloading, log querying, log ranking, log browsing, and the like.
(1) The system logs are managed, and the logs are stored according to a storage rule;
(2) Log files can be automatically deleted according to the storage days, so that the occupied space of a hard disk is reduced;
(3) Storing the failed data, recording a failure log, and maintaining and reusing the failure log through a system;
(4) And displaying the analyzed log, so that the user can check and analyze conveniently.
According to the data processing system based on the Internet of things, the current limiting management device limits and filters the communication request, so that stability and reliability of the system are guaranteed, overload of the system is avoided, system breakdown and slow running are prevented, and performance and usability of the system are improved.
In some optional embodiments, the system further comprises a policy management device, wherein the policy management device manages and adjusts the system policy according to system changes; the policy management device is used for performing any one or more of complement copy scheme configuration, acquisition scheme configuration, push scheme configuration, time setting scheme configuration, current limiting scheme configuration or log scheme configuration.
In the embodiment of the invention, the policy management device realizes various system policy configurations, and the main functions comprise: the method comprises the following steps of complement copy scheme configuration, timing acquisition scheme configuration, pushing scheme configuration, time setting scheme configuration, current limiting scheme configuration and log scheme configuration.
And each strategy in the system is managed and adjusted, the system functions can be effectively executed through strategy management, and meanwhile, the strategy needs to be adjusted and optimized along with the continuous development and change of the system so as to adapt to the continuous changing environment and requirement.
The main functions of the policy management device include:
(1) The method comprises the steps of (1) configuring a supplementary reading scheme, carrying out supplementary reading on the condition of meeting the data deficiency through supplementary reading frequency, supplementary reading range, supplementary reading mode, supplementary reading result processing and data storage;
(2) The timing acquisition scheme is configured, and the aim of automatically acquiring data is achieved through acquisition frequency, acquisition range, acquisition mode, acquisition result processing and data storage;
(3) The configuration of the pushing scheme, namely pushing data, a pushing processing mode, a pushing address, the number of failed retries, a retrying interval and a retry waiting after failure, when the equipment generates the data, a message can be pushed to a user;
(4) The time setting scheme is configured, and the equipment and a system or gateway clock are kept synchronous through time setting frequency, time setting mode, time setting processing and communication time difference, so that the accuracy of the equipment time is ensured, and some problems caused by inaccurate equipment time are avoided;
(5) The configuration of a current limiting scheme ensures the stability and the safety of the system through a current limiting algorithm, a current limiting threshold value, a current limiting strategy and current limiting monitoring;
(6) The log scheme is configured, and the log level, the log output, the format, the analysis and the storage timeliness are used for facilitating system management and fault removal, so that a user can be helped to find problems in time, the problem reasons can be rapidly positioned, and the reliability and the stability of the system are improved.
The data processing system based on the internet of things provided by the embodiment manages and adjusts the system policy so as to effectively execute the corresponding functions of the system and adapt to the system change and the requirement.
In some optional embodiments, the system further includes a communication channel management device, where the communication channel management device implements a channel state and a channel communication function, and performs a message queue buffering process on communication data, and the main functions include: NBiot, TCP/IP, UDP, MQTT subscription channel management, and the like.
The main functions of the communication channel management device include:
(1) Configuring a communication channel of the system, including a communication protocol, a communication port and the like;
(2) Monitoring a communication channel in a system, and finding out communication problems in time;
(3) Maintaining the communication channel, cleaning out expired connection and optimizing communication performance;
(4) And supporting the functions of channel opening, stopping, parameter setting, data transmission, data receiving and the like.
In some alternative embodiments, the system further comprises an analog communication test tool, wherein the analog communication test tool mainly simulates communication between the test data acquisition platform and the device, supports simulation of a plurality of communication protocols such as DL/T698.45, Q/GDW376.1, NB-Iot, MQTT and the like, and mainly comprises the following functions: analog communication gateway, analog meter, analog platform push, timing communication, etc.
The main functions of the analog communication test tool include:
(1) Support more than 1 ten thousand communication gateways;
(2) Supporting more than 10 tens of thousands of analog meters;
(3) Supporting high concurrency communication testing;
(4) Analog message timing communication is supported.
Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.
Claims (10)
1. The data processing system based on the Internet of things is characterized by comprising an acquisition management device, a platform management device and an I/O management device, wherein,
the platform management device is connected with each target device;
the acquisition management device performs information interaction with each target device through a communication protocol adapted to each target device, and performs batch acquisition on data of each target device according to a preset acquisition scheme corresponding to each target device;
the platform management device acquires the data acquired by the acquisition management device, when the platform management device receives an access online notification of the target equipment, the target equipment is connected to the platform management device in an access way, and when the platform management device receives a replacement offline notification of the target equipment, the platform management device releases the link of the target equipment and the platform management device;
the I/O management device stores the acquired data acquired by the platform management device into the middleware, the I/O management device calls the data from the middleware according to service requirements, and the I/O management device pushes the data corresponding to the pushing scheme to a user through an API interface according to the pushing scheme.
2. The system of claim 1, further comprising a rights management unit, wherein,
the right management device is used for acquiring the identity authentication information of the user in the data request when the data request sent by the user is received;
and the authority management device performs identity authentication on the identity verification information through an API interface, returns a temporary access identifier if the identity authentication passes, and returns error information if the identity authentication does not pass.
3. The system of claim 1, further comprising a system gateway and a gateway management device, wherein,
the gateway management device is used for providing a system gateway list and configuring a gateway file;
the gateway management device is also used for monitoring the online state of the system gateway, and is used for carrying out communication setting on the online system gateway when the system gateway is online and inquiring the historical communication setting of the system gateway when the system gateway is not online.
4. The system of claim 3, further comprising a device management apparatus, wherein,
the device management device is used for judging whether the target device has a management function, if the target device has the management function, the device management device determines the target device as a management device, and if the target device does not have the management function, the device management device determines the target device as a common device;
when receiving the equipment access request of the user, the equipment management device issues files to the system gateway according to the control authority of the user.
5. The system of claim 1, further comprising a state management device, wherein,
the state management device is used for monitoring the online state of the target equipment in real time;
the state management device is also used for recording the historical online state of the target equipment and the online state change of the target equipment.
6. The system of claim 1, further comprising a protocol management device, wherein,
the protocol management device manages a plurality of communication protocols in the protocol data item through a communication command library, and the plurality of communication protocols are used for realizing communication between the target equipment and the acquisition management device.
7. The system of claim 1, further comprising a clock management device, wherein,
the clock management device carries out time setting on a clock of the target equipment according to a preset time setting scheme, and generates a time setting communication record which comprises a time setting success notification and a time setting failure notification;
the clock management device generates a time-to-time failure record according to the time-to-time failure notification.
8. The system of claim 1, further comprising a current limit management device, wherein,
the current limiting management device carries out current limiting filtration on the communication request of the user according to a preset current limiting scheme;
the flow limit management device generates a communication request queue according to the communication request and distributes the communication request to a plurality of servers.
9. The system of claim 1, further comprising a log management device that collects and stores system logs according to a preset storage rule, the system logs including failure logs, the log management device generating failure logs according to storage failure data.
10. The system according to any one of claims 1-9, wherein the system further comprises policy management means, wherein,
the strategy management device manages and adjusts the system strategy according to the system change;
the policy management device is used for performing any one or more of complement copy scheme configuration, acquisition scheme configuration, push scheme configuration, time setting scheme configuration, current limiting scheme configuration or log scheme configuration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311574009.9A CN117614981A (en) | 2023-11-23 | 2023-11-23 | Data processing system based on Internet of things |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311574009.9A CN117614981A (en) | 2023-11-23 | 2023-11-23 | Data processing system based on Internet of things |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117614981A true CN117614981A (en) | 2024-02-27 |
Family
ID=89949132
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311574009.9A Pending CN117614981A (en) | 2023-11-23 | 2023-11-23 | Data processing system based on Internet of things |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117614981A (en) |
-
2023
- 2023-11-23 CN CN202311574009.9A patent/CN117614981A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2446610C2 (en) | Stream-oriented setup for working in amr/ami-service networks | |
| CN107832196B (en) | Monitoring device and monitoring method for abnormal content of real-time log | |
| US7523184B2 (en) | System and method for synchronizing the configuration of distributed network management applications | |
| CN105631522A (en) | IT system operation and maintenance management system | |
| CN107622084A (en) | Blog management method, system and computer-readable recording medium | |
| KR100982034B1 (en) | Monitoring method and system for database performance | |
| CN103067230A (en) | Method for achieving hyper text transport protocol (http) service monitoring through embedding monitoring code | |
| CN107864056A (en) | A kind of distributed event acquisition probe, distributed event high speed acquisition system and method | |
| CN110971464A (en) | Operation and maintenance automatic system suitable for disaster recovery center | |
| CN111078490A (en) | Server safety guarantee method and system based on monitoring analysis of operating system | |
| WO2021139322A1 (en) | Method and apparatus for processing network device data, computer equipment and storage medium | |
| CN110888786A (en) | Operation and maintenance monitoring system | |
| CN114154035A (en) | Data processing system for dynamic loop monitoring | |
| CN114756328A (en) | Container cloud platform inspection method and device | |
| CN113076229B (en) | General enterprise-level information technology monitoring system | |
| CN111309557B (en) | Monitoring method, device, equipment and medium for multiple operating systems | |
| CN117422434A (en) | Wisdom fortune dimension dispatch platform | |
| CN117614981A (en) | Data processing system based on Internet of things | |
| CN111176950A (en) | Method and equipment for monitoring network card of server cluster | |
| CN115460072A (en) | Log processing system integrating log collection, analysis, storage and service | |
| CN116258603A (en) | Method and system for early warning based on equipment state of power system | |
| CN113407415A (en) | Log management method and device of intelligent terminal | |
| RU2575991C2 (en) | Network configuration management in communication networks | |
| CN109684158A (en) | Method for monitoring state, device, equipment and the storage medium of distributed coordination system | |
| CN113824592B (en) | Quantum network management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |