CN116366692A

CN116366692A - High-performance intelligent edge terminal system

Info

Publication number: CN116366692A
Application number: CN202310314249.9A
Authority: CN
Inventors: 吴炎泉; 陈思恩
Original assignee: Tech Valley Xiamen Information Technology Co ltd
Current assignee: Tech Valley Xiamen Information Technology Co ltd
Priority date: 2023-03-28
Filing date: 2023-03-28
Publication date: 2023-06-30

Abstract

The invention discloses a high-performance intelligent edge terminal system which comprises an edge terminal, an internet of things platform and an application end, wherein the edge terminal is used for data acquisition, storage, calculation and retrieval, the internet of things platform is used for data storage and service management, and the application end is used for calling data for application. The invention provides a high-performance intelligent edge terminal system, which solves the defects that the traditional mode needs to be subjected to multi-level computing treatment to cause high delay, low efficiency, slow updating and difficult to meet the subsequent changeable requirements, and the innovative cloud-edge cooperative solution has short transmission distance, low delay and high efficiency, and can acquire, clean and fuse data in an edge domain and quickly acquire an analysis result.

Description

High-performance intelligent edge terminal system

Technical Field

The invention relates to the technical field of data processing, in particular to a high-performance intelligent edge terminal system.

Background

With the continuous development of internet of things, more and more devices and sensors are connected to the internet, and a large amount of data is generated, and the data needs to be collected, stored, calculated and retrieved for analysis and application. The internet of things platform can realize the up-and-down function of the equipment and the application platform, and can access equipment of different levels of various types to the bottom and support various platform applications to the top. Conventional computing modes have limitations such as data transmission delay, bandwidth consumption, data security, system reliability, and cost. Edge computing is an emerging computing model that places computing and storage resources closer to the data sources, and can perform data collection, storage, computation, and retrieval locally, thereby avoiding the limitations of traditional cloud computing models. In addition, the development of the artificial intelligence technology provides technical support for the research of high-performance intelligent edge acquisition, storage calculation and retrieval terminals, and more intelligent data processing and application can be realized by applying the artificial intelligence technology to edge calculation.

However, the existing edge termination technology still has the following disadvantages:

1) Resource limitation: the computing and storage resources of the edge devices are limited, and the requirements of large-scale data processing cannot be met.

2) System complexity issues: the technologies of edge computing, cloud edge coordination and the like need to establish a complex system architecture and a communication protocol, and the complexity and maintenance cost of the system are increased.

3) Delay and bandwidth consumption of data transmission.

Disclosure of Invention

The invention aims to provide a high-performance intelligent edge terminal system for solving the problems.

In order to achieve the above purpose, the present invention adopts the following technical scheme:

the high-performance intelligent edge terminal system comprises an edge terminal, an internet of things platform and an application end, wherein the edge terminal is used for data acquisition, storage, calculation and retrieval, the internet of things platform is used for data storage and service management, and the application end is used for calling data for application.

Preferably, the edge terminal comprises a cloud end, edge equipment and an equipment end; the cloud end is used for non-real-time and long-period big data analysis mining, data sharing, training and upgrading of an algorithm model and data backup; the edge equipment is used for processing tasks of real-time and short-period data, real-time processing and execution of local business and localization service through a cloud output model algorithm; the equipment end is used for data acquisition and input.

Preferably, the edge device adopts an edge intelligent gateway, and the edge intelligent gateway cleans, packages and uploads the upper data input by the device end to the cloud end through the edge computing service and the data transparent transmission service, so as to realize cloud-edge cooperative data acquisition.

Preferably, the cloud end comprises an edge node management module, a terminal equipment management module, a cloud edge message routing module, a rule engine module, an equipment access management module and a data integration module, wherein the edge equipment comprises a protocol adaptation module, a data transmission module, a data definition module, an alarm management module, a fault management module, an edge intelligent module and an edge safety module, and the equipment end comprises a camera, a radar, a pressure sensor, a temperature and humidity sensor, a water level sensor and a current sensor.

Preferably, the data storage and retrieval of the edge terminal comprises a cluster, a character-dividing node and a data fragment on a data architecture, wherein the cluster is provided with a plurality of nodes of different types, the nodes are used for storing the state information of the cluster, and the data fragment is used for executing the writing task of the data.

Preferably, the data storage of the edge terminal adopts data node separation type storage, and the data retrieval of the edge terminal adopts a metadata server retrieval engine.

Preferably, the data storage and service management performed by the internet of things platform specifically includes the following steps:

s1, the edge terminal enters the Internet of things platform, and a device access gateway on the Internet of things platform performs authentication and protocol analysis;

s2, after analysis is successful, the edge terminal inputs the data message into the Internet of things platform through a message gateway, and the data message is cached in a Redis after being processed through a device registry;

s3, the Internet of things platform performs persistent storage on data and sets a rule engine at the same time;

s4, the application end calls the data of the Internet of things platform to apply.

Preferably, the internet of things platform is provided with a service management functional unit, and the service management functional unit comprises a system management module, a device management module, a network component module, a notification management module, a rule engine module and a log management module.

After the technical scheme is adopted, compared with the background technology, the invention has the following beneficial effects:

1. the invention provides a high-performance intelligent edge terminal system, which solves the defects that the traditional mode needs to be subjected to multi-level computing treatment to cause high delay, low efficiency, slow updating and difficult to meet the subsequent changeable requirements, and the innovative cloud-edge cooperative solution has short transmission distance, low delay and high efficiency, and can acquire, clean and fuse data in an edge domain and quickly acquire an analysis result.

2. The invention provides a high-performance intelligent edge terminal system, which is used for storing, calculating and searching data at an edge terminal, and placing calculation and storage resources at a place closer to a data source, so that the time and bandwidth consumption for transmitting the data to a cloud for processing are avoided, the delay of data transmission is reduced, and the data processing efficiency is improved.

3. The invention provides a high-performance intelligent edge terminal system, which is used for carrying out data processing on an edge terminal, so that the risk of transmitting data to a cloud is avoided, the safety of the data is improved, and in addition, the acquisition and the processing of the data are not influenced even if a network fails, so that the reliability of the system is improved.

4. The invention provides a high-performance intelligent edge terminal system, which can realize more intelligent data processing and application by applying an artificial intelligent algorithm to edge calculation, thereby improving the value of data.

Drawings

FIG. 1 is a diagram of a cloud-edge co-architecture of the present invention;

FIG. 2 is a diagram of a storage technology architecture of a search engine according to the present invention;

FIG. 3 is a diagram of a search architecture of the present invention;

fig. 4 is a flow chart of data storage and service management performed by the internet of things platform according to the invention.

Detailed Description

The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Examples

The invention discloses a high-performance intelligent edge terminal system, which is matched with figures 1 to 4, and comprises an edge terminal, an internet of things platform and an application end, wherein the edge terminal is used for data acquisition, storage, calculation and retrieval, the internet of things platform is used for data storage and service management, and the application end is used for calling data for application.

The internet of things platform is used for accessing the edge terminal under the pair, the edge terminal comprises equipment ends of various types and different levels, the edge intelligent gateway and a third party cloud end, various application ends are supported on the pair, the internet of things platform is based on the internet of things platform for equipment networking application construction, the equipment of various levels are fused in the data and service layers to different degrees, and the equipment interconnection and intelligent processing of equipment networking are comprehensively realized.

The edge terminal comprises a cloud end, edge equipment and equipment ends, and the edge terminal adopts a cloud-edge cooperative data acquisition and fusion processing mode to combine a central node of the cloud end, a plurality of edge nodes of the edge equipment and equipment nodes of the equipment ends, so that acquisition and fusion calculation tasks are completed cooperatively among the cloud end, the edge equipment and the equipment ends.

The cloud end is used for non-real-time and long-period big data analysis mining, data sharing, training and upgrading of an algorithm model and data backup; the edge equipment is used for carrying out localization service through a model algorithm output by a cloud end, wherein the model algorithm is used for carrying out processing tasks of real-time and short-period data, carrying out real-time processing and execution of local business; the equipment end is used for data acquisition and input.

In order to meet the calculation mode of cloud edge cooperation, edge equipment adopts an edge intelligent gateway in an edge domain, and communication is carried out through physical interfaces such as an RS485/RS422/RS232 serial port and a network port or wireless transmission access equipment ends such as an AP and a LoRa and the like by using Modbus, TCP/IP or other protocols. The edge intelligent gateway provides edge computing service and data transparent transmission service, packages and uploads the data input by the equipment end to the cloud after cleaning, and realizes cloud edge cooperative data acquisition. A user can monitor the data acquisition and fusion processing system cooperated with the whole cloud edge by accessing the cloud node.

In the framework of the cloud access of the edge equipment, the cloud comprises an edge node management module, a terminal equipment management module, a cloud message routing module, a rule engine module, an equipment access management module and a data integration module, wherein the edge equipment comprises a protocol adaptation module, a data transmission module, a data definition module, an alarm management module, a fault management module, an edge intelligent module and an edge safety module, and the equipment comprises a camera, a radar, a pressure sensor, a temperature and humidity sensor, a water level sensor and a current sensor.

In the architecture, the edge equipment and the cloud end are more focused on the key fields of data collaboration, business collaboration and application collaboration. The edge equipment pays attention to data acquisition, preprocessing and task scheduling execution, cloud end is more heavy to data integration, real-time analysis and rule engines for various task business arrangement, and various equipment end access parameter configuration and network management.

The edge node of the edge device mainly faces to various ends, and encapsulates protocols and routing configuration of various ends through device management and gateway management on device end access. The cloud is mainly oriented to an edge node based on an http protocol and is mainly responsible for management of a network. The first step of coordination of the edge nodes and the cloud service is coordination of platform basic functions and security management, user registration and api service interaction are both required to be coordinated at two ends, and functions of a plurality of edge nodes are also required to be integrated from a cloud interface. The basic users, services, configuration and logs are cooperatively managed, and the cloud has unified panel display and system notification functions.

The data collaboration comprises modeling and marking of the data collected by the edge nodes, and the network flow and the calculation complexity can be greatly saved after preliminary cleaning. The cloud end can combine the actual business requirements, fusion and real-time calculation processing of the actual business data are performed according to a plurality of aggregation or classification algorithms, and data display and export services are provided at the cloud end.

The edge node provides network value-added application deployment and operation environment; the cloud end realizes life cycle management of the value added network application of the edge node, and comprises pushing, installing, unloading, updating, monitoring, journaling and the like of the application. The center node can hatch and start the existing application mirror image on different edge clouds, and high-availability guarantee and thermal migration of the application are completed.

The edge node provides a value added network service application instance in cooperation with service management; the cloud provides unified service arrangement capability of the value-added network service, and provides relevant network value-added service for clients as required.

The edge terminal designs a storage technology architecture of a search engine, and supports two modes of JSON and Document on storage to realize object storage.

The data storage and retrieval of the edge terminal comprises a cluster, a character dividing node and a data fragment on a data architecture, wherein the cluster is provided with a plurality of nodes of different types, the nodes are used for storing the state information of the cluster, and the data fragment is used for executing the writing task of data.

The nodes can be divided into a main node, a data node, a coordination node, a preprocessing node and the like according to roles.

And (3) a master node: is responsible for managing cluster state information, including processing requests to create, delete indexes, etc., deciding to which node a shard is assigned, maintaining and updating cluster state.

And (3) a coordination node: is responsible for receiving requests from clients, routing the requests to the appropriate nodes, and aggregating the results together.

Data node: the node is used for storing data, and the problem of horizontal expansion and single point of data can be solved by adding the data node. The data node is stored by adopting a data slicing technology (data is stored in the corresponding storage node according to a certain rule) and a data copying technology (data backup) in combination, so that high availability and reliability of the data are ensured.

Preprocessing nodes: the data preprocessing and conversion node supports pipeline setting, and can perform operations such as filtering and conversion on the data.

The edge terminal performs an elastiscearch index optimization. The index shards of the elastic search have two types of main shards and duplicate shards, and are stored in a cluster in a scattered manner on different nodes. When writing data, the system defaults that the main and auxiliary slices simultaneously execute the writing task of the data, but the writing quantity of the data in the same time is reduced, so that the writing efficiency of the data in the index is reduced. In order to solve the difficulty, in the embodiment, the remaining space within 85% of the disk space is calculated in optimization, then the size of each fragment is set to 25G recommended by authorities, and the setting of the number of index fragments in the cluster system is obtained by dividing the remaining space of the disk by the size of each fragment, so that the number of index fragments is reasonably distributed, the dynamic setting of the number of index fragments is realized, and the creation of an index library and the efficient writing of data are completed.

The data storage of the edge terminal adopts data node separation storage, and the data retrieval of the edge terminal adopts a metadata server retrieval engine. The data file is stored in the data node based on a separate storage of the metadata server search engine from the data node. The specific data of each file is divided into a plurality of data pieces, and the data pieces are stored in a data server in a redundant mode. Each file data block is characterized as a pair of files that are dequeued on a data node, one being a data file and one being a metafile of additional information.

When a user reads data, the user firstly acquires the position information of the target data block from the metadata node and then directly interacts with the storage server. At the same time, clusters of control nodes are utilized to ensure high availability of the system. Through load balancing and data concurrency access strategies, transmission rates up to tens of GB/s can be obtained under the common hardware condition.

Meanwhile, in the embodiment, the search of the edge terminal is designed with high concurrency and multiple elements.

High concurrency: in order to meet the high concurrent data read-write requirement of a search engine and the requirement of mass data expansion, and ensure the high availability of data and access, the engine supports three access architectures: the horizontal expansion architecture and the read-write separation architecture are used for improving the inquiry performance and realizing high concurrency retrieval by adopting the cold-hot separation architecture.

And (5) diversified search: the search requirements of various scenes are met by providing search of a query character string, language of a specific field, filtering query, full-text search, phrase search and highlight search. Meanwhile, the query language designates the corresponding data node to query, and after the data of the query node is obtained, the query node finds the main node, so that the fragments of other data nodes are found, and the query result is aggregated.

The internet of things platform for data storage and service management specifically comprises the following steps:

s1, enabling an edge terminal to enter an Internet of things platform, and enabling equipment on the Internet of things platform to access a gateway for authentication and protocol analysis;

s2, after analysis is successful, the edge terminal inputs the data message into the Internet of things platform through the message gateway, and the data message is cached in the Redis after being processed through the equipment registration center;

s3, the internet of things platform performs persistent storage on data in a plurality of modes such as Mysql, postgreSQL, elastciSearch, redis, and meanwhile, sets a rule engine to finish the operations such as equipment alarming, scene linkage, data forwarding and the like;

s4, the application end calls the data of the Internet of things platform to apply. The application end can call the platform data through the unified openAPI, and can also carry out statistical analysis and visual large-screen display on the platform data.

The internet of things platform is provided with a service management functional unit, and the service management functional unit comprises a system management module, a device management module, a network component module, a notification management module, a rule engine module and a log management module. The system management module comprises user management, authority management, role management, mechanism management and API management, the equipment management module comprises protocol management, products, equipment, gateway equipment, geographic position and visualization, the network component comprises certificate management, component management and equipment access, the notification management module comprises notification configuration, notification templates, mails, short messages and the like, the rule engine module comprises a rule model, a rule instance, equipment alarms, scene linkage and data transfer valves, and the log management module comprises an access log and a system log.

The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A high-performance intelligent edge terminal system is characterized in that: the system comprises an edge terminal, an internet of things platform and an application end, wherein the edge terminal is used for data acquisition, storage, calculation and retrieval, the internet of things platform is used for data storage and service management, and the application end is used for calling data for application.

2. The high performance intelligent edge termination system of claim 1, wherein: the edge terminal comprises a cloud end, edge equipment and an equipment end; the cloud end is used for non-real-time and long-period big data analysis mining, data sharing, training and upgrading of an algorithm model and data backup; the edge equipment is used for processing tasks of real-time and short-period data, real-time processing and execution of local business and localization service through a cloud output model algorithm; the equipment end is used for data acquisition and input.

3. A high performance intelligent edge termination system according to claim 2, wherein: the edge equipment adopts an edge intelligent gateway, and the edge intelligent gateway cleans, packages and uploads the upper data input by the equipment end to the cloud end through the edge computing service and the data transparent transmission service, so that cloud-edge collaborative data acquisition is realized.

4. A high performance intelligent edge termination system according to claim 2, wherein: the cloud comprises an edge node management module, a terminal equipment management module, a cloud edge message routing module, a rule engine module, an equipment access management module and a data integration module, wherein the edge equipment comprises a protocol adaptation module, a data transmission module, a data definition module, an alarm management module, a fault management module, an edge intelligent module and an edge safety module, and the equipment end comprises a camera, a radar, a pressure sensor, a temperature and humidity sensor, a water level sensor and a current sensor.

5. The high performance intelligent edge termination system of claim 1, wherein: the data storage and retrieval of the edge terminal comprises a cluster, a character dividing node and a data fragment on a data architecture, wherein the cluster is provided with a plurality of nodes of different types, the nodes are used for storing state information of the cluster, and the data fragment is used for executing a data writing task.

6. The high-performance intelligent edge termination system of claim 5, wherein: and the data storage of the edge terminal adopts data node separation type storage, and the data retrieval of the edge terminal adopts a metadata server retrieval engine.

7. The high performance intelligent edge termination system of claim 1, wherein: the internet of things platform for data storage and service management specifically comprises the following steps:

8. The high performance intelligent edge termination system of claim 1, wherein: the Internet of things platform is provided with a service management functional unit, and the service management functional unit comprises a system management module, a device management module, a network component module, a notification management module, a rule engine module and a log management module.