CN117743471A

CN117743471A - Processing system for data of Internet of things

Info

Publication number: CN117743471A
Application number: CN202410168478.9A
Authority: CN
Inventors: 路培杰; 杨辉; 周志忠; 刘文虎; 邹晨阳
Original assignee: Zhongke Yungu Technology Co Ltd
Current assignee: Zhongke Yungu Technology Co Ltd
Priority date: 2024-02-06
Filing date: 2024-02-06
Publication date: 2024-03-22
Anticipated expiration: 2044-02-06
Also published as: CN117743471B

Abstract

The embodiment of the application provides a processing system for data of the Internet of things. Comprising the following steps: the edge end comprises a container arrangement platform which is used for providing containers for container interface services of the distributed system and preset Internet of things data acquisition applications, wherein the preset Internet of things data acquisition applications are used for acquiring Internet of things data of engineering equipment in real time, and the container interface services of the distributed system are used for mapping the Internet of things data to a common cloud; the shared cloud comprises a distributed system which is used for receiving the data of the Internet of things and performing persistent storage; the private cloud end comprises a client end of the distributed system, wherein the client end of the distributed system is used for synchronizing the internet of things data stored in the distributed system to the private storage system in real time for storage, and the data analysis system is used for extracting target internet of things data corresponding to a calculation task from the private storage system under the condition that the private cloud end receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data.

Description

Processing system for data of Internet of things

Technical Field

The application relates to the technical field of industrial Internet of things, in particular to a processing system for Internet of things data.

Background

With the development of emerging technologies such as big data, cloud computing, internet of things, blockchain, artificial intelligence, 5G communication and the like, all things are interconnected, and man-machine interaction has become a very common application scene at present. The universal interconnection is that all intelligent devices located at the edge end can be required to be connected to the internet, data of the devices can be collected in real time, and meanwhile interaction data information can be required to be sent to the edge end through the cloud. At present, the real-time acquisition of mass internet of things data is still a very complex process with very high technical requirements. Particularly in the field of industrial Internet, industrial equipment has complex physical composition, high intelligent level and various operation conditions, a large number of sensors with different functions acquire the IOT data of the equipment in real time, the IOT data acquired in the process is required to be transmitted to a cloud platform in real time by a specific encryption transmission protocol, and data analysis, real-time decision making, feedback and the like are carried out at the cloud.

In the prior art, the acquired IOT data of all sensors needs to be summarized through a PLC bus located at a device side and then transmitted to an edge box, and the edge box encrypts the data according to a specific transmission protocol and then transmits the data to a cloud through a 5G network. Because of the large number of networked devices, it is necessary to load balance IOT data of different devices according to IP through an LVS load balancer and then transmit the IOT data to different cloud gateways. The cloud gateway has different types of IOT gateways according to different transmission protocols, and each gateway is customized and developed according to a specific protocol and is used for processing IOT data of IOT data transmission equipment adopting the specific protocol. The IOT data is sent to a message queue kafka of the cloud after preliminary processing at the IOT gateway side, is stored in a private cloud in a centralized mode after processing of a real-time computing engine Flink, and is used by an Internet of things platform. The method has complex flow, and not only results in longer acquisition and transmission flow of the IOT data. And the problem of data throughput exists, when the number of accessed devices is too large and the data peak value is high, the process cannot process huge data volume, and the acquisition of IOT data of mass Internet of things devices can be completed only with high software and hardware cost and labor cost investment.

Disclosure of Invention

The embodiment of the application aims to provide a processing system for internet of things data, which is used for solving the technical defects of low acquisition efficiency and high acquisition cost caused by complex internet of things data acquisition flow in the prior art.

To achieve the above object, a first aspect of the present application provides a processing system for data of the internet of things, the processing system comprising:

the edge end comprises a container arrangement platform, a container interface service of the distributed system and a preset internet of things data acquisition application, wherein the container arrangement platform is used for providing a container for the container interface service of the distributed system and the preset internet of things data acquisition application; the Internet of things data acquisition application is used for acquiring Internet of things data of engineering equipment in real time; the container interface service of the distributed system is used for establishing connection between the edge end and the common cloud end and mapping the data of the Internet of things to the common cloud end in real time;

the shared cloud comprises a distributed system, wherein the distributed system is used for receiving the internet of things data sent by the edge end in real time and performing persistent storage;

the private cloud comprises a client of the distributed system, a private storage system and a data analysis system, wherein the client of the distributed system is used for establishing connection between the distributed system and the private cloud so as to synchronize the Internet of things data stored in the distributed system to the private storage system for storage in real time; the data analysis system is used for extracting target internet of things data corresponding to the calculation task from the private storage system under the condition that the private cloud receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data.

In an embodiment of the present application, the common cloud further includes: the object storage system is connected with the distributed system and comprises a plurality of storage barrels, and each storage barrel is used for storing the internet of things data of the same type of engineering equipment; the metadata storage system is connected with the distributed system and is used for storing metadata information in the object storage system, the metadata information at least comprises a name of each storage bucket, a first connection address, a first login user name, a first login password and first data file directory information in each storage bucket, and the metadata storage system is also used for providing the first connection address, the first login user name and the first login password of a corresponding target storage bucket according to engineering equipment to which the Internet of things data belongs under the condition that the distributed system receives the Internet of things data so as to establish connection with the target storage bucket and store the Internet of things data into the target storage bucket.

In an embodiment of the present application, the common cloud is further configured to: respectively deploying a metadata storage system and an object storage system; respectively acquiring first access information of the object storage system and second access information of the metadata storage system, wherein the first access information comprises a name of each storage barrel, a first connection address, a first login user name and a first login password, and the second access information comprises a second login user name, a second login password and a second connection address of the metadata storage system; acquiring an installation package of the distributed system, and deploying the distributed system based on the installation package; constructing a first operation instruction for the distributed system based on the first access information and the second access information; the first operational instructions are executed to cause the distributed system to establish a connection with the metadata storage system and the object storage system, respectively.

In an embodiment of the present application, the data analysis system further includes: the message queue comprises a plurality of topics, and each topic is used for storing the internet of things data of the same type of engineering equipment calculated by the preset calculation engine.

In an embodiment of the present application, the edge is further configured to: acquiring a deployment script of the container programming platform, and constructing a second operation instruction of the service node based on the deployment script; executing a second operation instruction to deploy the service node to the edge end; obtaining a token of the service node, and constructing a third operation instruction of the proxy node based on the token of the service node; executing a third operation instruction to deploy the proxy node to the edge end; and under the condition that the operation states of the service node and the proxy node are detected to be opened, determining that the container arranging platform is completely deployed.

In an embodiment of the present application, the container orchestration platform is further for: acquiring a first containerized deployment script file for a container interface service of a distributed system; and under the condition that a first operation instruction input by a user through the container arrangement platform is acquired, executing the first containerized deployment script file according to the first operation instruction so as to deploy the container interface service of the distributed system to the container arrangement platform.

In an embodiment of the present application, the container orchestration platform is further for: defining a second containerized deployment script file of the key, and adding a second connection address of the metadata storage system and first access information of the object storage system to the second containerized deployment script file; under the condition that a second operation instruction input by a user through the container arrangement platform is acquired, executing a second containerized deployment script file according to the second operation instruction so as to deploy the secret key to the container arrangement platform; a third containerized deployment script file defining a storage class of the distributed system, wherein the third containerized deployment script file includes a name of the storage class, a plug-in name, and a name of a key; and under the condition that a third running instruction input by a user through the container arrangement platform is acquired, executing a third containerized deployment script file according to the third running instruction so as to deploy the storage class of the distributed system to the container arrangement platform.

In an embodiment of the present application, the edge is further configured to: defining a fourth containerized deployment script file of a preset Internet of things data acquisition application, wherein the fourth containerized deployment script file comprises an image file, a service port, a data persistence configuration file and a PVC configuration file of the preset Internet of things data acquisition application; adding the storage class into the PVC configuration file; and under the condition that a fourth operation instruction input by a user through the container arrangement platform is acquired, executing a fourth containerized deployment script file according to the fourth operation instruction so as to deploy the preset Internet of things data acquisition application to the container arrangement platform.

In an embodiment of the present application, the preset internet of things data acquisition application is further configured to: acquiring Internet of things data acquired by a sensor on engineering equipment in real time, and preprocessing the Internet of things data; the container interface service based on the distributed system establishes connection with the distributed system, and the preprocessed internet of things data is mapped to the object storage system in real time through the storage class for persistent storage.

In the embodiment of the present application, the private cloud is further used to: acquiring an installation script of a client of a distributed system; executing the installation script under the condition that a fifth running instruction aiming at the installation script is acquired, so as to deploy the client of the distributed system into the private cloud; establishing connection between the distributed system and the private cloud based on a client of the distributed system; constructing a mounting script for the distributed system, wherein the mounting script comprises second access information of the metadata storage system; and executing the mounting script to mount the first data file catalog in the object storage system to the second data file catalog in the private storage system so as to synchronize the internet of things data stored in each storage bucket in the object storage system to the private storage system for storage in real time.

In the embodiment of the present application, the private cloud is further used to: under the condition that a control instruction aiming at engineering equipment is acquired, caching the control instruction to a theme corresponding to the engineering equipment in a message queue; calculating control data cached in the theme based on a preset calculation engine, transmitting the calculated control data to a second data file directory in the private storage system, and mounting the control data to a first data file directory in the object storage system based on a client of the distributed system; and issuing control data to a preset data acquisition application of the Internet of things through a container interface service of the shared cloud based distributed system so as to control engineering equipment to execute corresponding operations.

According to the technical scheme, the processing system for the data of the Internet of things comprises: the edge end comprises a container arrangement platform which is used for providing containers for container interface services of the distributed system and preset Internet of things data acquisition applications, wherein the preset Internet of things data acquisition applications are used for acquiring Internet of things data of engineering equipment in real time, and the container interface services of the distributed system are used for mapping the Internet of things data to a common cloud; the shared cloud comprises a distributed system which is used for receiving the data of the Internet of things and performing persistent storage; the private cloud end comprises a client end of the distributed system, wherein the client end of the distributed system is used for synchronizing the internet of things data stored in the distributed system to the private storage system in real time for storage, and the data analysis system is used for extracting target internet of things data corresponding to a calculation task from the private storage system under the condition that the private cloud end receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data. According to the method, the data link between engineering equipment, the edge end, the shared cloud end and the private cloud end is opened, real-time synchronization and acquisition of the data of the Internet of things of the engineering equipment are realized based on the edge end, the acquisition flow of the data of the Internet of things is simplified, centralized storage is carried out, the acquisition efficiency of the data of the Internet of things is effectively improved, and the acquisition cost of the data of the Internet of things is effectively reduced.

Additional features and advantages of embodiments of the present application will be set forth in the detailed description that follows.

Drawings

The accompanying drawings are included to provide a further understanding of embodiments of the present application and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and together with the description serve to explain, without limitation, the embodiments of the present application. In the drawings:

FIG. 1 schematically illustrates a block diagram of a processing system for Internet of things data according to an embodiment of the present application;

FIG. 2 schematically illustrates a block diagram of yet another processing system for Internet of things data according to an embodiment of the present application;

FIG. 3 schematically illustrates a flow diagram of a processing system for Internet of things data according to an embodiment of the present application;

fig. 4 schematically shows a flow diagram of a data analysis system according to an embodiment of the present application.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the specific implementations described herein are only for illustrating and explaining the embodiments of the present application, and are not intended to limit the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.

In addition, if there is a description of "first", "second", etc. in the embodiments of the present application, the description of "first", "second", etc. is for descriptive purposes only and is not to be construed as indicating or implying a relative importance thereof or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.

Fig. 1 schematically shows a block diagram of a processing system for internet of things data according to an embodiment of the present application. As shown in fig. 1, an embodiment of the present application provides a processing system for data of the internet of things, where the processing system may include:

the edge 110 comprises a container arrangement platform 112, a container interface service 114 of the distributed system and a preset internet of things data acquisition application 116, wherein the container arrangement platform 112 is used for providing containers for the container interface service 114 of the distributed system and the preset internet of things data acquisition application 116; the preset internet of things data acquisition application 116 is used for acquiring internet of things data of engineering equipment in real time; the container interface service 114 of the distributed system is configured to establish a connection between the edge 110 and the common cloud 120, and map the internet of things data to the common cloud 120 in real time.

The edge is typically between the terminal and the cloud, gathering data on the side near the source of the data. In the technical scheme, the edge end can be arranged between the shared cloud end and engineering equipment and can be used for collecting the internet of things data of the engineering equipment. Specifically, the edge may include a container orchestration platform, a container interface service of a distributed system, and a preset internet of things data collection application. The container arrangement platform can refer to K3s, and can only remotely manage the micro-service application of the edge end through a lightweight container management tool such as K3s at present due to the limitation of computer hardware and related technical level of the edge end, so that the resource consumption of the edge end is further reduced, and meanwhile, the corresponding functions of remote management and network access are realized. K3s is a published version of Kubernetes, which is highly optimized for edges. Although K3s is a simplified version and a mini version of the Kubernetes, the consistency and the function of the API are not affected, from kubectl to Helm to the Kubernetes, almost all tools of the cloud native ecological system can be seamlessly docked with K3s, and the tools can be deployed in the production environment of K3 s. The K3s is characterized by the simplicity, is packaged and deployed as a single binary file (about 100 MB), can be successfully installed in a few seconds to obtain a fully mature Kubernetes cluster, has the installation experience as simple as running a script on each node of the cluster, and is very suitable for installation and deployment at the edge. Meanwhile, the K3s binary is a self-sufficient encapsulation entity that runs almost all components of the Kubernetes cluster, including the API server, the scheduler, and the controller. Specifically, in the technical scheme, after the container arrangement platform K3s is installed at the edge end, the container arrangement platform K3s can provide containers for the container interface service of the distributed system and the preset internet of things data acquisition application, so that the container interface service of the distributed system and the preset internet of things data acquisition application normally run at the edge end, and remote management and remote scheduling functions are provided for the internet of things data.

In the technical scheme, a java language code is selected to develop a lightweight micro-application app for data acquisition, so that the preset internet of things data acquisition application can refer to the lightweight micro-application app for data acquisition. After the engineering equipment at the edge starts to work, all the sensors positioned on the engineering equipment start to collect the internet of things data of the engineering equipment. When the micro-application app monitors that the PLC bus has the data transmission of the Internet of things through the interface, the data acquisition service can write the data into a specific directory in a container where the micro-application app is located after the data is simply processed by the back end.

The container interface service of the distributed system may refer to a container interface service juicefs CSI Driver of the distributed system juicefs, where the container interface service of the juicefs is a series of interfaces for the container running in K3s or K8s to interact with an external system, specifically, the external system interfaces with the container in the K3s cluster through the api server of K3s, and it means that data in the container may be persisted into the external system outside the container, or the container may be enabled to read data in the external system. Meanwhile, the container interface service in the technical scheme can integrate an interface of a client-side juicefs client of the distributed system juicefs internally, and the edge terminal can further realize communication with the shared cloud based on the container interface service. Meanwhile, after the internet of things data of the engineering equipment is obtained in real time by the preset internet of things data collection application app, the edge end can map the internet of things data obtained in real time by the preset internet of things data collection application app to the common cloud end based on the container interface service juicefs CSI Driver of the distributed system so as to perform persistent storage on the internet of things data through the common cloud end, so that the internet of things data can be called by the terminal internet of things platform.

The common cloud 120 includes a distributed system 122, where the distributed system 122 is configured to receive and store the internet of things data sent by the edge 110 in real time.

The public cloud is a public cloud, generally refers to a cloud which can be used and is provided by a third party provider for a user, the public cloud can be generally used through the Internet, the public cloud can be free or low in cost, and the core attribute of the public cloud is shared resource service.

The distributed system can refer to Juicefs, which is a high-performance distributed system facing to cloud native design, provides complete POSIX compatibility, can store and access almost all objects locally as mass local disks, and can mount read-write on different hosts across platforms and regions at the same time. Therefore, in the technical scheme, the juicefs can be selected to construct unified storage on the shared cloud. Specifically, after the distributed system Juicefs is deployed in the common cloud, the distributed system Juicefs may receive the internet of things data sent by the edge node in real time based on the container interface service Juicefs CSI Driver of the distributed system, and perform persistent storage on the mapped internet of things data.

The private cloud 130 comprises a client 132 of a distributed system, a private storage system 134 and a data analysis system 136, wherein the client 132 of the distributed system is used for establishing connection between the distributed system 122 and the private cloud 130 so as to synchronize the internet of things data stored in the distributed system 122 to the private storage system 134 in real time for storage; the data analysis system 136 is configured to extract, when the private cloud 130 receives the computing task, target internet of things data corresponding to the computing task from the private storage system 134, calculate the target internet of things data based on a preset computing engine, and output the calculated internet of things data.

The private cloud is a cloud computing infrastructure built inside an enterprise, and can also be understood as a cloud platform deployed inside the enterprise. Various cloud services such as computing resources, storage resources, and network resources may be included in this platform. The enterprise can autonomously deploy application programs, store data, realize allocation and management of resources and the like on the platform, which is equivalent to establishing an enterprise-level cloud platform. The private cloud is different from the public cloud in that cloud resources of the private cloud are all deployed in an enterprise's own data center or other places controlled by the enterprise, and the public cloud is provided by a cloud service provider, so that users can access the cloud resources through the internet. In addition, the private cloud end is usually in a closed network environment and is only opened to users and application programs in enterprises, so that the safety of data can be better ensured.

In the technical scheme, the private cloud comprises a client of a distributed system, a private storage system and a data analysis system, wherein the client of the distributed system can be a client of a distributed system juicefs, and the client juicefs can be used for creating and mounting a file system. Specifically, the client-side juicefs client of the distributed system can establish connection between the distributed system juicefs and the private cloud, so that a file system in the private storage system can be mounted to the file system in the distributed system juicefs, and internet of things data stored in the distributed system juicefs can be synchronized to the private storage system in real time for storage. The preset calculation engine in the data analysis system can be referred to as a flank calculation engine, which is a framework and a distributed processing engine, and can be used for calculating states of unbounded and bounded data, and the flank has the advantages of low delay, high throughput, structural accuracy and good fault tolerance. Specifically, the data analysis system may extract, when the private cloud receives the computing task, target internet of things data corresponding to the computing task from the private storage system, calculate the target internet of things data based on a preset computing engine Flink, and output the calculated internet of things data.

In an embodiment of the present application, as shown in fig. 2, there is provided a block diagram of a processing system for data of the internet of things, in fig. 2, the common cloud further includes: the object storage system 124 is connected with the distributed system 122 and comprises a plurality of storage buckets, and each storage bucket is used for storing the internet of things data of the same type of engineering equipment; the metadata storage system 126 is connected to the distributed system 122, and is configured to store metadata information in the object storage system, where the metadata information includes at least a name of each storage bucket, a first connection address, a first login user name, a first login password, and first data file directory information in each storage bucket, and the metadata storage system is further configured to, when the distributed system receives internet of things data, provide, according to engineering equipment to which the internet of things data belongs, the first connection address, the first login user name, and the first login password of a corresponding target storage bucket, so as to establish connection with the target storage bucket and store the internet of things data in the target storage bucket.

In the technical scheme, the distributed system juicefs adopts a framework of data and metadata separated storage, so that the distributed design of a file system is realized, and therefore, the metadata storage service and the data storage service of the juicefs are required to be deployed on a shared cloud respectively. Specifically, the object storage and metadata storage systems are deployed in a common cloud for providing data storage services and metadata storage services for juicefs, respectively. The object storage system can be referred to as a mini, which is a distributed, multi-copy, reliable and cheap object storage system, and can read and write data with the juicefs through an s3 protocol. Specifically, the object storage system minio is connected with the distributed system juicefs, and the object storage system minio comprises a plurality of storage bucket pockets, wherein each storage bucket pocket can be used for storing the internet of things data of the same type of engineering equipment. Specifically, after the distributed system Juicefs receives the internet of things data sent by the edge end based on the container interface service Juicefs CSI Driver of the distributed system in real time, the internet of things data of the same type of equipment engineering equipment is transmitted to a corresponding storage bucket socket in the object storage system minio for persistent storage.

The metadata storage system can be referred to as redis, which is a remote dictionary service, and is an in-memory database, and has high data reading and writing efficiency. Therefore, the redis can be selected as a metadata storage system of the distributed system juicefs, and the data access performance can be effectively improved by using the redis to store metadata. Specifically, the metadata storage system rediss is connected with the distributed system juicefs, the collection of metadata is a behavior of the juicefs, no human is needed for collection, and after the distributed system juicefs metadata storage system rediss and the distributed system juicefs are deployed to a shared cloud, the juicefs can automatically synchronize metadata to be stored in the rediss for storage. In particular, metadata stored by the metadata storage system redis may refer to metadata information in the storage object storage system mini. The metadata information in the minio at least comprises a name of each storage bucket socket, a first connection address, a first login user name, a first login password and first data file directory information in each storage bucket socket. Meanwhile, the metadata storage system redis is further used for providing a first connection address, a first login user name and a first login password of a corresponding target storage bucket socket according to engineering equipment to which the internet of things data belongs under the condition that the internet of things data is received by the distributed system juicefs, so that connection is established with the target storage bucket socket and the internet of things data is stored in the target storage bucket socket.

In an embodiment of the present application, as shown in fig. 3, a flow diagram of a processing system for internet of things data is provided. In fig. 3, the processing system includes an edge device end, a common cloud data center end and a private cloud data center end, wherein a lightweight cloud native technology K3S is introduced at the edge device end, and based on a container environment of the edge end, fusion of the common cloud distributed system juicefs and an edge application app is innovatively realized through a container interface service juicefs CSI Driver of the distributed system juicefs. And fully combines the advantages of large bandwidth, high-efficiency storage, good stability and low comprehensive cost of the shared cloud, and realizes the high-efficiency storage and synchronization of mass data of the Internet of things. Meanwhile, selecting an object storage system minio as a data storage service of the distributed system juicefs at the shared cloud, and selecting rediss as a metadata storage service of the distributed system juicefs. And simultaneously mounting and reading file systems of the common cloud distributed system Juicefs through a client Juicefs client of the Juicefs in the private cloud so as to provide data consistency assurance of the common cloud and the private cloud through the Juicefs, so that the private cloud can quickly acquire corresponding Internet of things data from the distributed system Juicefs in the common cloud, and locally analyze and process the data. And because the read-write process of the private cloud end and the shared cloud end is bidirectional, data such as instructions, parameter configuration, upgrade files and the like can be uploaded from the private cloud end to the shared cloud end, and the synchronization of the shared cloud end data to the edge end can be realized by means of a juicefs multi-end data synchronization mechanism in the shared cloud end, so that the management function of the private cloud end on the engineering equipment of the edge end is realized.

In the technical scheme, the distributed system can refer to juicefs, the metadata storage system can refer to redis, the object storage system can refer to minio, and because juicefs adopts a framework of data and metadata separated storage, the distributed design of the file system is realized, metadata storage services redis and data storage services of juicefs are required to be deployed on a shared cloud respectively, and the data storage selects the distributed object storage minio as a storage system of juicefs. Specifically, before deploying the distributed system juicefs to the common cloud, the object storage system minio and the metadata storage system redis need to be deployed to the common remote end. Specifically, the deployment of redis can select a sentinel mode for deployment to ensure the reliability of the juicefs metadata service. The redis sentinel mode is different from cluster deployment and is based on a redis master-slave deployment architecture. The nodes are required to be deployed with a sentinel mode, the sentinel mode can monitor whether all redis working nodes are normal, when a Master is in a problem, other nodes lose contact with a main node, so that voting is performed, the Master is considered to be in a problem indeed after the voting is performed in half, a sentinel is notified, and one Master is selected from Slaves to be a new Master. As an object storage system following the S3 protocol, the minio is more convenient to integrate with the juicefs, so in the technical scheme, erasure codes of the minio are selected to be started in the deployment process of the minio to prevent downtime of a plurality of nodes and bit attenuation bit rot, and a multi-copy mechanism is started at the same time, so that the reliability of data storage in the minio is improved. After the mini deployment is completed, a mini Console can be logged in, and a plurality of storage bucket pockets for the internet of things data storage of different types of engineering equipment are created, wherein the bucket names are defined as iot _pockets.

After the metadata storage system rediss and the object storage system minios are deployed to the shared cloud, respectively, the first access information of the object storage system minios and the second access information of the metadata storage system rediss are acquired for subsequent integration with the juicefs. The first access information of the object storage system minio comprises a name of each storage bucket socket, a first connection address, a first login user name and a first login password. The second access information of the metadata storage system redis includes a second login user name, a second login password, and a second connection address of the metadata storage system redis.

After the first access information of the object storage system minio and the second access information of the metadata storage system redis are acquired, an installation package of the distributed system juicefs is acquired, and the distributed system juicefs is deployed based on the installation package. Meanwhile, a first operation instruction aiming at the distributed system juicefs is constructed based on the first access information of the mini and the second access information of the redis, and the first operation instruction is executed to enable the distributed system juicefs to be respectively connected with the metadata storage system redis and the object storage system mini. Specifically, the installation package of the juicefs, i.e., sudo install juicefs/usr/local/bin, is downloaded, and after decompression operation, i.e., tar-zxvf juicefs-1.1.0-linux-amd64.tar.gz, is performed on the installation package, the relevant operation instruction of the juicefs can be executed. Specifically, a first operation instruction may be executed to construct a juicefs and integrate the juicefs with redis and minio respectively, where in the first operation instruction may define a storage as a type of object storage, in this scheme, minio is selected as an object storage system, socket refers to a connection address (web address) of a minio distributed object storage system, access-key and secret-key refer to login user names and passwords of storage buckets created in different object storage systems respectively, and { socket.name } refers to a name (iot _socket) of a storage bucket created in an object storage system, {user}:/>{ password } and +.>{ redis.ip: redis.port } refers to the login user name and password of redis, and the login addresses ip and port of redis, respectively. After the first operation instruction is executed on the servers sharing the cloud, the juicefs distributed file system is built, so that data can be written into the juicefs distributed file system. Meanwhile, the connection information and other metadata information of each storage bucket of the juicefs and the minio are written into a corresponding library table in the redis for storage, and when all follow-up operations of the minio with the object storage system occur, the corresponding connection information and the corresponding relation information are required to be searched from the metadata database of the redis, and a connection relation is established with the minio with the object storage system based on the connection information and the corresponding relation information.

As shown in fig. 4, a flow diagram of a data analysis system is provided. The data analysis system further comprises a message queue, which may refer to Kafka, which is a high throughput distributed publish-subscribe message system, which can process all action stream data of a user in a website, and provide real-time messages through a cluster, and each message published to the Kafka cluster has a category, namely Topic, also referred to as a Topic, and the Topic can be freely set. In the technical scheme, a plurality of topics can be set in Kafka, wherein each topic can be used for storing the Internet of things data of the same type of engineering equipment calculated by a preset calculation engine Flink so as to enable the Internet of things system of the terminal to carry out related data application. Specifically, a file system connector data reading functional component provided by a link calculation engine is selected to read data files in a mounting catalog/opt/jfs in a private storage system in a local private cloud into the link of the calculation engine in real time, various operators of the link are called through programming to process the extracted internet of things data in real time, the processed internet of things data is sent to a topic appointed by a specific message queue according to the type of engineering equipment, and accordingly an internet of things system of a terminal can read analyzed internet of things data in a kafka specific subject in real time through an api interface to apply relevant data.

In the technical scheme, the container arrangement platform can refer to K3s, and can only remotely manage the micro-service application of the edge terminal through a lightweight container management tool such as K3s at present, so that the resource consumption of the edge terminal is further reduced, and meanwhile, the functions of remote management, network communication and the like are realized. The K3s is characterized by the simplicity, is packaged and deployed as a single binary file (about 100 MB), can be successfully installed in a few seconds to obtain a fully mature Kubernetes cluster, has the installation experience as simple as running a script on each node of the cluster, and is very suitable for installation and deployment at the edge. Meanwhile, the K3s binary is a self-sufficient encapsulation entity that runs almost all components of the Kubernetes cluster, including the API server, the scheduler, and the controller. Typically, each K3s installation includes the operation of the control plane, kubelet and contained, which can support the K3s workload. Meanwhile, a special worker node only running kubelet agent and a con-tainerd runtime can be added to schedule and manage the K3s pod life cycle. In the K3s cluster, a node running the control plane component and kubelet is called a service node server, a node only running the kubelet is called a proxy node agent, and the service node server and the proxy node agent are provided with one kubelaxy when the K3s pod runs, so that the tunnel and the network traffic of the whole K3s cluster are managed.

In the technical scheme, when the K3s cluster is deployed on the Linux operation system of the communication box at the edge end of the engineering equipment for data acquisition, a service node server and a proxy node agent required by the operation of the K3s cluster are deployed first. Specifically, when the container orchestration platform K3s is deployed to the edge, a deployment script of the container orchestration platform K3s needs to be acquired first, a second operation instruction of the service node server is constructed based on the deployment script, and the second operation instruction is executed to deploy the service node server to the edge. Specifically, a conventional Linux distribution board can be automatically deployed into a server node by using a deployment script provided by a K3s official, deployment of a service node server can be completed by executing a second operation instruction of Curl-sfL https:// get.k3s.io|sh-, after the service node server deployment is completed, K3s services can be automatically started, and tools such as kubectl and the like can be installed to an edge end together. Running a script at the edge: sudo kubectl get nodes the running state of the service node server is checked, if the control-plane and master service roles can be seen and the state of the service node server is ready, it is indicated that the service node server has been deployed successfully to the edge.

After the service node server is deployed, a token of the service node server is obtained, a third operation instruction of the agent node agent is constructed based on the token of the service node, and the third operation instruction is executed to deploy the agent node to the edge. Specifically, execute command: the sudo-u root cat/var/lib/random/k 3s/server/node-token can acquire the token node-token of the service node server and store the value of the node-token. Meanwhile, based on the node-token, a third operation instruction of the agent node agent may be constructed, where the third operation instruction may refer to: curl-sfL https:// get.k3s.io|K3S_URL=http:/{ip}:/>{port} K3S_TOKEN=/>{ node-toekn } sh-, wherein +.>{ ip } is the ip address of the edge box linux system, +.>{ port } is a service port of the service node server, and may default to 6443, { port }>{ node-toekn } is the token node-token of the service node server. After the execution of the operation instruction is completed, executing the command: sudo kubectl get nodes, the running state of the agent node agent can be checked, and when the state is ready, the agent node agent is successfully deployed. Under the condition that the operation states of the service node and the proxy node are detected to be opened, the container arrangement platform is determined to be deployed, and thus the installation and deployment of the K3s cluster at the edge end of the engineering equipment are completed.

In the technical scheme, the container arrangement platform may refer to K3s, the container interface service of the distributed system may refer to container interface service juicefs CSI Driver of the distributed system juicefs, the container interface service of the juicefs is a series of interfaces for the containers running in K3s or K8s to interact with external systems, specifically, the external systems interact with the containers in the K3s cluster through the api server of the K3s, which means that data in the containers may be persisted into the external systems outside the containers, or the containers may read data in the external systems. The container arrangement platform K3s may provide containers for the container interface service juicefs CSI Driver of the distributed system and the preset internet of things data collection application app, so that the container interface service juicefs CSI Driver of the distributed system and the preset internet of things data collection application app are normally operated at the edge end, thereby providing remote management and remote scheduling functions for internet of things data. Specifically, when deploying juicefs CSI Driver to the containerization platform K3s, a first containerized deployment script file for the container interface service juicefs CSI Driver of the distributed system needs to be acquired first, and if a first operation instruction input by a user through the containerization platform K3s is acquired, the first containerized deployment script file is executed according to the first operation instruction, so that the container interface service juicefs CSI Driver of the distributed system is deployed to the containerization platform K3s. The first containerized deployment script may refer to: kubecl apply-fhttps:// raw.gikubusercontent.com/juicedata/juicefs-csi-driver/master/depoy/8 s.yaml, and executing the script, the installation of the container service interface juicefs CSI Driver of juicefs at the edge end K3s cluster can be completed.

After the container interface service juicefs CSI Driver of the distributed system is successfully deployed into the container orchestration platform K3s, a storage class of the distributed system juicefs needs to be created in the container orchestration platform K3s, so that the container interface service juicefs CSI Drive at the edge end can map the internet of things data acquired in real time into the object storage system minio in the common cloud for persistent storage based on the storage class. Wherein, the storage class may refer to a Storageclass. Specifically, a key secret for the storage class Storageclass needs to be defined before the storage class Storageclass is created. Specifically, when defining the key secret, a second containerized deployment script file of the key secret needs to be defined first, and a second connection address of the metadata storage system redis and first access information of the object storage system minio are added into the second containerized deployment script file. The second containerized deployment script file may refer to a juicefs-sc-secret.yaml, in which an address of a metadata storage system redisof the distributed system juicefs and a network address url of an object storage system minio, a storage path directory name under each bucket (iot-bucket) in the minio, a login name access-key (user name) and a secret-key (password) of each bucket are specified. After the definition of the second containerized deployment script file is completed, under the condition that a second operation instruction input by a user through the containerized platform K3s is acquired, executing the second containerized deployment script file juicefs-sc-secret.yaml according to the second operation instruction to deploy the key secret to the containerized platform K3s. Specifically, instructions may be executed to: kubecl apply-f juicefs-sc-secret.yaml can create the key secret.

After creating the key secret of the storage class Storageclass, a third containerized deployment script file of the storage class Storageclass of the distributed system is defined, wherein the third containerized deployment script file includes the name of the storage class, the plug-in name, and the name of the key. The third containerized deployment script file may refer to a juicefs-sc.yaml in which the name of Storageclass, juicefs-sc, provisioner, is specified: the plugin name csi.juicefs.com, storageclass requires associated secret information, where the name juicefs-sc-secret of the secret is only required to be configured. After the definition of the third containerized deployment script is finished, under the condition that a third operation instruction input by a user through the containerized platform K3s is acquired, executing the third containerized deployment script file according to the third operation instruction so as to deploy the storage class Storageclass of the distributed system to the containerized platform K3s. Specifically, instructions may be executed to: the installation and deployment of the storageClass in the K3S cluster can be completed by kuectl application-fjuicefs-sc.yaml. After the storageClass is installed, the edge can persist the internet of things data to the object storage system minio through the storageClass.

After the container interface service juicefs CSI Driver of the distributed system and the storage class of the distributed system are successfully deployed in the K3s container at the edge, the preset internet of things data collection application app can be deployed in the K3s container. Specifically, when the preset internet of things data collection application app is deployed into the K3s container, a fourth containerized deployment script file of the preset internet of things data collection application app needs to be defined first, wherein the fourth containerized deployment script file comprises an image file, a service port, a data persistence configuration file and a PVC configuration file of the preset internet of things data collection application app, and a storage class of the distributed system is added into the PVC configuration file. Specifically, the fourth containerized deployment script file may refer to data-collect-juicefs.yaml in which the container image of the app is to be specified, the service port, the data persistence configuration, i.e., the data mount directory within the container, and the PVC configuration mapped to the persistence system in which we are to specify the aforementioned StorageClass. After the fourth containerized deployment script is configured, under the condition that a fourth operation instruction input by a user through the containerized platform K3s is acquired, executing a fourth containerized deployment script file according to the fourth operation instruction so as to deploy the preset Internet of things data acquisition application app to the containerized platform K3s. Specifically, instructions may be executed to: the deployment of the data acquisition micro-application app in the edge end K3s cluster can be completed by kubecl application-fdata-collection-juicefs.yaml. Through the deployment, integration of the juicefs and the edge-end data acquisition micro-application app is also realized, namely, the target storage system juicefs sharing the cloud is used as persistent storage of the edge-end data acquisition application app.

In an embodiment of the present application, the preset internet of things data acquisition application is further configured to: acquiring Internet of things data acquired by a sensor on engineering equipment in real time, and preprocessing the Internet of things data; and establishing connection between the container interface service based on the distributed system and the distributed system sharing the cloud, and mapping the preprocessed internet of things data to the object storage system in real time through the storage class for persistent storage.

In the technical scheme, the preset internet of things data acquisition application app can be used for collecting internet of things data on engineering equipment in a centralized manner, specifically, the preset internet of things data acquisition application app can acquire the internet of things data acquired by a sensor on the engineering equipment in real time, preprocesses the internet of things data, establishes connection with a distributed system juicefs based on a container interface service juicefs CSI Driver of the distributed system, and maps the preprocessed internet of things data to an object storage system minio in real time through a storage type Storageclass for persistent storage. Specifically, the data acquisition service micro-application app is installed and deployed in the edge cloud native k3s cluster, and the juicefs and the app are successfully integrated. When the micro application app monitors that the PLC bus has data transmission through an interface after engineering equipment starts working, the data acquisition service can write the data into a specific directory in a container where the app is located after the back end performs simple processing on the data, the directory can be mapped to a specific directory of juicefs of a distributed file system located in a common cloud through the action of a storageClass, and the data of the Internet of things in the container under the directory can be persisted into the juicefs distributed file system located in the cloud and finally stored in a minio. In this process, the storageClass interacts with the juicefs distributed file system with the common cloud mainly by means of the juicefs client encapsulated in the container interface service juicefs CSI Driver, so that it is necessary to ensure that the network between the edge and the cloud juicefs is smooth. In this technical solution, it is assumed that all edge boxes have 5G communication modules. After equipment starts, all the internet of things data are written into the catalog file of the app container in real time, the process is a logical process, the internet of things data are not really written into the app container catalog, but are transmitted to the juicefs of the common cloud through the mapping action of the starageclass and the real-time transmission of the juicefs client in real time through the 5G network, the internet of things data are stored in the cloud in a lasting mode, the edge end does not store any data, and the storage pressure of the edge end is greatly reduced.

And after the data of the Internet of things are acquired in real time from the juicefs in the common cloud, the data are stored in the common cloud in a centralized way. If data analysis is to be performed in the private cloud environment, in the technical scheme, unified storage and unified consumption of the data of the Internet of things can be realized through technical innovation. Specifically, a client-side juicefs client of juicefs is installed on a Linux machine of a private cloud end, so that a private storage system data directory file in the private cloud end is installed into a data directory file of an object storage system in a public cloud end through the juicefs client, and internet of things data in the public cloud end is synchronized into the private storage system in real time for data analysis in the private cloud end. Specifically, when deploying the client-side juicefs client into the private cloud, the installation script of the client-side juicefs client of the distributed system needs to be acquired first, and under the condition that the fifth running instruction for the installation script is acquired, the installation script is executed, so that the client-side juicefs client of the distributed system is deployed into the private cloud. The fifth execution instruction may refer to: the script-sSL https:// d.juicefs.com/install|sh, after the script is run, the juicefs client will install and complete, execute the command: the juicefs-hellp can verify whether the juicefs client is installed correctly.

After the client of the juicefs is successfully deployed, the connection between the distributed system juicefs and the private cloud is established based on the client of the distributed system juicefs, and a mounting script for the distributed system juicefs is constructed, wherein the mounting script comprises second access information of a metadata storage system redis. Specifically, the mounting script may include{user}，/>{password}，/>{redis.ip}，/>{ redis. Port } is the connection information of the metadata system redis of the juicefs deployed in the shared cloud, and comprises the login user name, the secret, the ip and the port of the redis library, the opt/jfs in the mount script is the data directory in the local private cloud machine, and the cache-dir and the cache-size are the data cache purposesRecord and data buffer size for data acceleration.

After the definition of the mounting script is finished, the mounting script is executed to mount the first data file catalog in the object storage system minio to the second data file catalog in the private storage system so as to synchronize the internet of things data stored in each storage bucket in the object storage system minio to the private storage system in real time for storage. Specifically, after the creation of the juicefs of the common cloud is completed, information including the object storage key and the like is completely recorded in the redis. The juicefs client side juicefs client on the local private cloud machine can mount a file system for reading and writing juicefs only by having the redis database address, the user name and the password information. Therefore, the local juicefs client has no local configuration file, and the configuration file of the juicefs client sharing the cloud can be read and written only by executing the mounting script. The data and metadata of the juicefs file system are stored in the cloud service based on the network, so that the file system can be simultaneously mounted on any computer on which the juicefs client is mounted for sharing reading and writing, and when multiple clients are simultaneously mounted for reading and writing the same file system, the juicefs provides a close-to-open consistency guarantee, namely when two or more clients simultaneously read and write the same file (shared cloud), modification of the client A (edge end) cannot be seen immediately at the client B (local private cloud). However, once this file is written to and closed at client a, re-opening the file at any one of clients B may ensure that the newly written data is accessed. Based on the method, the data of the Internet of things which is written into the shared cloud end by the edge end can be seen in real time by locally mounting the catalog/opt/jfs.

In the embodiment of the present application, the private cloud is further used to: under the condition that a control instruction aiming at engineering equipment is acquired, caching the control instruction to a theme corresponding to the engineering equipment in a message queue; calculating control data cached in the theme based on a preset calculation engine, transmitting the calculated control data to a second data file directory in the private storage system, and mounting the control data to a first data file directory in the object storage system based on a client of the distributed system; and issuing control data to a preset data acquisition application of the Internet of things through a container interface service based on the distributed system by the shared cloud so as to control engineering equipment to execute corresponding operations.

In the technical scheme, in the data interaction process of the internet of things, the process of issuing data such as firmware program parameter configuration, firmware upgrading and control instructions of engineering equipment is generally involved. Therefore, when the private cloud acquires the control instruction for the engineering equipment, the control instruction is cached to the Topic corresponding to the engineering equipment in the message queue kafka, the control data cached to the Topic is calculated based on the preset calculation engine Flink, so that the calculated control data is transmitted to the second data file directory in the private storage system minio, the control data is mounted to the first data file directory in the object storage system minio based on the client-side juicefs client of the distributed system, and finally the control data is issued to the preset internet of things data acquisition application app through the shared cloud based on the container interface service juicefs CSI Driver of the distributed system, so that the engineering equipment is controlled to execute corresponding operation. In the technical scheme, data such as parameter configuration, firmware upgrading programs, control instructions and the like are issued to specified topic topics of different kafka message queues through a terminal Internet of things system, instruction type data in the kafka specified topic topics is consumed based on a Flink computing engine, and the downlink data are written into specific files of a local mount directory/opt/jfs through a FileSystemConnector connection tool. After that, through the real-time synchronization process of the cloud side end of the juicefs, the edge application app obtains instruction data, configuration parameters and firmware upgrading program data issued by the terminal internet of things system from a corresponding catalog in an edge container, and then the functions of instruction control, parameter configuration, firmware upgrading and the like are completed at the edge end, and specifically, the downlink process of the instruction data can be shown as shown in fig. 4. By the advanced flow design and technical innovation, real-time synchronous acquisition of mass internet of things data of engineering equipment, real-time data analysis, real-time transmission of downlink data such as instructions, parameter configuration and the like can be realized.

In the technical scheme, the application of the cloud native technology at the edge end is realized by adopting innovative flow design and technical innovation, and the data acquisition application and the juicefs container interface service are integrated in the container at the edge end by implementing the K3S container technology at the edge end, so that the real-time acquisition and the synchronous transmission of mass Internet of things data at the edge end to the shared cloud end are realized. By constructing the juicefs distributed storage system on the shared cloud, selecting the network database rediss as metadata storage of juicefs, and simultaneously adopting the distributed object storage system minio as data file storage of juicefs, a set of reliable and high-performance cloud distributed data storage system based on the shared cloud is constructed, so that the centralized storage of the edge data on the shared cloud is realized, and the data storage and the use cost are effectively reduced. By carrying out the file system mounting function of the juicefs on the private cloud server, the read-write operation of the private cloud on the data in the shared cloud juicefs is realized, the effect that the operation on the data files in the local file system is equivalent to the operation on the data files in the shared cloud juicefs is achieved, the repeated storage of the data of the Internet of things is avoided, and the problem of inconsistent data is reduced. The data files in the private cloud are processed in real time by using the Flink distributed computing engine, so that the requirement of the Internet of things platform on the Internet of things data is met. The mechanism of simultaneously mounting, reading and writing the same file system based on the multiple clients such as juicefs and further realizing data synchronization is realized, the functions of sending downstream data such as parameter configuration, instruction control, upgrading programs and the like to the edge equipment through the Internet of things platform are realized, and the real-time bidirectional synchronization function of cloud side data is realized. By means of the technical scheme, complex flow design of traditional Internet of things data acquisition is greatly simplified, technical input cost is lower, customized function development workload is reduced, and Internet of things data acquisition cost is reduced. Meanwhile, the data acquisition efficiency of the Internet of things is higher, more edge devices can be simultaneously accessed, and the data throughput is greatly improved. According to the scheme, all data of the edge end are stored in the shared cloud end in a centralized mode, the problems that data quality is poor, data inconsistency is serious due to repeated storage of the data are effectively avoided, data storage cost is greatly reduced, and data reliability is improved. Through the technical design based on the data collaboration, the operations of instruction control, parameter configuration and the like of engineering equipment are realized.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.

The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. that fall within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.

Claims

1. A processing system for internet of things data, the processing system comprising:

the edge end comprises a container arrangement platform, a container interface service of a distributed system and a preset internet of things data acquisition application, wherein the container arrangement platform is used for providing a container for the container interface service of the distributed system and the preset internet of things data acquisition application; the preset internet of things data acquisition application is used for acquiring internet of things data of engineering equipment in real time; the container interface service of the distributed system is used for establishing connection between the edge end and the common cloud end and mapping the internet of things data to the common cloud end in real time;

The shared cloud comprises the distributed system, and the distributed system is used for receiving the internet of things data sent by the edge end in real time and performing persistent storage;

the private cloud comprises a client of the distributed system, a private storage system and a data analysis system, wherein the client of the distributed system is used for establishing connection between the distributed system and the private cloud so as to synchronize the internet of things data stored in the distributed system to the private storage system in real time for storage; the data analysis system is used for extracting target internet of things data corresponding to the calculation task from the private storage system under the condition that the private cloud receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data.

2. The processing system for internet of things data according to claim 1, wherein the common cloud further comprises:

the object storage system is connected with the distributed system and comprises a plurality of storage barrels, and each storage barrel is used for storing the internet of things data of the same type of engineering equipment;

the metadata storage system is connected with the distributed system and is used for storing metadata information in the object storage system, the metadata information at least comprises a name of each storage barrel, a first connection address, a first login user name, a first login password and first data file directory information in each storage barrel, and the metadata storage system is further used for providing a first connection address, a first login user name and a first login password of a corresponding target storage barrel according to engineering equipment to which the Internet of things data belongs under the condition that the Internet of things data is received by the distributed system, so that connection is established between the metadata storage system and the target storage barrel and the Internet of things data is stored in the target storage barrel.

3. The processing system for internet of things data according to claim 2, wherein the common cloud is further configured to:

deploying the metadata storage system and the object storage system respectively;

respectively acquiring first access information of the object storage system and second access information of the metadata storage system, wherein the first access information comprises a name of each storage barrel, a first connection address, a first login user name and a first login password, and the second access information comprises a second login user name, a second login password and a second connection address of the metadata storage system;

acquiring an installation package of the distributed system, and deploying the distributed system based on the installation package;

constructing a first operation instruction for the distributed system based on the first access information and the second access information;

executing the first operation instruction to enable the distributed system to be connected with the metadata storage system and the object storage system respectively.

4. The processing system for internet of things data according to claim 1, wherein the data analysis system further comprises:

The message queue comprises a plurality of topics, and each topic is used for storing the internet of things data of the same type of engineering equipment calculated by the preset calculation engine.

5. The processing system for internet of things data according to claim 1, wherein the edge is further configured to:

acquiring a deployment script of the container arrangement platform, and constructing a second operation instruction of a service node based on the deployment script;

executing the second operation instruction to deploy the service node to the edge end;

obtaining a token of the service node, and constructing a third operation instruction of the proxy node based on the token of the service node;

executing the third operation instruction to deploy the proxy node to the edge end;

and under the condition that the operation states of the service node and the proxy node are detected to be opened, determining that the container arrangement platform is completely deployed.

6. The processing system for internet of things data according to claim 1, wherein the container orchestration platform is further configured to:

acquiring a first containerized deployment script file for a container interface service of the distributed system;

and under the condition that a first operation instruction input by a user through the container arrangement platform is acquired, executing the first containerized deployment script file according to the first operation instruction so as to deploy the container interface service of the distributed system to the container arrangement platform.

7. The processing system for internet of things data according to claim 6, wherein the container orchestration platform is further configured to:

defining a second containerized deployment script file of a key, and adding a second connection address of a metadata storage system and first access information of an object storage system into the second containerized deployment script file;

executing the second containerized deployment script file according to the second running instruction under the condition that the second running instruction input by the user through the containerized platform is acquired, so as to deploy the key to the containerized platform;

a third containerized deployment script file defining a storage class of the distributed system, wherein the third containerized deployment script file includes a name of the storage class, a plug-in name, and a name of the key;

and under the condition that a third running instruction input by the user through the container arrangement platform is acquired, executing the third containerized deployment script file according to the third running instruction so as to deploy the storage class of the distributed system to the container arrangement platform.

8. The processing system for internet of things data according to claim 7, wherein the edge is further configured to:

Defining a fourth containerized deployment script file of the preset internet of things data acquisition application, wherein the fourth containerized deployment script file comprises an image file, a service port, a data persistence configuration file and a PVC configuration file of the preset internet of things data acquisition application;

adding the storage class to the PVC configuration file;

and under the condition that a fourth operation instruction input by the user through the container arrangement platform is acquired, executing the fourth containerized deployment script file according to the fourth operation instruction so as to deploy the preset Internet of things data acquisition application to the container arrangement platform.

9. The processing system for internet of things data according to claim 8, wherein the preset internet of things data collection application is further configured to:

acquiring Internet of things data acquired by a sensor on the engineering equipment in real time, and preprocessing the Internet of things data;

and establishing connection with the distributed system based on the container interface service of the distributed system, and mapping the preprocessed internet of things data to the object storage system in real time through the storage class for persistent storage.

10. The processing system for internet of things data according to claim 1, wherein the private cloud is further configured to:

acquiring an installation script of a client of the distributed system;

executing the installation script under the condition that a fifth running instruction aiming at the installation script is acquired, so as to deploy the client of the distributed system into the private cloud;

establishing connection between the distributed system and the private cloud based on a client of the distributed system;

constructing a mounting script for the distributed system, wherein the mounting script comprises second access information of a metadata storage system;

and executing the mounting script to mount a first data file catalog in the object storage system to a second data file catalog in the private storage system so as to synchronize the internet of things data stored in each storage bucket in the object storage system to the private storage system in real time for storage.

11. The processing system for internet of things data according to claim 10, wherein the private cloud is further configured to:

under the condition that a control instruction aiming at the engineering equipment is acquired, caching the control instruction to a theme corresponding to the engineering equipment in a message queue;

Calculating control data cached in the subject based on the preset calculation engine, transmitting the calculated control data to a second data file directory in the private storage system, and mounting the control data to a first data file directory in the object storage system based on a client of the distributed system;

and issuing the control data to the preset internet of things data acquisition application through the shared cloud based on the container interface service of the distributed system so as to control the engineering equipment to execute corresponding operation.