CN117743471B

CN117743471B - Processing system for data of Internet of things

Info

Publication number: CN117743471B
Application number: CN202410168478.9A
Authority: CN
Inventors: 路培杰; 杨辉; 周志忠; 刘文虎; 邹晨阳
Original assignee: Zhongke Yungu Technology Co Ltd
Current assignee: Zhongke Yungu Technology Co Ltd
Priority date: 2024-02-06
Filing date: 2024-02-06
Publication date: 2024-05-07
Anticipated expiration: 2044-02-06
Also published as: CN117743471A

Abstract

The embodiment of the application provides a processing system for data of the Internet of things. Comprising the following steps: the edge end comprises a container arrangement platform which is used for providing containers for container interface services of the distributed system and preset Internet of things data acquisition applications, wherein the preset Internet of things data acquisition applications are used for acquiring Internet of things data of engineering equipment in real time, and the container interface services of the distributed system are used for mapping the Internet of things data to a common cloud; the shared cloud comprises a distributed system which is used for receiving the data of the Internet of things and performing persistent storage; the private cloud end comprises a client end of the distributed system, wherein the client end of the distributed system is used for synchronizing the internet of things data stored in the distributed system to the private storage system in real time for storage, and the data analysis system is used for extracting target internet of things data corresponding to a calculation task from the private storage system under the condition that the private cloud end receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data.

Description

Processing system for data of Internet of things

Technical Field

The application relates to the technical field of industrial Internet of things, in particular to a processing system for Internet of things data.

Background

With the development of emerging technologies such as big data, cloud computing, internet of things, blockchain, artificial intelligence, 5G communication and the like, all things are interconnected, and man-machine interaction has become a very common application scene at present. The universal interconnection is that all intelligent devices located at the edge end can be required to be connected to the internet, data of the devices can be collected in real time, and meanwhile interaction data information can be required to be sent to the edge end through the cloud. At present, the real-time acquisition of mass internet of things data is still a very complex process with very high technical requirements. Particularly in the field of industrial Internet, industrial equipment has complex physical composition, high intelligent level and various operation conditions, a large number of sensors with different functions acquire the IOT data of the equipment in real time, the IOT data acquired in the process is required to be transmitted to a cloud platform in real time by a specific encryption transmission protocol, and data analysis, real-time decision making, feedback and the like are carried out at the cloud.

In the prior art, the acquired IOT data of all sensors needs to be summarized through a PLC bus located at a device side and then transmitted to an edge box, and the edge box encrypts the data according to a specific transmission protocol and then transmits the data to a cloud through a 5G network. Because of the large number of networked devices, it is necessary to load balance IOT data of different devices according to IP through an LVS load balancer and then transmit the IOT data to different cloud gateways. The cloud gateway has different types of IOT gateways according to different transmission protocols, and each gateway is customized and developed according to a specific protocol and is used for processing IOT data of IOT data transmission equipment adopting the specific protocol. The IOT data is sent to a message queue kafka of the cloud after preliminary processing at the IOT gateway side, is stored in a private cloud in a centralized mode after processing of a real-time computing engine Flink, and is used by an Internet of things platform. The method has complex flow, and not only results in longer acquisition and transmission flow of the IOT data. And the problem of data throughput exists, when the number of accessed devices is too large and the data peak value is high, the process cannot process huge data volume, and the acquisition of IOT data of mass Internet of things devices can be completed only with high software and hardware cost and labor cost investment.

Disclosure of Invention

The embodiment of the application aims to provide a processing system for internet of things data, which is used for solving the technical defects of low acquisition efficiency and high acquisition cost caused by complex internet of things data acquisition flow in the prior art.

To achieve the above object, a first aspect of the present application provides a processing system for data of the internet of things, the processing system comprising:

The edge end comprises a container arrangement platform, a container interface service of the distributed system and a preset internet of things data acquisition application, wherein the container arrangement platform is used for providing a container for the container interface service of the distributed system and the preset internet of things data acquisition application; the Internet of things data acquisition application is used for acquiring Internet of things data of engineering equipment in real time; the container interface service of the distributed system is used for establishing connection between the edge end and the common cloud end and mapping the data of the Internet of things to the common cloud end in real time;

the shared cloud comprises a distributed system, wherein the distributed system is used for receiving the internet of things data sent by the edge end in real time and performing persistent storage;

The private cloud comprises a client of the distributed system, a private storage system and a data analysis system, wherein the client of the distributed system is used for establishing connection between the distributed system and the private cloud so as to synchronize the Internet of things data stored in the distributed system to the private storage system for storage in real time; the data analysis system is used for extracting target internet of things data corresponding to the calculation task from the private storage system under the condition that the private cloud receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data.

In an embodiment of the present application, the common cloud further includes: the object storage system is connected with the distributed system and comprises a plurality of storage barrels, and each storage barrel is used for storing the internet of things data of the same type of engineering equipment; the metadata storage system is connected with the distributed system and is used for storing metadata information in the object storage system, the metadata information at least comprises a name of each storage bucket, a first connection address, a first login user name, a first login password and first data file directory information in each storage bucket, and the metadata storage system is also used for providing the first connection address, the first login user name and the first login password of a corresponding target storage bucket according to engineering equipment to which the Internet of things data belongs under the condition that the distributed system receives the Internet of things data so as to establish connection with the target storage bucket and store the Internet of things data into the target storage bucket.

In an embodiment of the present application, the common cloud is further used to: respectively deploying a metadata storage system and an object storage system; respectively acquiring first access information of the object storage system and second access information of the metadata storage system, wherein the first access information comprises a name of each storage barrel, a first connection address, a first login user name and a first login password, and the second access information comprises a second login user name, a second login password and a second connection address of the metadata storage system; acquiring an installation package of the distributed system, and deploying the distributed system based on the installation package; constructing a first operation instruction for the distributed system based on the first access information and the second access information; the first operational instructions are executed to cause the distributed system to establish a connection with the metadata storage system and the object storage system, respectively.

In an embodiment of the application, the data analysis system further comprises: the message queue comprises a plurality of topics, and each topic is used for storing the internet of things data of the same type of engineering equipment calculated by the preset calculation engine.

In an embodiment of the present application, the edge is further configured to: acquiring a deployment script of the container programming platform, and constructing a second operation instruction of the service node based on the deployment script; executing a second operation instruction to deploy the service node to the edge end; obtaining a token of the service node, and constructing a third operation instruction of the proxy node based on the token of the service node; executing a third operation instruction to deploy the proxy node to the edge end; and under the condition that the operation states of the service node and the proxy node are detected to be opened, determining that the container arranging platform is completely deployed.

In an embodiment of the application, the container orchestration platform is further for: acquiring a first containerized deployment script file for a container interface service of a distributed system; and under the condition that a first operation instruction input by a user through the container arrangement platform is acquired, executing the first containerized deployment script file according to the first operation instruction so as to deploy the container interface service of the distributed system to the container arrangement platform.

In an embodiment of the application, the container orchestration platform is further for: defining a second containerized deployment script file of the key, and adding a second connection address of the metadata storage system and first access information of the object storage system to the second containerized deployment script file; under the condition that a second operation instruction input by a user through the container arrangement platform is acquired, executing a second containerized deployment script file according to the second operation instruction so as to deploy the secret key to the container arrangement platform; a third containerized deployment script file defining a storage class of the distributed system, wherein the third containerized deployment script file includes a name of the storage class, a plug-in name, and a name of a key; and under the condition that a third running instruction input by a user through the container arrangement platform is acquired, executing a third containerized deployment script file according to the third running instruction so as to deploy the storage class of the distributed system to the container arrangement platform.

In an embodiment of the present application, the edge is further configured to: defining a fourth containerized deployment script file of a preset Internet of things data acquisition application, wherein the fourth containerized deployment script file comprises an image file, a service port, a data persistence configuration file and a PVC configuration file of the preset Internet of things data acquisition application; adding the storage class into the PVC configuration file; and under the condition that a fourth operation instruction input by a user through the container arrangement platform is acquired, executing a fourth containerized deployment script file according to the fourth operation instruction so as to deploy the preset Internet of things data acquisition application to the container arrangement platform.

In an embodiment of the present application, the preset internet of things data acquisition application is further configured to: acquiring Internet of things data acquired by a sensor on engineering equipment in real time, and preprocessing the Internet of things data; the container interface service based on the distributed system establishes connection with the distributed system, and the preprocessed internet of things data is mapped to the object storage system in real time through the storage class for persistent storage.

In an embodiment of the present application, the private cloud is further configured to: acquiring an installation script of a client of a distributed system; executing the installation script under the condition that a fifth running instruction aiming at the installation script is acquired, so as to deploy the client of the distributed system into the private cloud; establishing connection between the distributed system and the private cloud based on a client of the distributed system; constructing a mounting script for the distributed system, wherein the mounting script comprises second access information of the metadata storage system; and executing the mounting script to mount the first data file catalog in the object storage system to the second data file catalog in the private storage system so as to synchronize the internet of things data stored in each storage bucket in the object storage system to the private storage system for storage in real time.

In an embodiment of the present application, the private cloud is further configured to: under the condition that a control instruction aiming at engineering equipment is acquired, caching the control instruction to a theme corresponding to the engineering equipment in a message queue; calculating control data cached in the theme based on a preset calculation engine, transmitting the calculated control data to a second data file directory in the private storage system, and mounting the control data to a first data file directory in the object storage system based on a client of the distributed system; and issuing control data to a preset data acquisition application of the Internet of things through a container interface service of the shared cloud based distributed system so as to control engineering equipment to execute corresponding operations.

According to the technical scheme, the processing system for the data of the Internet of things comprises: the edge end comprises a container arrangement platform which is used for providing containers for container interface services of the distributed system and preset Internet of things data acquisition applications, wherein the preset Internet of things data acquisition applications are used for acquiring Internet of things data of engineering equipment in real time, and the container interface services of the distributed system are used for mapping the Internet of things data to a common cloud; the shared cloud comprises a distributed system which is used for receiving the data of the Internet of things and performing persistent storage; the private cloud end comprises a client end of the distributed system, wherein the client end of the distributed system is used for synchronizing the internet of things data stored in the distributed system to the private storage system in real time for storage, and the data analysis system is used for extracting target internet of things data corresponding to a calculation task from the private storage system under the condition that the private cloud end receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data. According to the method, the data link between engineering equipment, the edge end, the shared cloud end and the private cloud end is opened, real-time synchronization and acquisition of the data of the Internet of things of the engineering equipment are realized based on the edge end, the acquisition flow of the data of the Internet of things is simplified, centralized storage is carried out, the acquisition efficiency of the data of the Internet of things is effectively improved, and the acquisition cost of the data of the Internet of things is effectively reduced.

Additional features and advantages of embodiments of the application will be set forth in the detailed description which follows.

Drawings

The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain, without limitation, the embodiments of the application. In the drawings:

FIG. 1 schematically illustrates a block diagram of a processing system for data of the Internet of things according to an embodiment of the application;

FIG. 2 schematically illustrates a block diagram of yet another processing system for data of the Internet of things, in accordance with an embodiment of the present application;

FIG. 3 schematically illustrates a flow diagram of a processing system for data of the Internet of things according to an embodiment of the application;

fig. 4 schematically shows a flow diagram of a data analysis system according to an embodiment of the application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the detailed description described herein is merely for illustrating and explaining the embodiments of the present application, and is not intended to limit the embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

In addition, if there is a description of "first", "second", etc. in the embodiments of the present application, the description of "first", "second", etc. is for descriptive purposes only and is not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be considered to be absent and not within the scope of protection claimed in the present application.

Fig. 1 schematically shows a block diagram of a processing system for internet of things data according to an embodiment of the application. As shown in fig. 1, an embodiment of the present application provides a processing system for data of the internet of things, where the processing system may include:

The edge 110 comprises a container arrangement platform 112, a container interface service 114 of the distributed system and a preset internet of things data acquisition application 116, wherein the container arrangement platform 112 is used for providing containers for the container interface service 114 of the distributed system and the preset internet of things data acquisition application 116; the preset internet of things data acquisition application 116 is used for acquiring internet of things data of engineering equipment in real time; the container interface service 114 of the distributed system is configured to establish a connection between the edge 110 and the common cloud 120, and map the internet of things data to the common cloud 120 in real time.

The edge is typically between the terminal and the cloud, gathering data on the side near the source of the data. In the technical scheme, the edge end can be arranged between the shared cloud end and engineering equipment and can be used for collecting the internet of things data of the engineering equipment. Specifically, the edge may include a container orchestration platform, a container interface service of a distributed system, and a preset internet of things data collection application. The container arrangement platform can refer to K3s, and can only remotely manage the micro-service application of the edge end through a lightweight container management tool such as K3s at present due to the limitation of computer hardware and related technical level of the edge end, so that the resource consumption of the edge end is further reduced, and meanwhile, the corresponding functions of remote management and network access are realized. K3s is a published version of Kubernetes, which is highly optimized for edges. Although K3s is a simplified and mini version of Kubernetes, the consistency and function of the API are not affected, from kubectl to Helm to Kubernetes, almost all tools of the cloud native ecosystem can be seamlessly interfaced with K3s, and can be deployed in the production environment of K3 s. The K3s is characterized by the simplicity, is packaged and deployed as a single binary file (about 100 MB), can be successfully installed in a few seconds to obtain a fully mature Kubernetes cluster, has the installation experience as simple as running a script on each node of the cluster, and is very suitable for installation and deployment at the edge. Meanwhile, the K3s binary file is a self-sufficient packaging entity that runs almost all components of the Kubernetes cluster, including API SERVER, schedulers, and controllers. Specifically, in the technical scheme, after the container arrangement platform K3s is installed at the edge end, the container arrangement platform K3s can provide containers for the container interface service of the distributed system and the preset internet of things data acquisition application, so that the container interface service of the distributed system and the preset internet of things data acquisition application normally run at the edge end, and remote management and remote scheduling functions are provided for the internet of things data.

In the technical scheme, a java language code is selected to develop a lightweight micro-application app for data acquisition, so that the preset internet of things data acquisition application can refer to the lightweight micro-application app for data acquisition. After the engineering equipment at the edge starts to work, all the sensors positioned on the engineering equipment start to collect the internet of things data of the engineering equipment. When the micro-application app monitors that the PLC bus has the data transmission of the Internet of things through the interface, the data acquisition service can write the data into a specific directory in a container where the micro-application app is located after the data is simply processed by the back end.

The container interface service of the distributed system may refer to the container interface service juicefs CSI DRIVER, juicefs of the distributed system juicefs, which is a series of interfaces for the container running in K3s or K8s to interact with the external system, specifically, the external system interfaces with the container in the K3s cluster through API SERVER of K3s, which means that the data in the container may be persisted into the external system outside the container, or the container may be enabled to read the data in the external system. Meanwhile, the container interface service in the technical scheme can integrate an interface of the client juicefs client of the distributed system juicefs internally, and the edge end can further realize communication with the shared cloud based on the container interface service. Meanwhile, after the internet of things data of the engineering equipment is obtained in real time by the preset internet of things data collection application app, the edge end can map the internet of things data obtained in real time by the preset internet of things data collection application app to the common cloud end based on the container interface service juicefs CSI DRIVER of the distributed system so as to perform persistent storage on the internet of things data through the common cloud end, and therefore the internet of things data can be called by the terminal internet of things platform.

The common cloud 120 includes a distributed system 122, where the distributed system 122 is configured to receive and store the internet of things data sent by the edge 110 in real time.

The public cloud is a public cloud, generally refers to a cloud which can be used and is provided by a third party provider for a user, the public cloud can be generally used through the Internet, the public cloud can be free or low in cost, and the core attribute of the public cloud is shared resource service.

The distributed system can be Juicefs and Juicefs, which is a cloud native design-oriented high-performance distributed system, provides complete POSIX compatibility, can store and access almost all objects to be used as a massive local disk, and can mount read-write on different hosts across platforms and regions at the same time. Therefore, juicefs can be selected to construct unified storage on the common cloud in the technical scheme. Specifically, after the distributed system Juicefs is deployed in the common cloud, the distributed system Juicefs may receive the internet of things data sent by the edge node in real time based on the container interface service juicefs CSI DRIVER of the distributed system, and perform persistent storage on the mapped internet of things data.

The private cloud 130 comprises a client 132 of a distributed system, a private storage system 134 and a data analysis system 136, wherein the client 132 of the distributed system is used for establishing connection between the distributed system 122 and the private cloud 130 so as to synchronize the internet of things data stored in the distributed system 122 to the private storage system 134 in real time for storage; the data analysis system 136 is configured to extract, when the private cloud 130 receives the computing task, target internet of things data corresponding to the computing task from the private storage system 134, calculate the target internet of things data based on a preset computing engine, and output the calculated internet of things data.

The private cloud is a cloud computing infrastructure built inside an enterprise, and can also be understood as a cloud platform deployed inside the enterprise. Various cloud services such as computing resources, storage resources, and network resources may be included in this platform. The enterprise can autonomously deploy application programs, store data, realize allocation and management of resources and the like on the platform, which is equivalent to establishing an enterprise-level cloud platform. The private cloud is different from the public cloud in that cloud resources of the private cloud are all deployed in an enterprise's own data center or other places controlled by the enterprise, and the public cloud is provided by a cloud service provider, so that users can access the cloud resources through the internet. In addition, the private cloud end is usually in a closed network environment and is only opened to users and application programs in enterprises, so that the safety of data can be better ensured.

In this technical solution, the private cloud includes a client of the distributed system, a private storage system, and a data analysis system, where the client of the distributed system may refer to a client juicefs client of the distributed system juicefs, and a client juicefs client may be used to create and mount a file system. Specifically, the client juicefs client of the distributed system may establish a connection between the distributed system juicefs and the private cloud, so that a file system in the private storage system may be mounted to a file system in the distributed system juicefs, so as to synchronize the internet of things data stored in the distributed system juicefs to the private storage system in real time for storage. The preset calculation engine in the data analysis system can be referred to as a flank calculation engine, which is a framework and a distributed processing engine, and can be used for calculating states of unbounded and bounded data, and the flank has the advantages of low delay, high throughput, structural accuracy and good fault tolerance. Specifically, the data analysis system may extract, when the private cloud receives the computing task, target internet of things data corresponding to the computing task from the private storage system, calculate the target internet of things data based on a preset computing engine Flink, and output the calculated internet of things data.

In an embodiment of the present application, as shown in fig. 2, there is provided a block diagram of a processing system for data of the internet of things, in fig. 2, the common cloud further includes: the object storage system 124 is connected with the distributed system 122 and comprises a plurality of storage buckets, and each storage bucket is used for storing the internet of things data of the same type of engineering equipment; the metadata storage system 126 is connected to the distributed system 122, and is configured to store metadata information in the object storage system, where the metadata information includes at least a name of each storage bucket, a first connection address, a first login user name, a first login password, and first data file directory information in each storage bucket, and the metadata storage system is further configured to, when the distributed system receives internet of things data, provide, according to engineering equipment to which the internet of things data belongs, the first connection address, the first login user name, and the first login password of a corresponding target storage bucket, so as to establish connection with the target storage bucket and store the internet of things data in the target storage bucket.

In this technical solution, since the distributed system juicefs adopts a structure in which data and metadata are separately stored, so as to implement a distributed design of a file system, it is necessary to deploy the metadata storage service and the data storage service of juicefs on a common cloud respectively. Specifically, the object storage and metadata storage systems are deployed in a common cloud for providing data storage services and metadata storage services for juicefs, respectively. Wherein, the object storage system may refer to minio, minio being a distributed, multi-copy, reliable and inexpensive object storage system, minio can read and write data with juicefs via s3 protocol. Specifically, the object storage system minio is connected to the distributed system juicefs, where the object storage system minio includes a plurality of bucket bundles, and each bucket bundle may be used to store data of the internet of things of the same type of engineering equipment. Specifically, after the distributed system Juicefs receives the internet of things data sent by the container interface service juicefs CSI DRIVER of the distributed system in real time, the internet of things data of the same type of equipment engineering equipment is transmitted to a corresponding bucket in the object storage system minio for persistent storage.

The metadata storage system can be referred to as redis, which is a remote dictionary service, and is an in-memory database, and has high data reading and writing efficiency. Therefore, redis can be selected as a metadata storage system of the distributed system juicefs, and the use of redis to store metadata can effectively improve data access performance. Specifically, the metadata storage system redis is connected with the distributed system juicefs, the collection of metadata is a behavior of juicefs, no human is needed for collection, and after the distributed system juicefs metadata storage system redis and the distributed system are deployed to a common cloud, juicefs can automatically synchronize metadata storage into the redis for storage. In particular, metadata stored by the metadata storage system redis may refer to metadata information in the storage object storage system minio. The metadata information in minio at least includes a name of each bucket socket, a first connection address, a first login user name, a first login password, and first data file directory information in each bucket socket. Meanwhile, the metadata storage system redis is further configured to provide a first connection address, a first login user name and a first login password of a corresponding target storage bucket socket according to engineering equipment to which the internet of things data belongs, so as to establish connection with the target storage bucket socket and store the internet of things data into the target storage bucket socket when the distributed system juicefs receives the internet of things data.

In an embodiment of the present application, as shown in fig. 3, a flow diagram of a processing system for internet of things data is provided. In fig. 3, the processing system includes an edge device end, a common cloud data center end and a private cloud data center end, wherein a lightweight cloud native technology K3S is introduced at the edge device end, and based on a container environment of the edge end, fusion of the common cloud distributed system juicefs and an edge application app is innovatively realized through a container interface service juicefs CSI DRIVER of the distributed system juicefs. And fully combines the advantages of large bandwidth, high-efficiency storage, good stability and low comprehensive cost of the shared cloud, and realizes the high-efficiency storage and synchronization of mass data of the Internet of things. Meanwhile, the object storage system minio is selected as a data storage service of the distributed system juicefs in the common cloud, and redis is selected as a metadata storage service of the distributed system juicefs. At the private cloud, the file system of the shared cloud distributed system juicefs is read and written by the client Juicefs client of juicefs at the same time, so that the data consistency assurance of the shared cloud and the private cloud is provided by juicefs, the private cloud can quickly acquire corresponding Internet of things data from the distributed system juicefs in the shared cloud, and data analysis processing is performed locally. And because the read-write process of the private cloud end and the shared cloud end is bidirectional, data such as instructions, parameter configuration, upgrade files and the like can be uploaded from the private cloud end to the shared cloud end, and synchronization from the shared cloud end to the edge end can be realized by means of a juicefs multi-end data synchronization mechanism in the shared cloud end, so that the management function of the private cloud end on the edge end engineering equipment is realized.

In the technical scheme, the distributed system may refer to juicefs, the metadata storage system may refer to redis, the object storage system may refer to minio, and because juicefs adopts a framework of data and metadata separate storage, the distributed design of the file system is realized, so that a juicefs metadata storage service redis and a data storage service are required to be deployed on a common cloud respectively, and the data storage selects a distributed object storage minio as a storage system of juicefs. Specifically, prior to deploying distributed system juicefs to a common cloud, object storage system minio and metadata storage system redis need to be deployed to a common remote site. Specifically, the deployment of redis may select a sentinel mode for deployment to ensure the reliability of juicefs metadata services. The redis sentinel mode is different from cluster deployment and is based on a redis master-slave deployment architecture. The nodes are required to be deployed with a sentinel mode, the sentinel mode can monitor whether all redis working nodes are normal, when a Master is in a problem, other nodes lose contact with a main node, so that voting is performed, the Master is considered to be in a problem indeed after the voting is performed in half, a sentinel room is notified, and one Master is selected from Slaves to be a new Master. minio is used as an object storage system following the S3 protocol, and is more convenient to integrate with juicefs, so in the technical scheme, the erasure code of minio is selected to be opened in the deployment process of minio to prevent downtime of a plurality of nodes and bit attenuation bit rot, and a multi-copy mechanism is simultaneously opened to improve the reliability of data storage in minio. minio can log in the Console of minio after deployment, and create a plurality of storage bucket pockets for the data storage of the Internet of things of different types of engineering equipment, and the bucket name is defined as iot _pockets.

After the metadata storage system redis and the object storage system minio are deployed to the common cloud, respectively, the first access information of the object storage system minio and the second access information of the metadata storage system redis are acquired for subsequent integration with juicefs. The first access information of the object storage system minio includes a name of each bucket, a first connection address, a first login user name, and a first login password. The second access information of the metadata storage system redis includes a second login user name, a second login password, and a second connection address of the metadata storage system redis.

After the first access information of the object storage system minio and the second access information of the metadata storage system redis are acquired, an installation package of the distributed system juicefs is acquired, and the distributed system juicefs is deployed based on the installation package. Meanwhile, a first operation instruction for the distributed system juicefs is constructed based on the first access information minio and the second access information of redis, and the first operation instruction is executed to cause the distributed system juicefs to establish connection with the metadata storage system redis and the object storage system minio, respectively. Specifically, the installation package juicefs-1.1.0-linux-amd64.Tar.gz of juicefs is downloaded, and after decompression operation, i.e., tar-zxvf juicefs-1.1.0-linux-amd64.Tar.gz, is performed on the installation package, an installation operation, i.e., sudo install juicefs/usr/local/bin, of juicefs is performed, and then the relevant operation instruction of juicefs can be executed. Specifically, a first operation instruction may be executed to construct juicefs and integrate juicefs with redis and minio, respectively, where in the first operation instruction, a storage may be defined as a type of object storage, minio is selected as an object storage system in this scheme, a socket refers to a connection address (web address) of the minio distributed object storage system, an access-key and a secret-key refer to login user names and passwords of storage buckets created in different object storage systems, respectively, and { socket.name } refers to a name (iot _socket) of a storage bucket created in the object storage system,{user}:/>{ Password } and/>{ Redis.ip: redis.port } refers to the login user name and password of redis, and the login addresses ip and port of redis, respectively. After executing the first operation instruction on the server sharing the cloud, the juicefs distributed file system is established, so that data can be written into the juicefs distributed system. Meanwhile, the connection information and other metadata information of each storage bucket of juicefs and minio are written into the corresponding library table in the redis for storage, and when all subsequent operations with the object storage system minio occur, the corresponding connection information and corresponding relation information need to be searched from the metadata database of the redis, and a connection relation is established with the object storage system minio based on the connection information and the corresponding relation information.

As shown in fig. 4, a flow diagram of a data analysis system is provided. The data analysis system further comprises a message queue, which may refer to Kafka, which is a high throughput distributed publish-subscribe message system, which can process all action stream data of a user in a website, and provide real-time messages through a cluster, and each message published to the Kafka cluster has a category, namely Topic, also referred to as a Topic, and the Topic can be freely set. In the technical scheme, a plurality of topics can be set in Kafka, wherein each topic can be used for storing the Internet of things data of the same type of engineering equipment calculated by a preset calculation engine Flink so as to enable the Internet of things system of the terminal to carry out related data application. Specifically, a FileSystemConnector data reading functional component provided by a link calculation engine is selected to read data files in a mounting catalog/opt/jfs in a private storage system in a local private cloud into the link of the calculation engine in real time, various operators of the link are called through programming to process the extracted internet of things data in real time, the processed internet of things data is sent to a topic designated by a specific message queue according to the type of engineering equipment, and therefore the internet of things system of the terminal can read the analyzed internet of things data in a kafka specific theme through an api interface in real time to perform related data application.

In the technical scheme, the container arrangement platform can refer to K3s, and can only remotely manage the micro-service application of the edge terminal through a lightweight container management tool such as K3s at present, so that the resource consumption of the edge terminal is further reduced, and meanwhile, the functions of remote management, network communication and the like are realized. The K3s is characterized by the simplicity, is packaged and deployed as a single binary file (about 100 MB), can be successfully installed in a few seconds to obtain a fully mature Kubernetes cluster, has the installation experience as simple as running a script on each node of the cluster, and is very suitable for installation and deployment at the edge. Meanwhile, the K3s binary file is a self-sufficient packaging entity that runs almost all components of the Kubernetes cluster, including API SERVER, schedulers, and controllers. Typically, each K3s installation includes the operation of control planes, kubelet and containerd, with the operation of control planes, kubelet and containerd being capable of supporting the K3s workload. Meanwhile, a special worker node only running kubelet agent and containerd can be added to schedule and manage the K3s pod life cycle. In the K3s cluster, the node running the control plane component and kubelet is called a service node server, and the node running only kubelet is called a proxy node agent, and the service node server and the proxy node agent are provided with one kubeproxy when the K3s pod runs, so that the tunnel and the network traffic of the whole K3s cluster are managed.

In the technical scheme, when the K3s cluster is deployed on the Linux operation system of the communication box at the edge end of the engineering equipment for data acquisition, a service node server and a proxy node agent required by the operation of the K3s cluster are deployed first. Specifically, when the container orchestration platform K3s is deployed to the edge, a deployment script of the container orchestration platform K3s needs to be acquired first, a second operation instruction of the service node server is constructed based on the deployment script, and the second operation instruction is executed to deploy the service node server to the edge. Specifically, a conventional Linux distribution board can be automatically deployed into a server node by using a deployment script provided by a K3s official, deployment of a service node server can be completed by executing a second operation instruction of CUrl-sfL https:// get.k3s.io|sh-, after the service node server deployment is completed, K3s services can be automatically started, and tools such as kubectl and the like can be installed to an edge end together. Running a script at the edge: sudo kubectl get nodes, checking the running state of the service node server, and if the control-plane and master service roles can be seen and the state of the service node server is ready, indicating that the service node server has been deployed successfully to the edge.

After the service node server is deployed, a token of the service node server is obtained, a third operation instruction of the agent node agent is constructed based on the token of the service node, and the third operation instruction is executed to deploy the agent node to the edge. Specifically, execute command: sudo-u root cat/var/lib/rancher/k 3s/server/node-token can then obtain the token node-token of the service node server and save the value of the node-token. Meanwhile, based on the node-token, a third operation instruction of the agent node agent may be constructed, where the third operation instruction may refer to: curl-sfL https:// get.k3s.io|K3S_URL=http:// for{ip}:/>{port} K3S_TOKEN=/>{ Node-toekn } sh-, wherein/>{ Ip } is the ip address of the edge box linux system,/>{ Port } is a service port of the service node server, and can default to 6443,/>{ Node-toekn } is the token node-token of the service node server. After the execution of the operation instruction is completed, executing the command: sudo kubectl get nodes, the running state of the agent node agent can be checked, and when the running state is ready, the agent node agent is successfully deployed. Under the condition that the operation states of the service node and the proxy node are detected to be opened, the container arrangement platform is determined to be deployed, and thus the installation and deployment of the K3s cluster at the edge end of the engineering equipment are completed.

In this technical solution, the container orchestration platform may refer to K3s, the container interface service of the distributed system may refer to container interface service juicefs CSI DRIVER of the distributed system juicefs, and the container interface service of juicefs is a series of interfaces for interaction between a container running in K3s or K8s and an external system, specifically, interface interaction between the external system and a container in a K3s cluster through API SERVER of K3s, and refer to that data in the container may be persisted to an external system outside the container, or that the container may read data in the external system. The container arrangement platform K3s may provide containers for the container interface service juicefs CSI DRIVER of the distributed system and the preset internet of things data collection application app, so that the container interface service juicefs CSI DRIVER of the distributed system and the preset internet of things data collection application app are normally operated at the edge end, thereby providing remote management and remote scheduling functions for internet of things data. Specifically, when juicefs CSI DRIVER is deployed into the containerization platform K3s, a first containerized deployment script file for the container interface service juicefs CSI DRIVER of the distributed system needs to be acquired first, and if a first operation instruction input by a user through the containerization platform K3s is acquired, the first containerized deployment script file is executed according to the first operation instruction, so that the container interface service juicefs CSI DRIVER of the distributed system is deployed to the containerization platform K3s. The first containerized deployment script may refer to ：kubectl apply -f https://raw.githubusercontent.com/juicedata/juicefs-csi-driver/master/deploy/k8s.yaml, executing the script, and may finish the installation of the container service interface juicefs CSI DRIVER of juicefs on the edge K3s cluster.

After the container interface service juicefs CSI DRIVER of the distributed system is successfully deployed into the container orchestration platform K3s, a storage class of the distributed system juicefs needs to be created in the container orchestration platform K3s, so that the container interface service juicefs CSI Drive at the edge can map the real-time collected internet of things data into the object storage system minio in the common cloud for persistent storage based on the storage class. Wherein, the storage class may refer to StorageClass. Specifically, a key secret for storage class StorageClass is defined prior to creating the storage class StorageClass. Specifically, when defining the key secret, a second containerized deployment script file of the key secret needs to be defined first, and the second connection address of the metadata storage system redis and the first access information of the object storage system minio are added to the second containerized deployment script file. The second containerized deployment script file may refer to juicefs-sc-secret. Yaml, in which the address of the metadata storage system redis of the distributed system juicefs, and the storage path directory name under each bucket (iot-bucket) in the network address url, minio of the object storage system minio, the login name access-key and secret-key (password) of each bucket are specified. After the definition of the second containerized deployment script file is completed, in the case that a second operation instruction input by the user through the containerized platform K3s is acquired, the second containerized deployment script file juicefs-sc-secret is executed according to the second operation instruction to deploy the key secret to the containerized platform K3s. Specifically, instructions may be executed to: kubectl apply-f juicefs-sc-secret. Yaml can create the key secret.

After the key secret of storage class StorageClass is created, a third containerized deployment script file of storage class StorageClass of the distributed system is defined, wherein the third containerized deployment script file includes the name of the storage class, the plug-in name, and the name of the key. The third containerized deployment script file may refer to juicefs-sc.yaml, in which the name juicefs-sc of StorageClass is specified, provisioner: the plugin name csi.juicefs.com, storageClass requires associated secret information, here only the names juicefs-sc-secret of the above-mentioned secrets need to be configured here. After the third containerized deployment script is defined, executing the third containerized deployment script file according to the third running instruction under the condition that the third running instruction input by the user through the containerized platform K3s is acquired, so as to deploy the storage class StorageClass of the distributed system to the containerized platform K3s. Specifically, instructions may be executed to: kuectl apply-fjuicefs-sc.yaml, and the installation and deployment of storageClass in the K3S cluster can be completed. After storageClass is installed, the edge may persist the internet of things data to the object storage system minio through the Storageclass.

After the container interface service juicefs CSI DRIVER of the distributed system and the storage class StorageClass of the distributed system are successfully deployed into the K3s container at the edge, the preset internet of things data collection application app can be deployed into the K3s container. Specifically, when the preset internet of things data collection application app is deployed into the K3s container, a fourth containerized deployment script file of the preset internet of things data collection application app needs to be defined first, wherein the fourth containerized deployment script file includes an image file, a service port, a data persistence configuration file and a PVC configuration file of the preset internet of things data collection application app, and a storage class StorageClass of the distributed system is added into the PVC configuration file. In particular, the fourth containerized deployment script file may refer to data-collect-juicefs.yaml in which the container image of the app is to be specified, the service port, the data persistence configuration, i.e., the data mount directory within the container, and the PVC configuration mapped to the persistence system in which we are to specify StorageClass described above. After the fourth containerized deployment script is configured, under the condition that a fourth operation instruction input by a user through the containerized platform K3s is acquired, executing a fourth containerized deployment script file according to the fourth operation instruction so as to deploy the preset Internet of things data acquisition application app to the containerized platform K3s. Specifically, instructions may be executed to: kubectl apply-f data-collect-juicefs.yaml, the deployment of the data acquisition micro-application app in the edge end K3s cluster can be completed. Through the above deployment, integration of juicefs and the edge data collection micro-application app is also realized at the same time, namely, the object storage system juicefs sharing the cloud is used as the persistent storage of the edge data collection micro-application app.

In an embodiment of the present application, the preset internet of things data acquisition application is further configured to: acquiring Internet of things data acquired by a sensor on engineering equipment in real time, and preprocessing the Internet of things data; and establishing connection between the container interface service based on the distributed system and the distributed system sharing the cloud, and mapping the preprocessed internet of things data to the object storage system in real time through the storage class for persistent storage.

In the technical scheme, the preset internet of things data acquisition application app can be used for intensively acquiring internet of things data on engineering equipment, specifically, the preset internet of things data acquisition application app can acquire the internet of things data acquired by a sensor on the engineering equipment in real time, preprocesses the internet of things data, establishes connection with the distributed system juicefs based on a container interface service juicefs CSI DRIVER of the distributed system, and maps the preprocessed internet of things data to the object storage system minio for persistence storage in real time through the storage class StorageClass. Specifically, the data acquisition service micro-application app is installed and deployed in the edge cloud native k3s cluster, and juicefs is successfully integrated with the app. When the micro application app monitors that the PLC bus has data transmission through an interface after engineering equipment starts working, the data acquisition service can write the data into a specific catalog in a container where the app is located after the back end performs simple processing on the data, the catalog can be mapped to a specific catalog of juicefs of a distributed file system located in a common cloud through storageClass, and then the data of the Internet of things in the container under the catalog can be persisted into a juicefs distributed file system located in the cloud and finally stored in minio. In this process storageClass interacts with the juicefs distributed file system with the common cloud, mainly by means of the juicefs client encapsulated in the container interface service juicefs CSI DRIVER, so it is necessary to ensure that the network between the edge and the cloud juicefs is smooth. In this technical solution, it is assumed that all edge boxes have 5G communication modules. After equipment starts, all Internet of things data are written into a catalog file of an app container in real time, the process is a logical process, the Internet of things data are not really written into the app container catalog, the Internet of things data are transmitted to a shared cloud end juicefs in real time through the mapping function of storageClass and the real-time transmission of juicefs clients, the Internet of things data are stored in the cloud end in a lasting mode, the edge end does not store any data, and the storage pressure of the edge end is greatly reduced.

And after the internet of things data is acquired into juicefs of the common cloud in real time, the internet of things data is stored in the common cloud in a centralized manner. If data analysis is to be performed in the private cloud environment, in the technical scheme, unified storage and unified consumption of the data of the Internet of things can be realized through technical innovation. Specifically, the client juicefs client of juicefs is installed on the Linux machine of the private cloud end first, so that the private storage system data directory file in the private cloud end is installed in the data directory file of the object storage system in the common cloud end through juicefs client, and the internet of things data in the common cloud end is synchronized to the private storage system in real time for data analysis in the private cloud end. Specifically, when the client juicefs client is deployed into the private cloud, the installation script of the client juicefs client of the distributed system needs to be acquired first, and if a fifth running instruction for the installation script is acquired, the installation script is executed, so that the client juicefs client of the distributed system is deployed into the private cloud. The fifth execution instruction may refer to: curl-sSL https:// d.juicefs.com/install|sh, after script execution is completed, juicefs client will install to complete, execute command: juicefs-hellp can then verify juicefs that the client is properly installed.

After successful deployment juicefs of the client, the distributed system based client juicefs client establishes a connection between the distributed system juicefs and the private cloud, and builds a mount script for the distributed system juicefs, where the mount script includes second access information of the metadata storage system redis. Specifically, the mounting script may include{user}，/>{password}，/>{redis.ip}，/>{ Redis. Port } is the connection information of the metadata system redis of juicefs deployed by the shared cloud, and comprises the login user name, the secret, the ip and the port of the redis library, the opt/jfs in the mount script is the data directory in the local private cloud machine, and the cache-dir and the cache-size are the data cache directory and the data cache size, so that data acceleration is performed.

After the definition of the mounting script is finished, the mounting script is executed to mount the first data file directory in the object storage system minio to the second data file directory in the private storage system, so that the internet of things data stored in each storage bucket in the object storage system minio is synchronized to the private storage system in real time for storage. Specifically, after the common cloud juicefs is created, information including the object storage key and the like is completely recorded in the redis. The juicefs client juicefs client on the local private cloud machine can mount the file system of the read-write juicefs as long as it has the redis database address, the user name and the password information. Therefore, the local juicefs client has no local configuration file, and the configuration file of the juicefs client sharing the cloud can be read and written only by executing the mounting script. Data and metadata of the juicefs file system are stored in the network-based cloud service, so that the file system can be simultaneously mounted on any computer on which the juicefs client is mounted for sharing reading and writing, and when multiple clients are simultaneously mounted for reading and writing the same file system, juicefs provides "close-to-open") consistency assurance, namely when two or more clients simultaneously read and write the same file (shared cloud), modification of the client a (edge) is not necessarily immediately visible on the client B (local private cloud). However, once the writing of this file at client a is completed and closed, re-opening the file at any one of clients B may ensure that the newly written data is accessible. Based on the method, the data of the Internet of things which is written into the shared cloud end by the edge end can be seen in real time by locally mounting the catalog/opt/jfs.

In an embodiment of the present application, the private cloud is further configured to: under the condition that a control instruction aiming at engineering equipment is acquired, caching the control instruction to a theme corresponding to the engineering equipment in a message queue; calculating control data cached in the theme based on a preset calculation engine, transmitting the calculated control data to a second data file directory in the private storage system, and mounting the control data to a first data file directory in the object storage system based on a client of the distributed system; and issuing control data to a preset data acquisition application of the Internet of things through a container interface service based on the distributed system by the shared cloud so as to control engineering equipment to execute corresponding operations.

In the technical scheme, in the data interaction process of the internet of things, the process of issuing data such as firmware program parameter configuration, firmware upgrading and control instructions of engineering equipment is generally involved. Therefore, when the private cloud acquires the control instruction for the engineering equipment, the private cloud caches the control instruction to the Topic corresponding to the engineering equipment in the message queue kafka, calculates the control data cached to the Topic based on the preset calculation engine Flink, transmits the calculated control data to the second data file directory in the private storage system minio, mounts the control data to the first data file directory in the object storage system minio based on the client juicefs client of the distributed system, and finally issues the control data to the preset internet of things data acquisition application app based on the container interface service juicefs CSI DRIVER of the distributed system through the shared cloud so as to control the engineering equipment to execute corresponding operations. In the technical scheme, data such as parameter configuration, firmware upgrading programs, control instructions and the like are issued to specified topic topics of different kafka message queues through a terminal Internet of things system, instruction type data in the kafka specified topic topics are consumed based on a Flink computing engine, and the downlink data are written into specific files of a local mount directory/opt/jfs through FileSystemConnector connection tools. After that, through the cloud edge real-time synchronization process of juicefs, the edge application app obtains instruction data, configuration parameters and firmware upgrading program data issued by the terminal internet of things system from the corresponding catalogue in the edge container, and then the functions of instruction control, parameter configuration, firmware upgrading and the like are completed at the edge, and specifically, the downlink process of the instruction data can be shown as fig. 4. By the advanced flow design and technical innovation, real-time synchronous acquisition of mass internet of things data of engineering equipment, real-time data analysis, real-time transmission of downlink data such as instructions, parameter configuration and the like can be realized.

In the technical scheme, the application of the cloud native technology at the edge end is realized by adopting innovative flow design and technical innovation, and the real-time acquisition and synchronous transmission of mass internet of things data at the edge end to the shared cloud end are realized by integrating the data acquisition application with juicefs container interface service in the container at the edge end by implementing the K3S container technology at the edge end. By constructing juicefs distributed storage systems on the shared cloud, selecting a network database redis as the metadata storage of juicefs, and simultaneously adopting a distributed object storage system minio as the data file storage of juicefs, a set of reliable and high-performance cloud distributed data storage system based on the shared cloud is constructed, the centralized storage of edge data in the shared cloud is realized, and the data storage and use cost are effectively reduced. By carrying out the juicefs file system mounting function on the private cloud server, the reading and writing operation of the private cloud on the data in the common cloud juicefs is realized, the effect that the operation of the data file in the local file system is equivalent to the operation of the data file in the common cloud juicefs is achieved, the repeated storage of the data of the Internet of things is avoided, and the problem of inconsistent data is reduced. The data files in the private cloud are processed in real time by using the Flink distributed computing engine, so that the requirement of the Internet of things platform on the Internet of things data is met. By means of a mechanism that the same file system is read and written by being mounted and mounted at the same time based on juicefs multiple clients, data synchronization is achieved, the functions of sending downstream data such as parameter configuration, instruction control and upgrading programs to edge equipment through an Internet of things platform are achieved, and the real-time bidirectional synchronization function of cloud side data is achieved. By means of the technical scheme, complex flow design of traditional Internet of things data acquisition is greatly simplified, technical input cost is lower, customized function development workload is reduced, and Internet of things data acquisition cost is reduced. Meanwhile, the data acquisition efficiency of the Internet of things is higher, more edge devices can be simultaneously accessed, and the data throughput is greatly improved. According to the scheme, all data of the edge end are stored in the shared cloud end in a centralized mode, the problems that data quality is poor, data inconsistency is serious due to repeated storage of the data are effectively avoided, data storage cost is greatly reduced, and data reliability is improved. Through the technical design based on the data collaboration, the operations of instruction control, parameter configuration and the like of engineering equipment are realized.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.

The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A processing system for internet of things data, the processing system comprising:

The edge end comprises a container arrangement platform, a container interface service of a distributed system and a preset internet of things data acquisition application, wherein the container arrangement platform is used for providing a container for the container interface service of the distributed system and the preset internet of things data acquisition application; the preset internet of things data acquisition application is used for acquiring internet of things data of engineering equipment in real time; the container interface service of the distributed system is used for establishing connection between the edge end and the common cloud end and mapping the internet of things data to the common cloud end in real time;

The shared cloud comprises the distributed system, an object storage system and a metadata storage system, wherein the distributed system is used for receiving the internet of things data sent by the edge end in real time and performing persistent storage; the object storage system is connected with the distributed system and comprises a plurality of storage barrels, and each storage barrel is used for storing the internet of things data of the same type of engineering equipment; the metadata storage system is connected with the distributed system and is used for storing metadata information in the object storage system, the metadata information at least comprises a name of each storage barrel, a first connection address, a first login user name, a first login password and first data file directory information in each storage barrel, and the metadata storage system is also used for providing a first connection address, a first login user name and a first login password of a corresponding target storage barrel according to engineering equipment to which the Internet of things data belongs under the condition that the distributed system receives the Internet of things data so as to establish connection with the target storage barrel and store the Internet of things data into the target storage barrel;

the private cloud comprises a client of the distributed system, a private storage system and a data analysis system, wherein the client of the distributed system is used for establishing connection between the distributed system and the private cloud so as to synchronize the internet of things data stored in the distributed system to the private storage system in real time for storage; the data analysis system is used for extracting target internet of things data corresponding to the calculation task from the private storage system under the condition that the private cloud receives the calculation task, calculating the target internet of things data based on a preset calculation engine and outputting the calculated internet of things data.

2. The processing system for internet of things data according to claim 1, wherein the common cloud is further configured to:

Deploying the metadata storage system and the object storage system respectively;

respectively acquiring first access information of the object storage system and second access information of the metadata storage system, wherein the first access information comprises a name of each storage barrel, a first connection address, a first login user name and a first login password, and the second access information comprises a second login user name, a second login password and a second connection address of the metadata storage system;

acquiring an installation package of the distributed system, and deploying the distributed system based on the installation package;

Constructing a first operation instruction for the distributed system based on the first access information and the second access information;

executing the first operation instruction to enable the distributed system to be connected with the metadata storage system and the object storage system respectively.

3. The processing system for internet of things data according to claim 1, wherein the data analysis system further comprises:

The message queue comprises a plurality of topics, and each topic is used for storing the internet of things data of the same type of engineering equipment calculated by the preset calculation engine.

4. The processing system for internet of things data according to claim 1, wherein the edge is further configured to:

Acquiring a deployment script of the container arrangement platform, and constructing a second operation instruction of a service node based on the deployment script;

executing the second operation instruction to deploy the service node to the edge end;

Obtaining a token of the service node, and constructing a third operation instruction of the proxy node based on the token of the service node;

executing the third operation instruction to deploy the proxy node to the edge end;

and under the condition that the operation states of the service node and the proxy node are detected to be opened, determining that the container arrangement platform is completely deployed.

5. The processing system for internet of things data according to claim 1, wherein the container orchestration platform is further configured to:

Acquiring a first containerized deployment script file for a container interface service of the distributed system;

and under the condition that a first operation instruction input by a user through the container arrangement platform is acquired, executing the first containerized deployment script file according to the first operation instruction so as to deploy the container interface service of the distributed system to the container arrangement platform.

6. The processing system for internet of things data according to claim 5, wherein the container orchestration platform is further configured to:

defining a second containerized deployment script file of a key, and adding a second connection address of a metadata storage system and first access information of an object storage system into the second containerized deployment script file;

executing the second containerized deployment script file according to the second running instruction under the condition that the second running instruction input by the user through the containerized platform is acquired, so as to deploy the key to the containerized platform;

A third containerized deployment script file defining a storage class of the distributed system, wherein the third containerized deployment script file includes a name of the storage class, a plug-in name, and a name of the key;

and under the condition that a third running instruction input by the user through the container arrangement platform is acquired, executing the third containerized deployment script file according to the third running instruction so as to deploy the storage class of the distributed system to the container arrangement platform.

7. The processing system for internet of things data according to claim 6, wherein the edge is further configured to:

defining a fourth containerized deployment script file of the preset internet of things data acquisition application, wherein the fourth containerized deployment script file comprises an image file, a service port, a data persistence configuration file and a PVC configuration file of the preset internet of things data acquisition application;

Adding the storage class to the PVC configuration file;

And under the condition that a fourth operation instruction input by the user through the container arrangement platform is acquired, executing the fourth containerized deployment script file according to the fourth operation instruction so as to deploy the preset Internet of things data acquisition application to the container arrangement platform.

8. The processing system for internet of things data according to claim 7, wherein the preset internet of things data collection application is further configured to:

Acquiring Internet of things data acquired by a sensor on the engineering equipment in real time, and preprocessing the Internet of things data;

And establishing connection with the distributed system based on the container interface service of the distributed system, and mapping the preprocessed internet of things data to the object storage system in real time through the storage class for persistent storage.

9. The processing system for internet of things data according to claim 1, wherein the private cloud is further configured to:

acquiring an installation script of a client of the distributed system;

Executing the installation script under the condition that a fifth running instruction aiming at the installation script is acquired, so as to deploy the client of the distributed system into the private cloud;

establishing connection between the distributed system and the private cloud based on a client of the distributed system;

constructing a mounting script for the distributed system, wherein the mounting script comprises second access information of a metadata storage system;

And executing the mounting script to mount a first data file catalog in the object storage system to a second data file catalog in the private storage system so as to synchronize the internet of things data stored in each storage bucket in the object storage system to the private storage system in real time for storage.

10. The processing system for internet of things data according to claim 9, wherein the private cloud is further configured to:

Under the condition that a control instruction aiming at the engineering equipment is acquired, caching the control instruction to a theme corresponding to the engineering equipment in a message queue;

Calculating control data cached in the subject based on the preset calculation engine, transmitting the calculated control data to a second data file directory in the private storage system, and mounting the control data to a first data file directory in the object storage system based on a client of the distributed system;

And issuing the control data to the preset internet of things data acquisition application through the shared cloud based on the container interface service of the distributed system so as to control the engineering equipment to execute corresponding operation.