CN108322548B - Industrial process data analysis platform based on cloud computing - Google Patents
Industrial process data analysis platform based on cloud computing Download PDFInfo
- Publication number
- CN108322548B CN108322548B CN201810185964.6A CN201810185964A CN108322548B CN 108322548 B CN108322548 B CN 108322548B CN 201810185964 A CN201810185964 A CN 201810185964A CN 108322548 B CN108322548 B CN 108322548B
- Authority
- CN
- China
- Prior art keywords
- sub
- cluster
- data
- industrial
- system host
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/75—Indicating network or usage conditions on the user display
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses an industrial process data analysis platform based on cloud computing, which comprises a cloud service end, an industrial site end, a client and a monitoring end, wherein the cloud service end consists of a system host and n sub-clusters, each sub-cluster comprises a sub-cluster host and a plurality of slave machines connected with the host, the system host controls the sub-cluster hosts, and the sub-cluster hosts control the slave machines connected with the sub-cluster hosts; the industrial field end reads the data of the industrial object in real time and sends the data to the cloud server end for analysis and processing; the client is used for remotely connecting the cloud server, reading and displaying a cloud server computing result in real time and remotely sending an instruction to the cloud server; the monitoring terminal is used for monitoring the cloud service terminal, managing user information and industrial object information and sending a control instruction to the industrial site terminal in real time through the cloud service terminal. The platform can realize remote monitoring of the industrial process, has high operation efficiency, and can bear large-scale users for simultaneous use.
Description
Technical Field
The invention belongs to the field of industrial process control, and particularly relates to an industrial process data analysis platform based on cloud computing.
Background
In industrial process control, an industrial process data analysis software platform often plays an important role, the platform mainly can collect and store process data in real time, and corresponding modeling algorithms are used for mainly realizing data-driven industrial process monitoring and soft measurement of process variables and quality variables so as to keep stable operation and improve product quality.
The industrial process monitoring based on data driving is to establish a mathematical model through actually measured process data so as to estimate and acquire the running state of the process, finally identify the abnormal behavior of the process, realize the timely diagnosis of the fault and ensure the safe running of the system; the process variable and quality variable soft measurement adopts a proper model, and the process variable and quality variable which are difficult to be directly measured by a sensor are predicted by using process variable data which are easy to measure, so that the product quality can be better controlled, and the production efficiency is improved.
At present, most industrial process data analysis software platforms are located in local computers of industrial fields, a client and an algorithm program are mutually nested, algorithms are calculated on one computer, and a data storage mode of the algorithms is that a certain computer is used as a database for storage, so that the calculation efficiency of the algorithms and the data storage capacity are limited. However, with the continuous development of modern industry, more and more industrial data can be stored, a series of industrial big data problems are formed, and with the continuous rise of internet and internet of things, the mode of realizing remote monitoring based on network is also the inevitable trend of modern industry by using mobile clients and other modes of fusing 'internet + industry'.
However, the traditional industrial process data analysis platform cannot meet the requirements of modern industry under the drive of the internet and ultra-large-scale data, firstly, the traditional system cannot store the ultra-large-scale data, and if the traditional system still uses less data to carry out operation, larger deviation is generated; secondly, due to excessive data quantity, the traditional single-machine algorithm operation mode cannot meet the real-time performance and stability of calculation; meanwhile, the client form of the system is single, and the client and the algorithm program cannot realize remote monitoring and control on site; finally, the algorithm in the traditional system has no shareability and reusability, a set of process monitoring and soft measurement algorithm must be independently written for each industrial object, the algorithm utilization rate is low, and the algorithm is not easy to manage. In summary, the conventional industrial process data analysis platform needs to be improved in terms of both algorithms and software platforms.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides an industrial process data analysis platform based on cloud computing, which can realize high independence of various algorithms, high sharing of data and high operation efficiency, and the specific technical scheme is as follows:
the industrial process data analysis platform based on cloud computing is characterized by comprising a cloud service end, an industrial site end, a client and a monitoring end;
the cloud service end is composed of a system host and n sub-clusters, the system host and each sub-cluster are positioned in the same local area network, wherein a Web server for providing service for the monitoring end, a data server for driving a system host database and a load balancer for connecting and managing each sub-cluster are installed in the system host; each sub-cluster comprises a sub-cluster host and a plurality of slave machines connected with the host, the system host controls the plurality of sub-cluster hosts, and the sub-cluster hosts control the slave machines connected with the sub-cluster hosts;
the industrial site end comprises a plurality of data processing modules, each data processing module is bound with a sensor of a corresponding industrial object, reads data of the industrial object in the working process in real time and sends the data to the cloud service end for analysis and processing; the data processing module is also used for remotely receiving control instructions sent by the client and the monitoring end to realize the start and stop of data collected by the data processing module, and the data processing module can also set corresponding industrial object control operation according to actual field conditions;
the client is used for remotely connecting the cloud server, reading a calculation result of the cloud server in real time, carrying out visual display, remotely sending an instruction to the cloud server according to needs, and remotely controlling the start and stop of data acquisition work of a data acquisition module of an industrial field end and the control operation of an industrial object under the condition allowed by the field by the cloud server;
the monitoring terminal is used for monitoring the working state of the cloud server and the industrial object operation data, managing user information and industrial object information, and sending a control instruction to the industrial site terminal in real time through the cloud server.
Further, the interaction process of the industrial site side, the cloud service side and the client side is as follows:
the industrial field side logs in a system host of the cloud server side, and the system host distributes corresponding calculation sub-clusters to the system host; the method comprises the steps that an industrial field terminal obtains variable data in real time from an industrial field local database and transmits the variable data to a corresponding sub-cluster of a cloud server according to a certain period to carry out algorithm analysis, the cloud server stores data analysis results into a database of a system host, and a client side polls the database of the system host to read the data analysis results and carry out visualization;
the client sends an instruction to the cloud server, the zone bit of the corresponding field in the system host database is modified, the industrial field side polls the corresponding field of the system host database, the zone bit is read, and corresponding control operation is started.
Further, the interaction between the monitoring terminal, the cloud service terminal and the industrial site terminal includes:
the monitoring terminal polls a system host database of the cloud service terminal and visualizes data so as to monitor the running state of the industrial site terminal; the monitoring end accesses a system host database in the cloud server end, reads user information and realizes the addition, deletion, modification and check of the user information; the monitoring end reads the working parameters of the system host and the sub-clusters in real time and dynamically displays the working parameters;
and the monitoring end sends an instruction to the cloud server end, modifies the zone bit of the corresponding field of the system host database, and the industrial field end polls the corresponding field of the system host database, reads the zone bit and starts corresponding control operation.
Further, the load balancer balances the load as follows:
the load balancer reads the working state information of the connected sub-groups at the same time interval, the working state information comprises the static configuration of each sub-group node and the dynamic performance in the working process, the dynamic performance comprises the CPU frequency, the memory occupation condition, the disk I/O reading rate and the communication network bandwidth use condition, the number of users distributed to each sub-group at the moment is dynamically calculated by adopting a load balancing algorithm after the dynamic performance is read, then the field of the IP address of each client-side connection sub-group node stored in a system host database is changed, and the client-side reads the IP address of the distributed computing node by polling the field and is connected with the node, so that the users are newly allocated to each sub-group.
Further, the load balancer adopts a dynamic load balancing algorithm based on load weight probability to control the load distribution of the sub-clusters, and the algorithm comprises the following steps:
the method comprises the following steps: defining an integrated static performance index C(s) describing compute nodes in a sub-clusterij) And a real-time dynamic performance index L(s)ij)：
C(sij)＝k1×nij×C(cij)+k2×C(mij)+k3×C(dij)+k4×C(pij)
L(sij)＝k1×L(cij)+k2×L(mij)+k3×L(dij)+k4×L(pij)
Wherein, i is 1, …, N, j is 1, …, MiN is the number of sub-clusters connected to the system host, MiThe number of computing nodes connected under the ith sub-cluster, sijIs the j computing node under the i sub-cluster, nijIs the number of CPU cores of the compute node, cijRepresenting the usage of the node CPU, C (C)ij) Representing a preconfigured static frequency, L (c)ij) Representing real-time dynamic CPU frequency during operation; m isijRepresents the use case of node memory, wherein C (m)ij) Representing a preconfigured memory capacity, L (m)ij) Representing the real-time dynamic memory occupancy rate during work; dijRepresenting the use of node disk I/O, C (d)ij) Representing a preconfigured disk I/O read rate, L (d)ij) Representing the I/O occupancy rate of the real-time dynamic disk in work; p is a radical ofijRepresents the network usage of each node, where C (p)ij) Representing a preconfigured node network throughput, L (p)ij) Representing real-time occupancy of the node network bandwidth, where k1,k2,k3,k4Is a weight coefficient, and k1+k2+k3+k4＝1；
Step two: defining a load weight W(s) of a node from static and dynamic performance indicators of the computing nodeij)
Step three: the load weight values of all the nodes are summed and averaged to obtain the load weight value W (S) of each sub-clusteri)
Step four: the maximum load weight value in each sub-cluster at a certain moment is represented as Wmax(S) bringing it into contact therewithLoad weights of all the sub-clusters are differenced to obtain a load difference value delta Q (S)i)
ΔQ(Si)＝Wmax(S)-W(Si)
Step five: defining the probability P (S) that the system host assigns users to each sub-clusteri)
Step six: multiplying the current user number using the platform by the user distribution probability to obtain the number of users to be distributed to each sub-cluster
R(S1)＝floor(P(S1)×Num)
R(S2)＝floor(P(S2)×Num)
R(Sk)＝0
Where Num is the number of people currently using the platform, floor is rounding down, R (S)i) The number of users assigned to each sub-cluster.
Further, the system host database is used for storing basic information of the user, industrial object information bound by the user, control instructions sent by the client and the monitoring terminal, and analysis data acquired from the sub-cluster.
Furthermore, the sub-cluster is configured with a distributed file storage architecture and a Map-Reduce-based distributed parallel computing framework, the sub-cluster host is a scheduling node, the sub-cluster host is connected with the sub-cluster host and is a data node and a computing node, all process monitoring and variable soft measurement algorithms are written on the computing node of the sub-cluster slave in each sub-cluster in the form of Map-Reduce, a functional interface is formed for each algorithm based on an MVC programming model and is stored in each sub-cluster host, and data is also stored in a distributed manner in a database of each computing node.
Furthermore, the client runs on a PC and/or a mobile terminal, and the monitoring terminal runs in a webpage form.
Furthermore, the process monitoring and variable soft measurement algorithms configured in the computing nodes of the sub-cluster of the cloud server side have a distributed parallel mode and a single machine mode.
Furthermore, the client can also directly upload local data to the cloud server for data analysis.
The invention has the following beneficial effects:
according to the industrial process data analysis platform based on cloud computing, a plurality of clusters are configured in a cloud server, each cluster is provided with a distributed parallel computing frame, all distributed industrial monitoring and soft measurement algorithms are compiled in the cloud server, a calling interface of each algorithm is formed, and meanwhile, a multithreading technology is adopted to realize that all the algorithms can be used at the same time; and the industrial site end acquires data in real time and remotely sends the data to the cloud end for calculation, and the client end reads a calculation result from the cloud server and visualizes the calculation result. The process monitoring and soft measurement algorithms are highly independent and highly shared, and the operation efficiency is improved.
Drawings
FIG. 1 is an overall architecture diagram of a cloud computing-based industrial process data analytics platform of the present invention;
FIG. 2 is a schematic diagram of a cloud server;
FIG. 3 is a schematic diagram of the operation of a load balancer at a cloud server;
FIG. 4 is a diagram illustrating a distributed storage architecture for files in a sub-cluster;
FIG. 5 is a block diagram of a distributed computing framework in a sub-cluster;
FIG. 6 is a schematic diagram illustrating an interaction process between an industrial site and a cloud server or a client;
FIG. 7 is a schematic diagram illustrating an interaction process between a client and a cloud server or an industrial site;
fig. 8 is a schematic view of an interaction process between a monitoring terminal and a cloud server terminal and between a monitoring terminal and an industrial site terminal.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the objects and effects of the present invention will become more apparent, and the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, an industrial process data analysis platform based on cloud computing includes a cloud service end, an industrial site end, a client end and a monitoring end;
the cloud service end consists of a system host and n sub-clusters, wherein the system host and each sub-cluster are positioned in the same local area network, and a Web server for providing service for the monitoring end, a data server for driving a system host database and a load balancer for connecting and managing each sub-cluster are arranged in the system host; each sub-cluster comprises a sub-cluster host and a plurality of slave machines connected with the host, the system host controls the plurality of sub-cluster hosts, and the sub-cluster hosts control the slave machines connected with the sub-cluster hosts; the sub-cluster is provided with a distributed file storage architecture and a Map-Reduce-based distributed parallel computing framework, the sub-cluster host is a scheduling node, the slave machines connected with the sub-cluster host are a data node and a computing node, all process monitoring and variable soft measurement algorithms are written on the computing nodes of the slave machines in each sub-cluster in a Map-Reduce mode, a functional interface is formed for each algorithm based on an MVC programming model and stored in each sub-cluster host, and meanwhile, data are also stored in a distributed mode in a database of each computing node.
The industrial site end is presented in a software form and comprises a plurality of data processing modules, each data processing module is bound with a sensor of a corresponding industrial object, reads data of the industrial object in the working process in real time and sends the data to the cloud service end for analysis and processing; the data processing module is also used for remotely receiving control instructions sent by the client and the monitoring end to realize the start and stop of data collected by the data processing module, and the data processing module can also set corresponding industrial object control operation according to actual field conditions;
the client is used for remotely connecting the cloud server, reading a calculation result of the cloud server in real time, carrying out visual display, remotely sending an instruction to the cloud server according to needs, and remotely controlling the start and stop of data acquisition work of a data acquisition module of the industrial field end and the control operation of an industrial object under the condition allowed by the field by the cloud server; the client runs on a PC and/or a mobile terminal.
The monitoring terminal adopts a webpage form and is used for monitoring the working state of the cloud service terminal and the operation data of the industrial object, managing user information and industrial object information, editing registration information of users, adding and cancelling the users, checking the industrial object information bound by the current cloud service terminal, and sending a control instruction to the industrial site terminal in real time through the cloud service terminal. The administrator can log in the monitoring end webpage at any time and place for management.
The specific working principles and functions of the platform cloud server, the industrial site, the client and the monitoring terminal are described in detail below.
1. Cloud server
A specific architecture of the cloud server is shown in fig. 2, and specific functions and working processes of each module in the cloud server are specifically described below.
1.1 System host
1.1.1 data Server
The data server is mainly used for driving the system host database and obtaining the algorithm analysis result stored in the distributed database in the sub-cluster. The user information data table in the system host database is firstly used for storing information of user such as login, registration information, use duration, purchase time and the like, and secondly storing industrial object information bound by the user, control sent by the client and the monitoring terminalInstructions and analytical data obtained from the sub-clusters, (e.g. using PCA (principal component analysis) for process monitoring, the analytical data obtained from the sub-clusters is T2And SPE statistics and control limits of both) for the client and the monitoring side to read data and send instructions using the data server.
1.1.2Web Server
The Web server is a server used for driving the monitoring end Web page, namely, all programs of the monitoring end Web site are executed in the Web server in the system host. Since the monitoring end needs to send a command to the cloud end in real time, read and manage user information, industrial object information, cloud end analysis data and the like stored in the system host database, the monitoring end needs to access the database of the system host through the Web server to perform the above dynamic operation on the system host database.
1.1.3 load balancer
The purpose of realizing load balancing of each sub-cluster in the system host is to solve the problem of uneven load caused when a large number of users access a certain sub-cluster at the same time. When the workload of a certain sub-cluster is already large, the computing capability and the computing speed of the certain sub-cluster are both greatly reduced, and the loads of other sub-clusters are possibly not large, so that the load (user) of the large-load sub-cluster needs to be distributed to other sub-clusters, each sub-cluster keeps the best working state, and the overall computing speed of the cloud server is ensured.
The working principle of the load balancer designed in the platform is shown in fig. 3, the load balancer is installed in a system host, the load balancer reads the working state information of the connected sub-clusters at the same time, including the static configuration of each sub-cluster node and the dynamic performance in the working process, the dynamic performance comprises CPU frequency, memory occupation, disk I/O reading rate and communication network bandwidth use condition, the number of users assigned to each sub-cluster at that time is dynamically calculated using a load balancing algorithm after reading, and then changing the field of the IP address of each client connection sub-cluster node stored in the system host database, and the client reads the IP address of the distributed computing node by polling the field and is connected with the node, thereby realizing the re-allocation of users for each sub-cluster.
The load balancer designed in the platform has an internal load balancing strategy of a dynamic load balancing algorithm based on load weight probability, and the algorithm has the following principle:
the method comprises the following steps: defining an integrated static performance index C(s) describing compute nodes in a sub-clusterij) And a real-time dynamic performance index L(s)ij)：
C(sij)＝k1×nij×C(cij)+k2×C(mij)+k3×C(dij)+k4×C(pij)
L(sij)＝k1×L(cij)+k2×L(mij)+k3×L(dij)+k4×L(pij)
Wherein, i is 1, …, N, j is 1, …, MiN is the number of sub-clusters connected to the system host, MiThe number of computing nodes connected under the ith sub-cluster, sijIs the j computing node under the i sub-cluster, nijIs the number of CPU cores of the compute node, cijRepresenting the usage of the node CPU, C (C)ij) Representing a preconfigured static frequency, L (c)ij) Representing real-time dynamic CPU frequency during operation; m isijRepresents the use case of node memory, wherein C (m)ij) Representing a preconfigured memory capacity, L (m)ij) Representing the real-time dynamic memory occupancy rate during work; dijRepresenting the use of node disk I/O, C (d)ij) Representing a preconfigured disk I/O read rate, L (d)ij) Representing the I/O occupancy rate of the real-time dynamic disk in work; p is a radical ofijRepresents the network usage of each node, where C (p)ij) Representing a preconfigured node network throughput, L (p)ij) Representing real-time occupancy of the node network bandwidth, where k1,k2,k3,k4Is a weight coefficient, and k1+k2+k3+k4＝1；
Step two: defining a load weight W(s) of a node from static and dynamic performance indicators of the computing nodeij)
Step three: the load weight values of all the nodes are summed and averaged to obtain the load weight value W (S) of each sub-clusteri)
Step four: the maximum load weight value in each sub-cluster at a certain moment is represented as Wmax(S), the load weight values of all the sub-clusters are subtracted to obtain a load difference value delta Q (S)i)
ΔQ(Si)＝Wmax(S)-W(Si)
Step five: defining the probability P (S) that the system host assigns users to each sub-clusteri)
Step six: multiplying the current user number using the platform by the user distribution probability to obtain the number of users to be distributed to each sub-cluster
R(S1)＝floor(P(S1)×Num)
R(S2)＝floor(P(S2)×Num)
R(Sk)＝0
Where Num is the number of people currently using the platform, floor is rounding down, R (S)i) The number of users assigned to each sub-cluster.
1.2 sub-clusters
1.2.1 File distributed storage architecture in a sub-cluster
From the above, in order to improve the storage and extraction speed of mass data, the platform adopts a distributed file storage architecture, and the user data volume is less and only needs to be stored in the database attached to the system host, while a large amount of data used for calculation and algorithm modeling needs to be stored in each sub-cluster in a parallel distribution manner.
For each sub-cluster, the cluster master may be considered a scheduling node and the slaves it is connected to may be considered data nodes. When the file is stored, the scheduling node in the cluster host firstly divides the data into blocks according to a certain size, then informs the data nodes in the slave of the distribution result of each block of data, and finally stores the obtained data blocks into the disk of the computer or the server by each data node.
When the file is read, the scheduling node in the cluster host informs each data node of the file to be extracted, and after the notification is completed, each data node extracts and splices corresponding data blocks into a new file in parallel according to the requirement and sends the new file to the client.
The file distributed storage process is shown in fig. 4.
1.2.2 distributed computing framework in a sub-Cluster
The part of the platform mainly completing algorithm operation is a computing node in a sub-cluster, and in order to improve the capability of the algorithm in the process of processing big data, the platform is provided with a distributed parallel computing framework based on MapReduce in each sub-cluster.
The distributed parallel computing framework is similar to the file distributed storage framework, the operation process of each computing node is managed and scheduled by the cluster host, and the task or algorithm parallel is realized by simultaneously computing data blocks by a plurality of computing nodes under the control of the cluster host.
The MapReduce is one kind of distributed parallel computation, and mainly extracts and processes data blocks stored in each computing node in parallel through a Map link, maps the data blocks into key value pairs and stores the key value pairs into a disk of each computing node, and the Reduce link extracts the key value pairs generated by the Map in parallel, reduces the key value pairs according to data with the same key value and processes the reduced data.
The MapReduce process is performed in the computing nodes, and the cluster host is still responsible for scheduling among processes in the process, so that the distributed computing framework in the sub-cluster is shown in fig. 5.
1.2.3 Process monitoring and Soft measurement Algorithm library
Each sub-cluster comprises a process monitoring and soft measurement algorithm library which comprises a distributed parallel algorithm and a single machine algorithm from an operation mode; from a functional perspective, process monitoring algorithms as well as soft measurement algorithms are included. The codes of all algorithms are stored in the computing nodes of the sub-cluster, and a calling interface for transmitting parameters and data in and transmitting results out is formed.
When a plurality of users use the same algorithm at the same time, the computing node can process two requests at the same time in a multi-thread mode, and an exclusive folder is formed for each user, so that all algorithms can be used by large-scale users at the same time.
The user can also directly upload local data to the cloud server through the client to perform data analysis.
The interaction process between the various ends of the platform is described below.
The interaction process of the industrial site end, the cloud server and the client is shown in fig. 6, the industrial site end logs in a system host of the cloud server, and the system host distributes corresponding computing sub-clusters to the system host; the method comprises the steps that an industrial field terminal obtains variable data in real time from an industrial field local database and transmits the variable data to a corresponding sub-cluster of a cloud server according to a certain period to carry out algorithm analysis, the cloud server stores data analysis results into a database of a system host, and a client side polls the database of the system host to read the data analysis results and carry out visualization;
the client sends an instruction to the cloud server, the zone bit of the corresponding field in the system host database is modified, the industrial field side polls the corresponding field of the system host database, the zone bit is read, and corresponding control operation is started.
Fig. 7 shows an interaction process between the client and the cloud server and an industrial site, and a method for reading data of the cloud server in real time by the client still adopts a polling technology for a database of a host of the cloud server system, that is, an analysis result is read from the database of the host of the cloud server system at regular intervals, and is visually displayed on the client. Meanwhile, when the client sends an instruction to the industrial object, the relevant field of the system host database is updated, and the industrial field end can poll the system host database to read the field, so that the industrial field end updates the local database of the industrial monitoring software to complete control operation.
The interaction between the monitoring end and the cloud service end and the interaction between the monitoring end and the industrial site end are shown in fig. 8. The monitoring end is presented in a webpage form and mainly has the functions of monitoring the working state of the cloud server, managing user information, managing industrial object information, monitoring industrial object operation data, sending simple control instructions and the like.
The monitoring terminal polls a system host database of the cloud service terminal and visualizes data so as to monitor the running state of the industrial site terminal; the monitoring end accesses a system host database in the cloud server end, reads user information and realizes the addition, deletion, modification and check of the user information; the monitoring end reads working parameters of a system host and a sub-cluster in real time, dynamically displays the working parameters and draws a real-time dynamic curve;
and the monitoring end sends an instruction to the cloud server end, modifies the zone bit of the corresponding field of the system host database, and the industrial field end polls the corresponding field of the system host database, reads the zone bit and starts corresponding control operation.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and although the invention has been described in detail with reference to the foregoing examples, it will be apparent to those skilled in the art that various changes in the form and details of the embodiments may be made and equivalents may be substituted for elements thereof. All modifications, equivalents and the like which come within the spirit and principle of the invention are intended to be included within the scope of the invention.
Claims (9)
1. The industrial process data analysis platform based on cloud computing is characterized by comprising a cloud service end, an industrial site end, a client and a monitoring end;
the cloud service end is composed of a system host and n sub-clusters, the system host and each sub-cluster are positioned in the same local area network, wherein a Web server for providing service for the monitoring end, a data server for driving a system host database and a load balancer for connecting and managing each sub-cluster are installed in the system host; each sub-cluster comprises a sub-cluster host and a plurality of slave machines connected with the host, the system host controls the plurality of sub-cluster hosts, and the sub-cluster hosts control the slave machines connected with the sub-cluster hosts;
the industrial site end comprises a plurality of data processing modules, each data processing module is bound with a sensor of a corresponding industrial object, reads data of the industrial object in the working process in real time and sends the data to the cloud service end for analysis and processing; the data processing module is also used for remotely receiving control instructions sent by the client and the monitoring end to realize the start and stop of data collected by the data processing module, and the data processing module can also set corresponding industrial object control operation according to actual field conditions;
the client is used for remotely connecting the cloud server, reading a calculation result of the cloud server in real time, carrying out visual display, remotely sending an instruction to the cloud server according to needs, and remotely controlling the start and stop of data acquisition work of a data acquisition module of an industrial field end and the control operation of an industrial object under the condition allowed by the field by the cloud server;
the monitoring terminal is used for monitoring the working state of the cloud server and the industrial object operation data, managing user information and industrial object information, and sending a control instruction to the industrial field terminal in real time through the cloud server;
the interaction process of the industrial site end, the cloud service end and the client is as follows:
the industrial field side logs in a system host of the cloud server side, and the system host distributes corresponding calculation sub-clusters to the system host; the method comprises the steps that an industrial field terminal obtains variable data in real time from an industrial field local database and transmits the variable data to a corresponding sub-cluster of a cloud server according to a certain period to carry out algorithm analysis, the cloud server stores data analysis results into a database of a system host, and a client side polls the database of the system host to read the data analysis results and carry out visualization;
the client sends an instruction to the cloud server, the zone bit of the corresponding field in the system host database is modified, the industrial field side polls the corresponding field of the system host database, the zone bit is read, and corresponding control operation is started.
2. The cloud computing-based industrial process data analytics platform of claim 1, wherein the interaction of the monitor, cloud server and industrial site comprises:
the monitoring terminal polls a system host database of the cloud service terminal and visualizes data so as to monitor the running state of the industrial site terminal; the monitoring end accesses a system host database in the cloud server end, reads user information and realizes the addition, deletion, modification and check of the user information; the monitoring end reads the working parameters of the system host and the sub-clusters in real time and dynamically displays the working parameters;
and the monitoring end sends an instruction to the cloud server end, modifies the zone bit of the corresponding field of the system host database, and the industrial field end polls the corresponding field of the system host database, reads the zone bit and starts corresponding control operation.
3. The cloud computing-based industrial process data analytics platform of claim 1, wherein the load balancer balances the load as follows:
the load balancer reads the working state information of the connected sub-groups at the same time interval, the working state information comprises the static configuration of each sub-group node and the dynamic performance in the working process, the dynamic performance comprises the CPU frequency, the memory occupation condition, the disk I/O reading rate and the communication network bandwidth use condition, the number of users distributed to each sub-group at the moment is dynamically calculated by adopting a load balancing algorithm after the dynamic performance is read, then the field of the IP address of each client-side connection sub-group node stored in a system host database is changed, and the client-side reads the IP address of the distributed computing node by polling the field and is connected with the node, so that the users are newly allocated to each sub-group.
4. The cloud computing-based industrial process data analytics platform of claim 3, wherein said load balancer uses a dynamic load balancing algorithm based on load weight probability to control the load distribution of said sub-clusters, said algorithm comprising the steps of:
the method comprises the following steps: defining an integrated static performance index C(s) describing compute nodes in a sub-clusterij) And a real-time dynamic performance index L(s)ij)：
C(sij)＝k1×nij×C(cij)+k2×C(mij)+k3×C(dij)+k4×C(pij)
L(sij)＝k1×L(cij)+k2×L(mij)+k3×L(dij)+k4×L(pij)
Wherein, i is 1, …, N, j is 1, …, MiN is the number of sub-clusters connected to the system host, MiThe number of computing nodes connected under the ith sub-cluster, sijIs the j computing node under the i sub-cluster, nijIs the number of CPU cores of the compute node, cijRepresenting the usage of the node CPU, C (C)ij) Representing a preconfigured static frequency, L (c)ij) Representing real-time dynamic CPU frequency during operation; m isijRepresents the use case of node memory, wherein C (m)ij) Representing a preconfigured memory capacity, L (m)ij) Representing the real-time dynamic memory occupancy rate during work; dijRepresenting the use of node disk I/O, C (d)ij) Representing a preconfigured disk I/O read rate, L (d)ij) Representing the I/O occupancy rate of the real-time dynamic disk in work; p is a radical ofijRepresents the network usage of each node, where C (p)ij) Representing a preconfigured node network throughput, L (p)ij) Representing real-time occupancy of the node network bandwidth, where k1,k2,k3,k4Is a weight coefficient, and k1+k2+k3+k4＝1；
Step two: defining a load weight W(s) of a node from static and dynamic performance indicators of the computing nodeij)
Step three: the load weight values of all the nodes are summed and averaged to obtain the load weight value W (S) of each sub-clusteri)
Step four: the maximum load weight value in each sub-cluster at a certain moment is represented as Wmax(S), the load weight values of all the sub-clusters are subtracted to obtain a load difference value delta Q (S)i)
ΔQ(Si)＝Wmax(S)-W(Si)
Step five: defining the probability P (S) that the system host assigns users to each sub-clusteri)
Step six: multiplying the current user number using the platform by the user distribution probability to obtain the number of users to be distributed to each sub-cluster
R(S1)＝floor(P(S1)×Num)
R(S2)＝floor(P(S2)×Num)
R(Sk)＝0
Where Num is the number of people currently using the platform, floor is rounding down, R (S)i) The number of users assigned to each sub-cluster.
5. The cloud computing-based industrial process data analytics platform of claim 1, wherein the system host database is configured to store basic information of a user, industrial object information bound by the user, control instructions sent by the client and the monitor, and analysis data obtained from the sub-cluster.
6. The cloud-computing-based industrial process data analytic platform of claim 1, wherein the sub-clusters are configured with a distributed file storage architecture and a Map-Reduce-based distributed parallel computing architecture, the sub-cluster hosts are scheduling nodes, the slaves connected to the sub-cluster hosts are data nodes and computing nodes, all process monitoring and variable soft measurement algorithms are written on the computing nodes of the slaves in each sub-cluster in a Map-Reduce form, a functional interface is formed for each algorithm based on an MVC programming model and stored in each sub-cluster host, and data are also stored in a database of each computing node in a distributed manner.
7. The cloud computing-based industrial process data analytics platform of claim 1, wherein said client runs on a PC and/or a mobile terminal and said monitor runs in the form of a web page.
8. The cloud computing-based industrial process data analytic platform of claim 1, wherein the process monitoring and variable soft measurement algorithms configured in the compute nodes of the sub-cluster of the cloud server have a distributed parallel mode and a stand-alone mode.
9. The cloud computing-based industrial process data analytics platform as claimed in claim 1, wherein said client can also directly upload local data to a cloud server for data analysis.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810185964.6A CN108322548B (en) | 2018-03-07 | 2018-03-07 | Industrial process data analysis platform based on cloud computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810185964.6A CN108322548B (en) | 2018-03-07 | 2018-03-07 | Industrial process data analysis platform based on cloud computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108322548A CN108322548A (en) | 2018-07-24 |
| CN108322548B true CN108322548B (en) | 2020-06-05 |
Family
ID=62901354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810185964.6A Active CN108322548B (en) | 2018-03-07 | 2018-03-07 | Industrial process data analysis platform based on cloud computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108322548B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109889585B (en) * | 2019-02-01 | 2022-02-15 | 中电科思仪科技股份有限公司 | Remote monitoring system and method for running state of electronic test instrument |
| CN110071918B (en) * | 2019-04-19 | 2020-10-09 | 北京那镁克科技有限公司 | Data communication method and device based on hybrid cloud |
| CN110716986B (en) * | 2019-10-17 | 2021-05-04 | 深圳市蓝思网络技术有限公司 | Big data analysis system and application method thereof |
| CN111221807B (en) * | 2019-12-25 | 2023-07-18 | 中国特种设备检测研究院 | Cloud service-oriented industrial equipment big data quality testing method and framework |
| CN111638672B (en) * | 2020-06-05 | 2024-01-16 | 昆山艾控智能科技有限公司 | Automatic control system of industrial machine |
| CN111885146B (en) * | 2020-07-21 | 2022-06-24 | 合肥学院 | Data transmission method for industrial data cloud service platform |
| CN112051771B (en) * | 2020-09-01 | 2021-05-25 | 北京视界云天科技有限公司 | Multi-cloud data acquisition method and device, computer equipment and storage medium |
| CN112217894A (en) * | 2020-10-12 | 2021-01-12 | 浙江大学 | Load balancing system based on dynamic weight |
| CN112822049A (en) * | 2021-01-05 | 2021-05-18 | 浙江工业大学 | Remote monitoring system for access of multiple embedded gateway devices |
| CN113156909A (en) * | 2021-05-30 | 2021-07-23 | 广东新征程科技有限公司 | Ultrashort wave signal transmission automatic production line system |
| CN113518127B (en) * | 2021-07-15 | 2023-03-24 | 罗慧 | Industrial Internet of things information integration system |
| CN113566864A (en) * | 2021-09-03 | 2021-10-29 | 合肥米克光电技术有限公司 | Distributed machine vision system based on 5G and edge calculation |
| CN115277704B (en) * | 2022-09-27 | 2022-12-27 | 美冠(北京)科技有限公司 | Cloud playing system based on distributed preloading |
| CN116366660A (en) * | 2023-03-31 | 2023-06-30 | 广州大学 | Communication management intelligent system and method for distributed parallel simulation calculation |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981440A (en) * | 2012-11-02 | 2013-03-20 | 武汉理工大学 | Intelligent device monitoring and managing system based on software as a service (SaaS) |
| CN103546536A (en) * | 2013-08-28 | 2014-01-29 | 北京清控人居环境研究院有限公司 | Internet of things system of sewage treatment plant |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170238073A1 (en) * | 2016-02-16 | 2017-08-17 | Premiere Concrete Solutions, LLC | Concrete Maturity System |
-
2018
- 2018-03-07 CN CN201810185964.6A patent/CN108322548B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102981440A (en) * | 2012-11-02 | 2013-03-20 | 武汉理工大学 | Intelligent device monitoring and managing system based on software as a service (SaaS) |
| CN103546536A (en) * | 2013-08-28 | 2014-01-29 | 北京清控人居环境研究院有限公司 | Internet of things system of sewage treatment plant |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108322548A (en) | 2018-07-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108322548B (en) | Industrial process data analysis platform based on cloud computing | |
| Lin et al. | A cloud server energy consumption measurement system for heterogeneous cloud environments | |
| DE112011100143B4 (en) | Optimize electrical power consumption in a data center | |
| CN103164283A (en) | Method and system for dynamic scheduling management of virtualized resources in virtualized desktop system | |
| CN109766182A (en) | The scalable appearance method, apparatus of system resource dynamic, computer equipment and storage medium | |
| CN109192248B (en) | Biological information analysis system and method based on cloud platform and cloud computing platform system | |
| CN105491138B (en) | Distributed load scheduling method based on load rate graded triggering | |
| US20090119233A1 (en) | Power Optimization Through Datacenter Client and Workflow Resource Migration | |
| US20140229949A1 (en) | Balancing virtual machine loads | |
| CN105677836A (en) | Big data processing and solving system simultaneously supporting offline data and real-time online data | |
| CN104994145B (en) | A kind of load-balancing method based on KVM virtualization cluster | |
| CN104683450B (en) | Video service monitors cloud system | |
| US20150271023A1 (en) | Cloud estimator tool | |
| CN109614227A (en) | Task resource concocting method, device, electronic equipment and computer-readable medium | |
| CN104660690B (en) | cloud video service monitoring system | |
| CN102339233A (en) | Cloud computing centralized management platform | |
| CN114896068A (en) | Resource allocation method, resource allocation device, electronic device, and storage medium | |
| CN114138501B (en) | Processing method and device for edge intelligent service for field safety monitoring | |
| Kang et al. | Design of scheduler plugins for reliable function allocation in kubernetes | |
| JP6191695B2 (en) | Virtual resource control system and virtual resource control method | |
| CN105934915A (en) | Method and apparatus for presenting device load state in cloud computing network | |
| CN116991558B (en) | Computing power resource scheduling method, multi-architecture cluster, device and storage medium | |
| CN105471986B (en) | A kind of Constructing data center Scale Revenue Ratio method and device | |
| CN117221069A (en) | Management method and device based on micro-server architecture | |
| CN110083672A (en) | Geography information big data three-dimension visible sysem |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |