Disclosure of Invention
in order to solve the above technical problem, an embodiment of the present application provides an industrial big data modeling platform to achieve the purpose of analyzing industrial data, and a technical scheme is as follows:
an industrial big data modeling platform, comprising: the system comprises data management equipment, algorithm management equipment, model release and deployment equipment and prediction control equipment;
The input interface of the data management equipment is connected with an industrial database, and the output interface of the data management equipment is connected with the first input interface of the model management equipment;
the second input interface of the model management equipment is connected with the output interface of the algorithm management equipment;
The output interface of the model management equipment is connected with the input interface of the model release deployment equipment, and the output interface of the model release deployment equipment is connected with the input interface of the prediction control equipment;
the data management equipment is used for acquiring industrial data from the industrial database and managing the acquired industrial data;
the algorithm management equipment is used for managing a preset machine learning algorithm;
The model management device is used for constructing an industrial big data analysis model based on machine learning by using a machine learning algorithm managed by the algorithm management device, acquiring target data from the industrial data acquired by the data management device to serve as training data, and training the industrial big data analysis model based on machine learning by using the training data;
The model issuing and deploying equipment is used for deploying the machine learning-based industrial big data analysis model obtained by training the model management equipment to a prediction server;
the prediction control device is used for controlling the prediction server to predict industrial data generated by target equipment by using the industrial big data analysis model based on machine learning, and sending a control instruction generated according to a prediction result of the prediction server to the target equipment so as to control the target equipment to operate.
Preferably, the industrial big data modeling platform further comprises: model effectiveness monitoring and optimizing equipment;
The input interface of the model effectiveness monitoring and optimizing equipment is connected with the prediction control equipment, and the output interface of the model effectiveness monitoring and optimizing equipment is connected with the model management equipment;
The model effectiveness monitoring and optimizing equipment is used for monitoring the prediction performance of the industrial big data analysis model based on machine learning, and if the prediction performance is reduced, a control command is sent to the model management equipment;
and the model management equipment is further used for optimizing the industrial big data analysis model based on machine learning according to the control command, and replacing the optimized industrial big data analysis model based on machine learning with the optimized industrial big data analysis model based on machine learning.
Preferably, the model effectiveness monitoring and optimizing device is further configured to:
and if the monitored prediction performance of the industrial big data analysis model based on the machine learning is invalid within a set time, sending out a warning.
preferably, the model management device is specifically configured to:
Arranging the machine learning algorithm managed by the algorithm management equipment into a modeling process diagram;
And constructing an industrial big data analysis model based on machine learning according to the modeling process diagram.
Preferably, the model management device is further configured to:
And saving the modeling process diagram.
Preferably, the model management device is further configured to:
and copying the modeling process diagram.
Preferably, the model management device is further configured to:
And configuring parameters of a machine learning algorithm for constructing the machine learning-based industrial big data analysis model.
preferably, the model management device is further configured to:
And combining the constructed multiple industrial big data analysis models based on machine learning.
Preferably, the algorithm management device further comprises an input interface for receiving a custom algorithm uploaded by a user;
The algorithm management device is also used for storing and managing the custom algorithm uploaded by the user.
preferably, the model management device is further configured to:
And managing each industrial big data analysis model based on machine learning obtained by training.
Compared with the prior art, the beneficial effect of this application is:
in the present application, an industrial big data modeling platform is provided, comprising: the industrial big data analysis model based on machine learning can be constructed and used for analyzing industrial data, and the operation of target equipment is controlled.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
the embodiment of the application discloses industry big data modeling platform includes: the system comprises data management equipment, algorithm management equipment, model release and deployment equipment and prediction control equipment; the input interface of the data management equipment is connected with an industrial database, and the output interface of the data management equipment is connected with the first input interface of the model management equipment; the second input interface of the model management equipment is connected with the output interface of the algorithm management equipment; the output interface of the model management equipment is connected with the input interface of the model release deployment equipment, and the output interface of the model release deployment equipment is connected with the input interface of the prediction control equipment; the data management equipment is used for acquiring industrial data from the industrial database and managing the acquired industrial data; the algorithm management equipment is used for managing a preset machine learning algorithm; the model management device is used for constructing an industrial big data analysis model based on machine learning by using a machine learning algorithm managed by the algorithm management device, acquiring target data from the industrial data acquired by the data management device to serve as training data, and training the industrial big data analysis model based on machine learning by using the training data; the model issuing and deploying equipment is used for deploying the machine learning-based industrial big data analysis model obtained by training the model management equipment to a prediction server; the prediction control device is used for controlling the prediction server to predict industrial data generated by target equipment by using the industrial big data analysis model based on machine learning, and sending a control instruction generated according to a prediction result of the prediction server to the target equipment so as to control the target equipment to operate. In the present application, analysis of industrial data can be achieved.
next, a description is given to an industrial big data modeling platform disclosed in an embodiment of the present application, as shown in fig. 1, for a structural schematic diagram of an embodiment 1 of the industrial big data modeling platform provided in the present application, the industrial big data modeling platform may include:
the system comprises a data management device 11, an algorithm management device 12, a model management device 13, a model distribution deployment device 14 and a prediction control device 15.
an input interface of the data management device 11 is connected with an industrial database, and an output interface of the data management device 11 is connected with a first input interface of the model management device 13;
A second input interface of the model management device 13 is connected with an output interface of the algorithm management device 12;
An output interface of the model management device 13 is connected with an input interface of the model publishing and deployment device 14, and an output interface of the model publishing and deployment device 14 is connected with an input interface of the prediction control device 15;
The data management device 11 is configured to acquire industrial data from the industrial database and manage the acquired industrial data.
Managing the acquired industrial data may include, but is not limited to: and creating a data table aiming at the acquired industrial data, and deleting, downloading or browsing the data table for management.
the industrial database can include, but is not limited to, an industrial time series database and/or an industrial non-time series database.
and acquiring industrial data from the industrial time sequence database as industrial time sequence data. The industrial time series data can be understood as: industrial data collected in chronological order.
the industrial data obtained from the industrial non-time sequence database is industrial non-time sequence data. Industrial non-time series data can be understood as: industrial data collected not in chronological order.
The algorithm management device 12 is configured to manage a preset machine learning algorithm.
managing preset machine learning algorithms may include, but is not limited to: and adding, deleting, modifying, searching or classifying the preset machine learning algorithm.
The preset machine learning algorithm may include, but is not limited to: a feature selection algorithm, a data preprocessing algorithm, a machine learning algorithm, a neural network algorithm, and a performance evaluation algorithm.
it should be noted that the algorithm management device 12 provides a multi-framework algorithm distributed execution environment, such as spark-mllb, sciit-spare, tensorflow, xgboost. The algorithm distributed operation environment of the framework can be understood as the operation environment of the machine learning algorithm.
Preferably, a spark framework algorithm distributed execution environment can be used.
the model management device 13 is configured to construct an industrial big data analysis model based on machine learning by using the machine learning algorithm managed by the algorithm management device 12, acquire target data from the industrial data acquired by the data management device 11, use the target data as training data, and train the industrial big data analysis model based on machine learning by using the training data.
the process of the model management device 13 building the industrial big data analysis model based on machine learning by using the machine learning algorithm managed by the algorithm management device 12 can include but is not limited to:
A11, arranging the machine learning algorithm managed by the algorithm management device 12 into a modeling process diagram.
The modeled process graph may be, but is not limited to: there is a directed acyclic graph.
and A12, constructing an industrial big data analysis model based on machine learning according to the modeling process diagram.
in this embodiment, the model management device 13 may be further configured to:
And saving the modeling process diagram.
Preferably, the model management device 13 may be specifically configured to store the modeling process diagram according to a custom json format. When the modeling process diagram is stored according to the custom json format, the stored contents may include, but are not limited to: instance id, type, parameter, position of each algorithm node, mutual relation among nodes, showing mode of each node and design of anchor points capable of being connected.
in this embodiment, the model management device 13 may be further configured to:
And storing the operation result of each node in the modeling process diagram in an HDFS (Hadoop distributed file system). Wherein, the storage position is uniquely determined by the task id, the node instance id and the output parameter name.
Each node in the modeled process graph can be understood as: each algorithm (cleaning, preprocessing, feature engineering, machine learning, evaluation, etc.).
In this embodiment, the model management device 13 may be further configured to:
and copying the modeling process diagram.
And copying the modeling process diagram, realizing cloning of the modeling process, and supporting the reuse of the models of the same process.
In this embodiment, the model management device 13 may be further configured to:
and configuring parameters of a machine learning algorithm for constructing the machine learning-based industrial big data analysis model.
In this embodiment, the model management device 13 may be further configured to:
And combining the constructed multiple industrial big data analysis models based on machine learning.
In this embodiment, the model management device 13 may further be configured to:
And managing each industrial big data analysis model based on machine learning obtained by training.
the model publishing and deploying device 14 is configured to deploy the machine learning-based industrial big data analysis model obtained by training of the model management device 13 to a prediction server.
The prediction control device 15 is configured to control the prediction server to predict industrial data generated by a target device by using the machine learning-based industrial big data analysis model, and issue a control instruction generated according to a prediction result of the prediction server to the target device to control the target device to operate.
The industrial data generated by the target device can be understood as: real-time industrial data generated during the operation of the target device.
the forecast service environment of the forecast service side can support but is not limited to: pmml format model, pkl format model.
The prediction result of the prediction server can be understood as: and controlling the prediction server to predict the industrial data generated by the target equipment by using the industrial big data analysis model based on the machine learning.
in the present application, an industrial big data modeling platform is provided, comprising: the data management device 11, the algorithm management device 12, the model management device 13, the model release deployment device 14 and the prediction control device 15 cooperate with each other, so that an industrial big data analysis model based on machine learning can be constructed, industrial data is analyzed by using the industrial big data analysis model based on machine learning, and the operation of target devices is controlled.
As another alternative embodiment of the present application, referring to fig. 2, a schematic structural diagram of an embodiment 2 of an industrial big data modeling platform provided by the present application is provided, where this embodiment mainly relates to an extension scheme of the industrial big data modeling platform described in the foregoing embodiment 1, and as shown in fig. 2, on the basis of the industrial big data modeling platform shown in fig. 1, the present application may further include: model validity monitoring and optimization device 16.
An input interface of the model effectiveness monitoring and optimizing device 16 is connected with the predictive control device 15, and an output interface of the model effectiveness monitoring and optimizing device 16 is connected with a third input interface of the model management device 13.
The model effectiveness monitoring and optimizing device 16 is configured to monitor the prediction performance of the machine learning-based industrial big data analysis model, and send a control command to the model management device 13 if the prediction performance is reduced.
The model management device 13 is further configured to optimize the machine learning-based industrial big data analysis model according to the control command, and replace the optimized machine learning-based industrial big data analysis model with the machine learning-based industrial big data analysis model.
The process of optimizing the machine learning-based industrial big data analysis model by the model management device 13 may include:
and A21, updating the training data.
updating the training data may include:
acquiring new target data from the industrial data acquired by the data management equipment 11 to replace the training data;
Or, new target data is acquired from the industrial data acquired by the data management device 11, the new target data is merged with the training data, and the merged data replaces the training data.
and A22, training the industrial big data analysis model based on machine learning by using the updated training data.
in this embodiment, the model effectiveness monitoring and optimizing device 16 may further be configured to:
And if the monitored prediction performance of the industrial big data analysis model based on the machine learning is invalid within a set time, sending out a warning.
After the warning is issued, human intervention can be performed to optimize the industrial big data analysis model based on machine learning.
As another optional embodiment of the present application, referring to fig. 3, a schematic structural diagram of an embodiment 3 of the industrial big data modeling platform provided by the present application is provided, and this embodiment mainly relates to a refinement scheme of the industrial big data modeling platform described in the above embodiment 1, as shown in fig. 3, the algorithm management device 12 further includes an input interface for receiving a custom algorithm uploaded by a user.
the custom algorithm can be understood as: and compiling according to a platform development document 'custom algorithm development standard'. The custom algorithm development Specification defines the application development Specification and algorithms as recognizable template files for the visualization components.
The algorithm management device 12 is further configured to store and manage the custom algorithm uploaded by the user.
As another alternative embodiment of the present application, referring to fig. 4, a schematic structural diagram of an embodiment 4 of an industrial big data modeling platform provided by the present application is provided, where this embodiment mainly relates to an extension scheme of the industrial big data modeling platform described in the foregoing embodiment 2, as shown in fig. 4, on the basis of the industrial big data modeling platform shown in fig. 1, the present application may further include: a display device 17.
The display device 17 is connected to the data management device 11, the algorithm management device 1212, the model management device 13, the model distribution deployment device 14, the prediction control device 15, and the model validity monitoring and optimizing device 16, respectively, and visualizes the programs and operations of the data management device 11, the algorithm management device 12, the model management device 13, the model distribution deployment device 14, the prediction control device 15, and the model validity monitoring and optimizing device 16.
It should be noted that fig. 4 is a simplified schematic structural diagram, and connection relationships between the display device 17 and the data management device 11, the algorithm management device 12, the model management device 13, the model distribution deployment device 14, the prediction control device 15, and the model validity monitoring and optimizing device 16 are not shown in fig. 4.
It should be noted that each embodiment is mainly described as a difference from the other embodiments, and the same and similar parts between the embodiments may be referred to each other. For the device-like embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
for convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
from the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
The industrial big data modeling platform provided by the application is described in detail, a specific example is applied in the description to explain the principle and the implementation mode of the application, and the description of the embodiment is only used for helping to understand the method and the core idea of the application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.