CN115713099B - Model design method, device, equipment and storage medium - Google Patents
Model design method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115713099B CN115713099B CN202310000603.0A CN202310000603A CN115713099B CN 115713099 B CN115713099 B CN 115713099B CN 202310000603 A CN202310000603 A CN 202310000603A CN 115713099 B CN115713099 B CN 115713099B
- Authority
- CN
- China
- Prior art keywords
- target
- network model
- model
- data
- dnn
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a model design method, device, equipment and storage medium, relating to the technical field of computers, comprising the following steps: acquiring target table data, and acquiring a corresponding data set based on the target table data; integrating all the data sets by using a preset integration method to obtain a target data set; and constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result. According to the method, the deep learning target network model is built based on the DNN network model, the naive Bayesian model and the CNN network model, the DNN model performs preliminary prediction, the naive Bayesian model performs probability analysis, and the CNN model makes decisions, so that automatic replacement of scripts is realized, the operation and maintenance efficiency is improved, the labor cost is saved, and the risk of human errors is reduced.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for model design.
Background
With the gradual huge and stable operation of the system, the production system faces more problems on the system level than on the service level, such as error data, redundant data, dirty data, etc. In the case of the business problem related to the data, the operation and maintenance personnel often need to query a background database, judge the occurrence condition according to the characteristic field of the business table, and further issue an operation and maintenance script. Because operation and maintenance personnel are limited, the probability of occurrence of repeatability problems is greatly increased along with the continuous perfection of a service system, so that the maintenance efficiency is affected.
Disclosure of Invention
In view of the above, the present invention aims to provide a method, a device and a storage medium for model design, which can realize automatic replacement of scripts, improve operation and maintenance efficiency, save labor cost and reduce risk of human error. The specific scheme is as follows:
in a first aspect, the present application discloses a model design method, including:
acquiring target table data, and acquiring a corresponding data set based on the target table data;
integrating all the data sets by using a preset integration method to obtain a target data set;
And constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result.
Optionally, the acquiring the target table data and acquiring the corresponding data set based on the target table data includes:
and obtaining target table data corresponding to a target service table, and carrying out number generation based on characteristic field data of the target table data to obtain a data set corresponding to the characteristic field data under the condition.
Optionally, the integrating all the data sets by using a preset integration method to obtain a target data set includes:
extracting all the characteristic field data of the data set to obtain a current array;
performing preset normalization processing operation on the current array to obtain a normalized array;
performing preset labeling operation on the data in the normalized array to obtain a target array, and acquiring the target data set based on the target array; wherein the target data set comprises training data, validation data, and test data.
Optionally, the constructing a target network model based on the target data set, the original DNN network model, the original naive bayes model, and the original CNN network model includes:
acquiring a trained DNN network model based on the target data set and the original DNN network model;
acquiring first output data corresponding to the trained DNN network model, and acquiring a trained naive Bayes model based on the first output data and the original naive Bayes model;
acquiring second output data corresponding to the trained naive Bayes model, and acquiring a trained CNN network model based on an original CNN network model, the target data set, the first output data and the second output data;
and constructing the target network model based on the trained DNN network model, the trained naive Bayesian model and the trained CNN network model.
Optionally, the acquiring the trained DNN network model based on the target data set and the original DNN network model includes:
determining DNN model parameters based on the training data and the validation data in the target dataset;
generating a current DNN network model based on the DNN model parameters and the original DNN network model;
Adjusting DNN model parameters in the current DNN network model through a first preset loss function and a first preset accuracy calculation formula to obtain DNN model final parameters meeting current requirements;
and acquiring the trained DNN network model based on the DNN model final parameters and the current DNN network model.
Optionally, the acquiring the trained naive bayes model based on the first output data and the original naive bayes model includes:
setting prior probability and determining current posterior probability based on the prior probability;
updating the current posterior probability by using the first output data to obtain a target posterior probability;
and acquiring the trained naive Bayes model based on the target posterior probability and the original naive Bayes model.
Optionally, the acquiring the trained CNN network model based on the original CNN network model, the target data set, the first output data, and the second output data includes:
inputting the target data set, the first output data and the second output data into the original CNN network model to execute a preset training mode so as to obtain a current output value;
Acquiring a second preset loss function and a second preset accuracy calculation formula so as to adjust preset CNN parameters in the original CNN network model based on the current output value, the second preset loss function and the second preset accuracy calculation formula to obtain target CNN parameters;
acquiring the trained CNN network model based on the target CNN parameters and the original CNN network model;
correspondingly, the constructing a target network model based on the target data set, the original DNN network model, the original naive bayes model and the original CNN network model so as to predict the received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result, including:
and constructing the target network model based on the target data set, the trained DNN network model, the trained naive Bayesian model and the trained CNN network model so as to predict the received service table to be analyzed, automatically acquiring a corresponding target script according to the prediction result, and replacing the current script with the target script by a preset script replacement method.
In a second aspect, the present application discloses a model design apparatus comprising:
The first data set acquisition module is used for acquiring target table data and acquiring a corresponding data set based on the target table data;
the second data set acquisition module is used for integrating all the data sets by utilizing a preset integration method so as to obtain a target data set;
the model construction module is used for constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model and perform preset script replacement operation according to a prediction result.
In a third aspect, the present application discloses an electronic device comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the model design method as previously disclosed.
In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the model design method as previously disclosed.
It can be seen that the present application provides a model design method, comprising: acquiring target table data, and acquiring a corresponding data set based on the target table data; integrating all the data sets by using a preset integration method to obtain a target data set; and constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result. Therefore, the deep learning target network model is built based on the DNN network model, the naive Bayesian model and the CNN network model, the preliminary prediction service condition is carried out through the DNN model, the naive Bayesian model carries out probability analysis on the result of the DNN model, the CNN model carries out decision making according to the target data set, the DNN model output result and the naive Bayesian output result, and the final prediction result is analyzed and output, so that automatic replacement of a script is realized according to the prediction result, and the fitting property and accuracy are improved by combining a plurality of models; the training time is shortened by optimizing an algorithm in the model; the operation and maintenance efficiency is improved, the labor cost is saved, and the risk of human error is reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a model design method disclosed in the present application;
FIG. 2 is a flow chart of a specific model design method disclosed in the present application;
FIG. 3 is a flow chart of the dataset fabrication disclosed herein;
FIG. 4 is a flow chart of a specific model design method disclosed in the present application;
FIG. 5 is a schematic diagram of a model network structure disclosed in the present application;
FIG. 6 is a schematic diagram of a model framework for deep learning as disclosed herein;
FIG. 7 is a schematic structural diagram of a model design apparatus provided in the present application;
fig. 8 is a block diagram of an electronic device provided in the present application.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
At present, as the system scale becomes huge and stably operates, the production system faces more problems on the system level than on the service level, such as error data, redundant data, dirty data, and the like. In the case of the business problem related to the data, the operation and maintenance personnel often need to query a background database, judge the occurrence condition according to the characteristic field of the business table, and further issue an operation and maintenance script. Because operation and maintenance personnel are limited, the probability of occurrence of repeatability problems is greatly increased along with the continuous perfection of a service system, so that the maintenance efficiency is affected. Therefore, the method for designing the model can realize automatic replacement of scripts, improves operation and maintenance efficiency, saves labor cost and shortens model training time.
The embodiment of the invention discloses a model design method, which is shown in fig. 1, and comprises the following steps:
step S11: and acquiring target table data, and acquiring a corresponding data set based on the target table data.
In this embodiment, target table data is acquired, and a corresponding data set is acquired based on the target table data. Specifically, target table data corresponding to a target service table is obtained, and the number of creation is performed based on characteristic field data of the target table data, so as to obtain a data set corresponding to the characteristic field data under the condition. It will be appreciated that based on the business acquisition related target table data, the creation is made from the characteristic field data of the target table data, thereby forming several data sets. The method comprises the steps of collecting table data related to service problems of a user under two conditions, for example, collecting three service table data, and carrying out the number of the two conditions according to characteristic fields because characteristic field values of the service problems under the two conditions are inconsistent, wherein the number of the two conditions is 500, and 500 cases of A and 500 cases of B are obtained.
Step S12: and integrating all the data sets by using a preset integration method to obtain a target data set.
In this embodiment, after target table data is obtained and a corresponding data set is obtained based on the target table data, all the data sets are integrated by using a preset integration method to obtain a target data set. Specifically, a plurality of data sets are integrated into one data set, namely target table data, which comprises training data, verification data and test data, through data cleaning processing, normalization processing and labeling processing on all the data sets. It can be understood that, for example, the feature fields of three service tables are extracted to form a one-dimensional array, the larger values are normalized, and 1000 pieces of data (including 500 pieces for the case a and 500 pieces for the case B) are obtained from the processed data set, where the data are divided according to a preset proportion, for example, 650 pieces of data are used as training sets, 300 pieces of data are used as verification sets, and 50 pieces of data are used as test sets.
Step S13: and constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result.
In this embodiment, after integrating all the data sets by using a preset integration method to obtain a target data set, a target network model is constructed based on the target data set, an original DNN (Deep Neural Networks, fully connected neural network) network model, an original naive bayes model and an original CNN (convolutional neural network) network model, so as to predict a received service table to be analyzed by using the target network model, and a preset script replacement operation is performed according to a prediction result. It can be understood that the DNN model parameters obtained by training based on the training data and the verification data of the data set are used for establishing a trained DNN network model for primarily distinguishing the service problems; setting prior probability, and updating posterior probability in real time based on DNN training results of DNN network models after each round of training so as to establish a probability model (namely a naive Bayesian model after training) for distinguishing DNN model accuracy based on the posterior probability; inputting the target data set, the first output data and the second output data into the original CNN network model to execute a preset training mode so as to obtain a current output value; acquiring a second preset loss function and a second preset accuracy calculation formula so as to adjust preset CNN parameters in the original CNN network model based on the current output value, the second preset loss function and the second preset accuracy calculation formula to obtain target CNN parameters; and establishing a trained CNN network model for finally distinguishing service problems based on the target CNN parameters and the original CNN network model. And constructing the target network model based on the trained DNN network model, the trained naive Bayesian model and the trained CNN network model so as to predict the received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result. It can be understood that the input data are used for training the CNN model parameters, and the CNN model parameters in the original CNN network model are preset CNN parameters.
It should be noted that, the data in the scheme not only has text data, but also has logic data and integer data, the data is stored in a structural database, the text data types in the service table are not related by meaning, and most of the trained data fields are logic data and integer data; the target network model is a model in deep learning and comprises a DNN model, a naive Bayes model and a CNN model; the input of the naive Bayes model is from a preliminary prediction result of the DNN model (the naive Bayes acts on an upper model and is not concerned about the original data), different results are generated after the original data are repeatedly input into the DNN model in each round, so that the posterior probability of the naive Bayes is updated in each round, and when the DNN model is optimal, the probability distribution of the naive Bayes is more fit with reality. The DNN model and the CNN model are classification models, and extraction classification is carried out based on the characteristic properties of data; the training data of the invention not only has text data without context semantics, but also has logic data and integer data as characteristic values of the data, and the characteristic values are extracted and then classified by using DNN and CNN models on the premise of the data environment. In the invention, a DNN model is used for carrying out preliminary prediction service conditions, a naive Bayesian model is used for carrying out probability analysis on DNN model results, a final CNN model is used for making decisions according to input data, DNN model output results and naive Bayesian output results, the final prediction results are analyzed and output, and the probability model and the decision model are added to enable the final output to be more fit. The method mainly comprises the steps of distinguishing characteristic fields of database business table data through machine learning and outputting operation and maintenance scripts, and distinguishing can be realized through analyzing field meanings through a labeling method, wherein the labeling is equivalent to manually operating input notes and storing the manually operating input notes in a computer program. In actual life, one service table can have hundreds to thousands of fields, and one key service can be related to a plurality of service tables, the field meaning is analyzed by a labeling method in the prior art to realize identification, but the labeling is equivalent to manually operating and inputting notes to be stored in a computer program, so that the labeling is realized to identify the labor cost; the invention is based on deep learning, only needs big data support, does not need to manually mark the meaning of each field, and realizes intelligent recognition of abnormal business conditions by a model after parameter adjustment training, thereby realizing an intelligent operation and maintenance scheme and reducing labor cost investment; the intelligent operation and maintenance script issuing through the target network model is not only beneficial to saving the cost of labor time and improving the operation and maintenance efficiency, but also provides a new direction for AIOps (Artficial Intelligence for Operations, intelligent operation and maintenance).
It can be seen that the present application provides a model design method, comprising: acquiring target table data, and acquiring a corresponding data set based on the target table data; integrating all the data sets by using a preset integration method to obtain a target data set; and constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result. Therefore, the deep learning target network model is built based on the DNN network model, the naive Bayesian model and the CNN network model, the preliminary prediction service condition is carried out through the DNN model, the naive Bayesian model carries out probability analysis on the result of the DNN model, the CNN model carries out decision making according to the target data set, the DNN model output result and the naive Bayesian output result, and the final prediction result is analyzed and output, so that automatic replacement of a script is realized according to the prediction result, and the fitting property and accuracy are improved by combining a plurality of models; the training time is shortened by optimizing an algorithm in the model; the operation and maintenance efficiency is improved, the labor cost is saved, and the risk of human error is reduced.
Referring to fig. 2, the embodiment of the invention discloses a model design method, and compared with the previous embodiment, the embodiment further describes and optimizes the technical scheme.
Step S21: and acquiring target table data, and acquiring a corresponding data set based on the target table data.
Step S22: and extracting all the characteristic field data of the data set to obtain a current array.
In this embodiment, after the corresponding data set is obtained based on the target table data, the target table data is obtained, and the corresponding data set is obtained based on the target table data. It will be understood that, as shown in fig. 3, the data cleaning operation is performed first, that is, the feature field values of three service tables are extracted and stored in a one-dimensional array, and useless fields are discarded.
Step S23: and executing a preset normalization processing operation on the current array to obtain a normalized array.
In this embodiment, after extracting the feature field data of all the data sets to obtain a current array, a preset normalization processing operation is performed on the current array to obtain a normalized array. It will be appreciated that the data normalization process maps field values to the [0,1] interval range to obtain a normalized array.
Step S24: and performing preset labeling operation on the data in the normalized array to obtain a target array, and acquiring the target data set based on the target array.
In this embodiment, after performing a preset normalization processing operation on the current array to obtain a normalized array, performing a preset labeling operation on data in the normalized array to obtain a target array, and obtaining the target data set based on the target array; wherein the target data set comprises training data, validation data, and test data. It can be understood that since the number of processes is made according to two different processing situations under the service, a column of result values is added after each group of data in the one-dimensional array, and the result values are used for identifying the current processing situation of the data, for example, the result values are respectively set to 0 and 1. And (3) storing 65% of data in the target array as training data, 30% of data as verification data and 5% of data as test data after being scrambled.
Step S25: and constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result.
For the specific content of the steps S21 and S25, reference may be made to the corresponding content disclosed in the foregoing embodiment, and no detailed description is given here.
Therefore, the embodiment of the application obtains the target table data and obtains the corresponding data set based on the target table data; extracting all the characteristic field data of the data set to obtain a current array; performing preset normalization processing operation on the current array to obtain a normalized array; performing preset labeling operation on the data in the normalized array to obtain a target array, and acquiring the target data set based on the target array; constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result to realize automatic replacement of a script, and improving the fitting property and accuracy by combining a plurality of models; the training time is shortened by optimizing an algorithm in the model; the operation and maintenance efficiency is improved, the labor cost is saved, and the risk of human error is reduced.
Referring to fig. 4, the embodiment of the invention discloses a model design method, and compared with the previous embodiment, the embodiment further describes and optimizes the technical scheme.
Step S31: and acquiring target table data, and acquiring a corresponding data set based on the target table data.
Step S32: and integrating all the data sets by using a preset integration method to obtain a target data set.
Step S33: and acquiring a trained DNN network model based on the target data set and the original DNN network model.
In this embodiment, after integrating all the data sets by using a preset integration method to obtain a target data set, a trained DNN network model is obtained based on the target data set and the original DNN network model. Specifically, determining DNN model parameters based on the training data and the validation data in the target dataset; generating a current DNN network model based on the DNN model parameters and the original DNN network model; adjusting DNN model parameters in the current DNN network model through a first preset loss function and a first preset accuracy calculation formula to obtain DNN model final parameters meeting current requirements; and establishing the trained DNN network model for primarily distinguishing service problems based on the DNN model final parameters and the current DNN network model.
It can be understood that the network structure of the original DNN network model is designed and developed based on the fully connected neural network in deep learning, and the network structure is shown in fig. 5, and the main composition structure of the DNN network model includes an input layer, two fully connected network layers of nested algorithms (for example, a fully connected neural network layer with 256 neurons and a fully connected neural network layer with 128 neurons), and an output layer; and further comprises a BN (standardized processing algorithm) layer, a releaserelu (activation function in the neural network) layer, a Dropout (random deletion hidden neuron algorithm in the neural network) layer, and a Sigmoid (activation function in the neural network) layer.Training an original DNN network model based on data of a training set and a verification set, defining a loss function and accuracy to judge a training result of the model, enabling model weight parameters to reach the optimal by adjusting the learning rate in an optimizer, and inputting an output result into a naive Bayesian model and a CNN model. The training flow of the original DNN network model under the network structure is as follows: first, the input layer parameters are two-dimensional data, and the dimensions of the input data in the input layer are [ K, V ] ]Where K is the data size and V is the number of feature fields. For example, training set data is divided into 130 sets of 50 pieces of data each, K has a value of 50, and V has a value of 9 (consisting of 8 feature fields and 1 tag result), wherein the feature fields are arranged from left to right by determinants. Second, a weight ratio v is set for each feature field in the data of the input layer, for example, the input layer data is ((A1, A2, A3, A4, A5, A6, A7, A8, y 0)) and then input into the fully connected neural network layer of 256 neurons, and after passing through the first fully connected neural network layer, the output values of each neuron are as follows:
(wherein
And b is a weight parameter, n is the number of neurons of the fully-connected neural network layer), and then the output value is obtained by normalization processing through the BN layer: />
The data values input to the activation function are all in [0,1 ] through the BN layer]Therefore, the condition that the gradient disappears when the model is trained is eliminated, the influence of the change of input data is reduced, and finally, the output value is activated through a releaserelu layer as follows:
third, the upper layer output value R is input into the fully connected neural network layer of 128 neurons, and is fully connected through the second layerThe output values of each neuron after connecting the neural network layer are as follows: 
(n is the number of neurons of the fully connected neural network layer), and then the output value is obtained by normalization processing through the BN layer: />
The processed data input, leakyRelu layer activation output values are as follows:
Fourth, the upper layer output value R is input into the fully connected neural network layer (i.e., output layer) of 2 neurons, the output values are as follows:
Fifthly, defining a loss function and accuracy, and adopting binary cross entropy as the loss function:
the accuracy in each round of training is: />
(wherein n=50 represents the current number of rounds to be performed, i.e. the number of middle data per group, +.>
As a label result).
Setting total training times and learning rate, and updating weight parameters in the neural network through loss values and accuracy rate in each training, for example, 130 times are needed for each round of training in the invention, 50 pieces of data are input in each training, 50 times are needed for each training, and the steps two to five are repeated continuously until the training is finished.
It can be understood that the fully connected network layer is used for sensing classification, and needs to set input parameters, output parameters, neuron number, weight initialization function and offset value initialization function; the perceptibility and complexity of the model can be increased by increasing the number of hidden layers, i.e. the number of fully connected neural networks. The BN layer, the leakage Relu layer, the Dropout layer and the Sigmoid layer are all nested in the fully-connected neural network layer, wherein the output layer can be regarded as a special fully-connected neural network layer; the BN layer is used for eliminating gradient explosion and quickly converging a model; the releaserelu layer and the Sigmoid layer are used for increasing the nonlinear capability of a fully connected neural network layer (linearization output), eliminating the problem of gradient disappearance and accelerating the speed of gradient descent and counter propagation process; the Dropout layer is used for reducing overfitting, and the deleting rate is required to be set; the size of the output layer is the number of the occurrence situations of the service data.
Step S34: acquiring first output data corresponding to the trained DNN network model, and acquiring a trained naive Bayes model based on the first output data and the original naive Bayes model.
In this embodiment, after acquiring the trained DNN network model based on the target data set and the original DNN network model, first output data corresponding to the trained DNN network model is acquired, and a trained naive bayes model is acquired based on the first output data and the original naive bayes model. Specifically, setting a priori probability, and determining a current posterior probability based on the priori probability; updating the current posterior probability in real time by utilizing the first output data to obtain a target posterior probability; and acquiring the trained naive Bayes model based on the target posterior probability and the original naive Bayes model.
It can be appreciated that the prior probability is set first, the posterior probability is updated in real time based on each round of DNN training results (i.e., the first output data), and a probability model (i.e., a naive bayes model after training) for discriminating the accuracy of the DNN model is established based on the posterior probability. The prior probability is the probability of each service condition and the probability of each result output by the DNN model, the prior probability of the service condition can be customized based on the frequency of service occurrence, for example, the prior probability of the A condition is set to be 0.65, the prior probability of the B condition is set to be 0.35, and the prior probability of each output result of the DNN model is set to be 0.5. The posterior probability is the probability of the DNN model outputting a correct result under the condition of known service, and the result is inconsistent and gradually updated to the optimal trend due to each round of training of the DNN model, so that the posterior probability updated in real time is more fit with the reality.
The naive Bayes model is a probability model, the accurate probability of the preliminary discrimination result of the DNN model is mainly predicted, the accurate probability is used as a reference basis for the subsequent CNN model decision, and the output result of the naive Bayes model is as follows:
wherein the method comprises the steps of
For the prior probability of the traffic situation +.>
The prior probability of the result is output for the DNN model, 
For the posterior probability of the DNN model output result under the known service condition, namely the probability of the DNN model output correctness, under the condition that the DNN model is iteratively updated towards the optimal trend, the DNN model is ++>
The value of (2) will approach 1 indefinitely, so thatThe verification probability of the naive Bayes model is more accurate.
Step S35: and acquiring second output data corresponding to the trained naive Bayesian model, and acquiring the trained CNN network model based on the original CNN network model, the target data set, the first output data and the second output data.
In this embodiment, first output data corresponding to the trained DNN network model is obtained, after the trained naive bayes model is obtained based on the first output data and the original naive bayes model, second output data corresponding to the trained naive bayes model is obtained, and a trained CNN network model for finally distinguishing a service problem is built based on the original CNN network model, the target data set, the first output data and the second output data.
It can be understood that the network structure of the CNN network model is designed and developed based on a convolutional neural network in deep learning, and the main composition structure comprises an input layer, a convolutional layer, a flat layer and an output layer. As shown in fig. 6, the target data set data, DNN network model output data y1, and naive bayes model output data y2 are input to the CNN network model for training, the CNN model serves as a main decision model, and the final discrimination result is output. The model training flow is as follows: first, the input layer parameters are that the dimension of the two-dimensional data input in the input layer is [ K, V ], where K is the data size and V is the number of data (V is composed of the feature field, the DNN model output result and the naive bayes model output result). For example, K has a value of 50, v has a value of 11, and consists of 8 feature fields, 1 label result, DNN model output result, and naive bayes model output result.
Secondly, data ((A1, A2, A3, A4, A5, A6, A7, A8, y0, y1, y 2)) of an input layer is input into a convolutional neural network layer with a convolutional kernel size of 1 and an output channel number of 16, the convolutional layer is composed of the convolutional neural network layer and a releaserelu layer and is mainly used for extracting a data characteristic value, and input parameters, the convolutional kernel size and output parameters need to be set, and the output is as follows:
finally, the output value is activated by the releaserlu layer as follows:
thirdly, inputting an upper layer output value R into a flat layer, wherein the flat layer mainly flattens multi-dimensional data output by a convolution layer into one-dimensional data so as to be subsequently input into a full-connection layer for perception classification, and converts sixteen-dimensional data into one-dimensional data for output, wherein the output value is as follows:
。
fourth, the output layer is mainly composed of a fully connected neural network layer and a Sigmoid layer of two neurons, and finally, a prediction result of the service condition is output. The upper layer output value Z is input into a fully connected neural network layer of 2 neurons, and the output value is as follows:
then activating the output final value through the Sigmoid layer, and the formula is as follows:
fifthly, defining a loss function and accuracy, and adopting binary cross entropy as the loss function:
the accuracy in each round of training is as follows:
Because the DNN model sets the total training times, repeated setting is not needed in the CNN model training, only the learning rate is needed to be set, the weight parameters in the neural network are updated through the loss value and the accuracy rate in each training, and the steps two to five are repeated until the training is finished.
Step S36: and constructing the target network model based on the trained DNN network model, the trained naive Bayesian model and the trained CNN network model so as to predict the received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result.
In this embodiment, the target network model is constructed based on the trained DNN network model, the trained naive bayes model, and the trained CNN network model, so as to predict the received service table to be analyzed by using the target network model, and perform a preset script replacement operation according to the prediction result. Specifically, a target network model is built based on the target data set, an original DNN network model, an original naive bayes model and an original CNN network model, so that the received business table to be analyzed is predicted by using the trained DNN network model, the trained naive bayes model and the trained CNN network model, a corresponding target script is automatically obtained according to the prediction result, and the current script is replaced by the target script through a preset script replacing method. And inputting the data of the test set into a trained target network model to predict service problems, judging script content based on a prediction result of the model, and automatically replacing fixed script content according to a field value of a service table by a preset script replacement code to realize automatic service operation and maintenance script.
According to the method and the device, service table data related to a user are required to be input, and the replacement of the fixed operation and maintenance script is realized after the problem of the service is predicted through the target model. By the mode, the problems of business judgment and manual replacement of operation and maintenance scripts are not needed, the situation that a great deal of time is spent to verify the alignment of the replacement script conditions when the scripts are manually issued is avoided, the investment of manual time is reduced, the speed of issuing the operation and maintenance scripts is improved, and the labor cost is saved. The training set data are used for training model parameters in the training process, and the verification set data are used for feeding back the prediction condition of the current model parameters on the real data, so that the model weight can be changed in a better and more proper direction. The influence duty ratio of the model to the input data is realized by setting the weight duty ratio of the characteristic field, a BN algorithm is added in the model structure and mixed algorithm is used for activation, different activation functions are used for different neural network layers, and the binary cross entropy is combined, so that the time for training the model is shortened and the overfitting is reduced. And carrying out preliminary prediction service conditions by using the DNN model, carrying out probability analysis on DNN model results by using a naive Bayesian model, deciding by using a final CNN model according to input data, DNN model output results and naive Bayesian output results, analyzing and outputting final prediction results, and adding a probability model and a decision model to enable final output to be more fit. In addition, the scheme can realize corresponding effects through adding and subtracting the number of hidden layers or adding and subtracting an algorithm, and the whole model framework of deep learning is not limited to a DNN model, a naive Bayesian model and a CNN model.
For the specific content of the steps S31 and S32, reference may be made to the corresponding content disclosed in the foregoing embodiment, and no detailed description is given here.
Therefore, according to the embodiment of the application, the target table data are obtained, and the corresponding data set is obtained based on the target table data; integrating all the data sets by using a preset integration method to obtain a target data set; acquiring a trained DNN network model based on the target data set and the original DNN network model; acquiring first output data corresponding to the trained DNN network model, and acquiring a trained naive Bayes model based on the first output data and the original naive Bayes model; acquiring second output data corresponding to the trained naive Bayes model, and acquiring a trained CNN network model based on an original CNN network model, the target data set, the first output data and the second output data; constructing the target network model based on the trained DNN network model, the trained naive Bayesian model and the trained CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result to realize automatic replacement of a script, wherein the fitting property and accuracy are increased by combining a plurality of models; the training time is shortened by optimizing an algorithm in the model; the operation and maintenance efficiency is improved, the labor cost is saved, and the risk of human error is reduced.
Referring to fig. 7, the embodiment of the application further correspondingly discloses a model design device, which includes:
a first data set obtaining module 11, configured to obtain target table data, and obtain a corresponding data set based on the target table data;
a second data set obtaining module 12, configured to integrate all the data sets by using a preset integration method to obtain a target data set;
the model construction module 13 is configured to construct a target network model based on the target data set, the original DNN network model, the original naive bayes model, and the original CNN network model, so as to predict the received service table to be analyzed by using the target network model, and perform a preset script replacement operation according to the prediction result.
As can be seen, the present application includes: acquiring target table data, and acquiring a corresponding data set based on the target table data; integrating all the data sets by using a preset integration method to obtain a target data set; and constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result. Therefore, the deep learning target network model is built based on the DNN network model, the naive Bayesian model and the CNN network model, the preliminary prediction service condition is carried out through the DNN model, the naive Bayesian model carries out probability analysis on the result of the DNN model, the CNN model carries out decision making according to the target data set, the DNN model output result and the naive Bayesian output result, and the final prediction result is analyzed and output, so that automatic replacement of a script is realized according to the prediction result, and the fitting property and accuracy are improved by combining a plurality of models; the training time is shortened by optimizing an algorithm in the model; the operation and maintenance efficiency is improved, the labor cost is saved, and the risk of human error is reduced.
In some specific embodiments, the first data set obtaining module 11 specifically includes:
the target table data acquisition unit is used for acquiring target table data corresponding to the target service table;
and the data set acquisition unit is used for carrying out the number generation based on the characteristic field data of the target table data so as to obtain the data set corresponding to the characteristic field data under the condition.
In some specific embodiments, the second data set acquiring module 12 specifically includes:
the current array acquisition unit is used for extracting the characteristic field data of all the data sets to obtain a current array;
the normalization unit is used for executing preset normalization processing operation on the current array to obtain a normalized array;
the marking unit is used for executing preset marking operation on the data in the normalized array to obtain a target array;
a target data set acquisition unit configured to acquire the target data set based on the target array; wherein the target data set comprises training data, validation data, and test data.
In some embodiments, the model building module 13 specifically includes:
a DNN model parameter determination unit configured to determine DNN model parameters based on the training data and the verification data in the target data set;
A current DNN network model generating unit, configured to generate a current DNN network model based on the DNN model parameters and the original DNN network model;
the final parameter obtaining unit is used for adjusting the DNN model parameters in the current DNN network model through a first preset loss function and a first preset accuracy calculation formula so as to obtain DNN model final parameters meeting the current requirements;
a trained DNN network model obtaining unit, configured to obtain the trained DNN network model based on the DNN model final parameter and the current DNN network model;
the first output data acquisition unit is used for acquiring first output data corresponding to the trained DNN network model;
the prior probability setting unit is used for setting prior probability;
a current posterior probability determination unit configured to determine a current posterior probability based on the prior probability;
a target posterior probability obtaining unit, configured to update the current posterior probability with the first output data to obtain a target posterior probability;
a trained naive bayes model obtaining unit, configured to obtain the trained naive bayes model based on the target posterior probability and the original naive bayes model;
The second output data acquisition unit is used for acquiring second output data corresponding to the trained naive Bayes model;
the current output value obtaining unit is used for inputting the target data set, the first output data and the second output data into the original CNN network model to execute a preset training mode so as to obtain a current output value;
the target CNN parameter acquisition unit is used for acquiring a second preset loss function and a second preset accuracy calculation formula so as to adjust preset CNN parameters in the original CNN network model based on the current output value, the second preset loss function and the second preset accuracy calculation formula to obtain target CNN parameters;
a trained CNN network model obtaining unit, configured to obtain the trained CNN network model based on the target CNN parameter and the original CNN network model;
and the script replacing unit is used for constructing the target network model based on the target data set, the trained DNN network model, the trained naive Bayesian model and the trained CNN network model so as to predict the received service table to be analyzed, automatically acquiring a corresponding target script according to the prediction result, and replacing the current script with the target script by a preset script replacing method.
Further, the embodiment of the application also provides electronic equipment. Fig. 8 is a block diagram of an electronic device 20, according to an exemplary embodiment, and the contents of the diagram should not be construed as limiting the scope of use of the present application in any way.
Fig. 8 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein the memory 22 is used for storing a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the model design method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be specifically an electronic computer.
In this embodiment, the power supply 23 is configured to provide an operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol to be followed is any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input/output interface 25 is used for acquiring external input data or outputting external output data, and the specific interface type thereof may be selected according to the specific application requirement, which is not limited herein.
The memory 22 may be a carrier for storing resources, such as a read-only memory, a random access memory, a magnetic disk, or an optical disk, and the resources stored thereon may include an operating system 221, a computer program 222, and the like, and the storage may be temporary storage or permanent storage.
The operating system 221 is used for managing and controlling various hardware devices on the electronic device 20 and computer programs 222, which may be Windows Server, netware, unix, linux, etc. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the model design method performed by the electronic device 20 as disclosed in any of the previous embodiments.
Further, the embodiment of the application also discloses a storage medium, and the storage medium stores a computer program, and when the computer program is loaded and executed by a processor, the steps of the model design method disclosed in any one of the previous embodiments are realized.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing has outlined a detailed description of the method, apparatus, device and storage medium of model design, wherein specific examples are provided herein to illustrate the principles and embodiments of the present invention, and the above examples are provided to assist in understanding the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (7)
1. A model design method, comprising:
acquiring target table data, and acquiring a corresponding data set based on the target table data;
integrating all the data sets by using a preset integration method to obtain a target data set;
constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received business table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result;
the obtaining the target table data, and obtaining the corresponding data set based on the target table data includes:
acquiring target table data corresponding to a target service table, and creating numbers based on characteristic field data of the target table data to obtain a data set corresponding to the characteristic field data under the condition;
the integrating all the data sets by using a preset integration method to obtain a target data set includes:
extracting all the characteristic field data of the data set to obtain a current array;
performing preset normalization processing operation on the current array to obtain a normalized array;
Performing preset labeling operation on the data in the normalized array to obtain a target array, and acquiring the target data set based on the target array; wherein the target data set comprises training data, verification data and test data;
the constructing a target network model based on the target data set, the original DNN network model, the original naive bayes model, and the original CNN network model includes:
acquiring a trained DNN network model based on the target data set and the original DNN network model;
acquiring first output data corresponding to the trained DNN network model, and acquiring a trained naive Bayes model based on the first output data and the original naive Bayes model;
acquiring second output data corresponding to the trained naive Bayes model, and acquiring a trained CNN network model based on an original CNN network model, the target data set, the first output data and the second output data;
and constructing the target network model based on the trained DNN network model, the trained naive Bayesian model and the trained CNN network model.
2. The model design method according to claim 1, wherein the acquiring a trained DNN network model based on the target data set and the original DNN network model comprises:
Determining DNN model parameters based on the training data and the validation data in the target dataset;
generating a current DNN network model based on the DNN model parameters and the original DNN network model;
adjusting DNN model parameters in the current DNN network model through a first preset loss function and a first preset accuracy calculation formula to obtain DNN model final parameters meeting current requirements;
and acquiring the trained DNN network model based on the DNN model final parameters and the current DNN network model.
3. The model design method according to claim 1, wherein the acquiring a trained naive bayes model based on the first output data and the original naive bayes model includes:
setting prior probability and determining current posterior probability based on the prior probability;
updating the current posterior probability by using the first output data to obtain a target posterior probability;
and acquiring the trained naive Bayes model based on the target posterior probability and the original naive Bayes model.
4. A model design method according to any one of claims 1 to 3, wherein the acquiring a trained CNN network model based on the original CNN network model, the target data set, the first output data, and the second output data comprises:
Inputting the target data set, the first output data and the second output data into the original CNN network model to execute a preset training mode so as to obtain a current output value;
acquiring a second preset loss function and a second preset accuracy calculation formula so as to adjust preset CNN parameters in the original CNN network model based on the current output value, the second preset loss function and the second preset accuracy calculation formula to obtain target CNN parameters;
acquiring the trained CNN network model based on the target CNN parameters and the original CNN network model;
correspondingly, the constructing a target network model based on the target data set, the original DNN network model, the original naive bayes model and the original CNN network model so as to predict the received service table to be analyzed by using the target network model, and performing a preset script replacement operation according to a prediction result, including:
and constructing the target network model based on the target data set, the trained DNN network model, the trained naive Bayesian model and the trained CNN network model so as to predict the received service table to be analyzed, automatically acquiring a corresponding target script according to the prediction result, and replacing the current script with the target script by a preset script replacement method.
5. A model designing apparatus, comprising:
the first data set acquisition module is used for acquiring target table data and acquiring a corresponding data set based on the target table data;
the second data set acquisition module is used for integrating all the data sets by utilizing a preset integration method so as to obtain a target data set;
the model construction module is used for constructing a target network model based on the target data set, the original DNN network model, the original naive Bayesian model and the original CNN network model so as to predict a received service table to be analyzed by using the target network model and perform preset script replacement operation according to a prediction result;
wherein the device is further for: acquiring target table data corresponding to a target service table, and creating numbers based on characteristic field data of the target table data to obtain a data set corresponding to the characteristic field data under the condition; extracting all the characteristic field data of the data set to obtain a current array; performing preset normalization processing operation on the current array to obtain a normalized array; performing preset labeling operation on the data in the normalized array to obtain a target array, and acquiring the target data set based on the target array; wherein the target data set comprises training data, verification data and test data; acquiring a trained DNN network model based on the target data set and the original DNN network model; acquiring first output data corresponding to the trained DNN network model, and acquiring a trained naive Bayes model based on the first output data and the original naive Bayes model; acquiring second output data corresponding to the trained naive Bayes model, and acquiring a trained CNN network model based on an original CNN network model, the target data set, the first output data and the second output data; and constructing the target network model based on the trained DNN network model, the trained naive Bayesian model and the trained CNN network model.
6. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the model design method as claimed in any one of claims 1 to 4.
7. A computer-readable storage medium storing a computer program; wherein the computer program, when executed by a processor, implements the model design method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310000603.0A CN115713099B (en) | 2023-01-03 | 2023-01-03 | Model design method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310000603.0A CN115713099B (en) | 2023-01-03 | 2023-01-03 | Model design method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115713099A CN115713099A (en) | 2023-02-24 |
| CN115713099B true CN115713099B (en) | 2023-05-09 |
Family
ID=85236146
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310000603.0A Active CN115713099B (en) | 2023-01-03 | 2023-01-03 | Model design method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115713099B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021007812A1 (en) * | 2019-07-17 | 2021-01-21 | 深圳大学 | Deep neural network hyperparameter optimization method, electronic device and storage medium |
| CN114943372A (en) * | 2022-05-09 | 2022-08-26 | 国网浙江省电力有限公司宁波供电公司 | Method and device for predicting life of proton exchange membrane based on Bayesian recurrent neural network |
| CN115169490A (en) * | 2022-07-25 | 2022-10-11 | 济南浪潮数据技术有限公司 | Log classification method, device and equipment and computer readable storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11461650B2 (en) * | 2020-03-26 | 2022-10-04 | Fujitsu Limited | Validation of deep neural network (DNN) prediction based on pre-trained classifier |
| CN112017061A (en) * | 2020-07-15 | 2020-12-01 | 北京淇瑀信息科技有限公司 | Financial risk prediction method and device based on Bayesian deep learning and electronic equipment |
| CN114862763A (en) * | 2022-04-13 | 2022-08-05 | 华南理工大学 | Gastric cancer pathological section image segmentation prediction method based on EfficientNet |
-
2023
- 2023-01-03 CN CN202310000603.0A patent/CN115713099B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021007812A1 (en) * | 2019-07-17 | 2021-01-21 | 深圳大学 | Deep neural network hyperparameter optimization method, electronic device and storage medium |
| CN114943372A (en) * | 2022-05-09 | 2022-08-26 | 国网浙江省电力有限公司宁波供电公司 | Method and device for predicting life of proton exchange membrane based on Bayesian recurrent neural network |
| CN115169490A (en) * | 2022-07-25 | 2022-10-11 | 济南浪潮数据技术有限公司 | Log classification method, device and equipment and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115713099A (en) | 2023-02-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108089843B (en) | Intelligent bank enterprise-level demand management system | |
| CN112085565B (en) | Deep learning-based information recommendation method, device, equipment and storage medium | |
| CN111897528B (en) | Low-code platform for enterprise online education | |
| CN110175235A (en) | Intelligence commodity tax sorting code number method and system neural network based | |
| CN115713099B (en) | Model design method, device, equipment and storage medium | |
| CN113221516B (en) | Method and system for assisting in editing document | |
| CN113792187A (en) | Crowd-sourcing software development contribution quality assessment method, device, equipment and medium | |
| CN113190562A (en) | Report generation method and device and electronic equipment | |
| CN111400413A (en) | Method and system for determining category of knowledge points in knowledge base | |
| CN111813837A (en) | Method for intelligently detecting data quality | |
| US20200327319A1 (en) | Agency Notice Processing System | |
| Wang | Retracted: Multi‐data multiple gray clustering analysis based on layered mining for ubiquitous clouds and social internet of things | |
| CN110458383A (en) | Demand handles implementation method, device and the computer equipment of serviceization, storage medium | |
| Zhang et al. | RandAlign: A Parameter-Free Method for Regularizing Graph Convolutional Networks | |
| CN113642727B (en) | Training method of neural network model and processing method and device of multimedia information | |
| CN117573904B (en) | Multimedia teaching resource knowledge graph generation method and system based on recognition analysis | |
| US11928128B2 (en) | Construction of a meta-database from autonomously scanned disparate and heterogeneous sources | |
| CN111881190B (en) | Key data mining system based on customer portrait | |
| CN108763462A (en) | Update method, equipment and the computer readable storage medium of parallel statement library | |
| CN114117022B (en) | FAQ similarity problem generation method and system | |
| US20230368013A1 (en) | Accelerated model training from disparate and heterogeneous sources using a meta-database | |
| CN116050508A (en) | Neural network training method and device | |
| CN117291708A (en) | Enterprise credit assessment method, system, equipment and medium | |
| Singh et al. | Fuzzy-based interactive method for solution of bi-and multi-level programming problems | |
| CN117635243A (en) | Intelligent software cost assessment method and system for enabling large language model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |