CN104504442A - Neural network optimization method - Google Patents
Neural network optimization method Download PDFInfo
- Publication number
- CN104504442A CN104504442A CN201410843528.5A CN201410843528A CN104504442A CN 104504442 A CN104504442 A CN 104504442A CN 201410843528 A CN201410843528 A CN 201410843528A CN 104504442 A CN104504442 A CN 104504442A
- Authority
- CN
- China
- Prior art keywords
- mrow
- msub
- hidden layer
- mtr
- mtd
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 95
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000005457 optimization Methods 0.000 title claims abstract description 27
- 230000002068 genetic effect Effects 0.000 claims abstract description 16
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 15
- 210000002569 neuron Anatomy 0.000 claims description 99
- 239000013598 vector Substances 0.000 claims description 34
- 238000004364 calculation method Methods 0.000 claims description 17
- 230000001186 cumulative effect Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 5
- 230000005284 excitation Effects 0.000 claims description 4
- 230000035772 mutation Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 2
- 230000006870 function Effects 0.000 description 10
- 230000008569 process Effects 0.000 description 6
- 238000002474 experimental method Methods 0.000 description 3
- 101100134058 Caenorhabditis elegans nth-1 gene Proteins 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 230000001488 breeding effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000004576 sand Substances 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a neural network optimization method. A genetic algorithm is adopted for determining optimal neural network weight and hidden layer output parameter values, the hidden layer output parameter values are in correspondence to hidden layers in number, and the number of the hidden layers can be determined according to the hidden layer output parameter values and changes along with the hidden layer output parameter values, and a neural network is constructed according to the weight and the number of the hidden layers. Due to the fact that weight and the number of the hidden layer are optimal, output result predication errors can be greatly reduced, and predication performance of the whole neural network can be improved to a great extent.
Description
Technical Field
The invention relates to an artificial neural network, in particular to a neural network optimization method.
Background
In recent years, Artificial Neural Networks (ANN), which have been widely used in various fields such as intelligent control, robotics, pattern recognition, computer vision, production process optimization, and information processing, have provided a new approach to simulate complex and difficult-to-define problems. Research results show that if the network structure of the ANN is too large, the generalization is poor, and conversely, if the network structure is too small, the learning capability is low. And the weights and thresholds in the network directly influence the training and recognition performance of the neural network. Therefore, how to optimize the ANN network structure is critical in order to improve the performance of the artificial neural network as much as possible.
The conventional ANN network structure optimization methods are various, the number of nodes of an input layer is determined mainly by using a principal component analysis method, so that the problem can be solved to a certain extent, but when sample data is large, the sample set does not contain the characteristics of all samples, so that a large error occurs in a predicted result.
Disclosure of Invention
Therefore, it is necessary to provide a neural network optimization method capable of reducing errors for solving the problem of large error of the neural network prediction result.
A neural network optimization method, comprising the steps of:
encoding the weight value of the neural network and the output parameter value of the hidden layer to form a genetic algorithm individual;
randomly generating an initial population comprising a plurality of said individuals;
calculating the fitness of each individual in the population;
judging whether a stop condition is met;
if the judgment result does not meet the requirement, the following steps are executed:
selecting the individual;
randomly crossing the selected individuals;
performing mutation on the individuals after crossing;
generating a new population of new individuals;
counting the number of iterations, and returning to continue to execute the step of calculating the fitness of each individual in the population;
if the stop condition is judged to be met, the following steps are executed:
selecting the individual with the greatest fitness;
decoding the individuals to obtain weights of the neural network and hidden layer output parameters;
and constructing a neural network according to the weight and the hidden layer output parameter value.
According to the neural network optimization method, the optimal neural network weight and hidden layer output parameters are determined through a genetic algorithm, the number of hidden layer layers is obtained according to the hidden layer output parameter values, the neural network is initialized through the weight and the number of hidden layer layers, the weight and the number of hidden layer layers are better, in the process of predicting through the neural network, the error of an output result is greatly reduced, and the performance of the whole neural network is greatly improved.
Drawings
FIG. 1 is a flow diagram of a neural network optimization method according to one embodiment;
fig. 2 is a flowchart of selecting an individual in a neural network optimization method according to another embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
The genetic algorithm is a heuristic search algorithm, and aims to solve the problems that a biological evolution process is simulated, individuals of the next generation are generated through operations such as copying, crossing, mutation and the like, the individuals with low fitness are gradually eliminated, the individuals with high fitness are increased, the chances of the individuals with high fitness being inherited to the next generation are large, and the individuals with high fitness are possibly evolved. By combining the genetic algorithm and the neural network, the weight and the hidden layer number of the neural network can be regarded as an individual after being coded, the process of determining the weight and the hidden layer number is the process of searching for an approximately optimal individual, and when the individual with high fitness is finally obtained, the weight and the hidden layer number are better.
Referring to fig. 1, a neural network optimization method includes the following steps:
s100: and coding the weight value of the neural network and the output parameter value of the hidden layer to form a genetic algorithm individual.
When a neural network is established, the neural network needs to be initialized according to weights connected between neurons in each layer in the neural network and hidden layer numbers, and the selection of the weights and the hidden layer numbers plays a crucial role in the prediction output of the subsequent neural network, so that the weights and the hidden layer output parameter values of the neural network are selected through a genetic algorithm, and a better neural network weight and a better hidden layer number are obtained to construct the neural network. In the process of finding the optimal solution, the genetic algorithm can be used only when the individual is coded into a character string, and the common code is binary code, namely, the problem solution is coded into a binary array. The weight of the neural network and the output parameter value of the hidden layer are encoded into an individual, and the optimal individual is selected through a genetic algorithm, namely the optimal selection of the weight of the neural network and the output parameter value of the hidden layer.
S200: an initial population comprising a plurality of individuals is randomly generated.
The population contains a certain number of individuals, which are randomly generated. The genetic algorithm is to optimally select individuals in a population to obtain the optimal individuals.
S300: and calculating the fitness of each individual in the population.
In the genetic algorithm, the fitness of each individual is used for evaluating the degree of goodness of each individual so as to determine the genetic chance of the individual.
S400: and judging whether the stop condition is met.
If the judgment result does not meet the requirement, the following steps are executed:
s410: the individual is selected.
Selecting a plurality of individuals in the population, and carrying out the next operation on the individuals with more selected times, wherein the probability of the individual selection is in direct proportion to the fitness of the individual, so that the individuals with more selected times indicate that the fitness is higher, and the quantity of the individuals copied to the next generation population is determined.
S420: and randomly crossing the selected individuals.
After crossing the individuals, different individuals are formed. I.e. the weights and hidden layer output parameter values in an individual may change.
S430: and (5) carrying out variation on the crossed individuals.
And (5) carrying out mutation on the individuals obtained by crossing to obtain new individuals. And obtaining an individual formed by the new weight value and the hidden layer output parameter value.
S440: a new population of new individuals is generated.
Since the population is composed of individuals, when new individuals are generated, the population is subsequently updated to a new population.
S450: and counting the number of iterations, and returning to continue to execute the step S300 of calculating the fitness of each individual in the population.
The number of statistical iterations is prepared for the stopping condition, and the fitness of each individual in the new population is calculated, and at the moment, the individual is updated, and the calculated fitness is changed accordingly. Before the weight value of the neural network and the output parameter value of the hidden layer are coded to form a genetic algorithm individual, the iteration times are cleared.
If the stop condition is judged to be met, the following steps are executed:
s500: the individual with the greatest fitness is selected.
S600: and decoding the individuals to obtain the weight of the neural network and the output parameter value of the hidden layer.
S700: and constructing a neural network according to the weight and the hidden layer output parameter value.
When the fitness of the individual meets the stopping condition, the individual with the maximum fitness is selected from the individuals in the population and decoded to obtain the optimal weight of the neural network and the output parameter value of the hidden layer, the number of the hidden layers of the neural network is determined according to the output parameter value of the hidden layer, and after the connection parameter between the optimal hidden layers is determined, the number of the hidden layers of the neural network is determined accordingly, so that the neural network is initialized according to the weight and the number of the hidden layers.
According to the neural network optimization method, the optimal neural network weight and the hidden layer output parameter value are determined through a genetic algorithm, the hidden layer output parameter value corresponds to the hidden layer number, the number of layers of the hidden layer can be determined according to the hidden layer output parameter value, when the hidden layer output parameter value changes, the number of layers of the hidden layer changes, the neural network constructs the neural network through the weight and the hidden layer number, because the weight and the hidden layer number are better, the error of a predicted output result is greatly reduced, and the whole prediction performance is greatly improved.
In one embodiment, the stopping condition includes that the number of iterations exceeds a maximum number of iterations or that the fitness of the individual exceeds a threshold.
And when the fitness of the individual exceeds a preset threshold value, considering that the optimal individual is obtained through a genetic algorithm, obtaining the optimal weight and the number of hidden layer layers according to the weight and the hidden layer output parameter value in the optimal individual, and constructing the optimal neural network. However, when the fitness of the individual does not reach the preset threshold value all the time, the individual updating is stopped through the maximum iteration number, so that the updating process is prevented from continuing all the time.
In one embodiment, the step S700 of constructing the neural network according to the weights and the hidden layer output parameter values includes the following steps:
s800: and predicting through a neural network, and outputting a prediction result.
After the neural network is constructed, the neural network is tested through the test sample, and a prediction result is obtained.
In one embodiment, the step S300 of calculating the fitness of each individual in the population includes the steps of:
and respectively calculating output vectors of the 1 st hidden layer to the Nth hidden layer.
The output of the output layer neurons is computed from the output vectors of the nth hidden layer.
From the output of the output layer neurons, an output vector of the output layer is calculated.
An objective function is calculated.
And calculating the fitness of the individual according to the objective function.
Wherein, N is the number of hidden layers.
The neural network comprises an input layer, a hidden layer and an output layer, the number of hidden layer layers is determined according to the output parameter value of the hidden layer, when the individual is updated by the neural network optimization method, the weight value of the neural network and the output parameter value of the hidden layer are updated accordingly, and the number of hidden layer layers is also updated.
In one embodiment, the step of calculating the output vectors of the 1 st hidden layer to the Nth hidden layer respectively comprises the following steps:
and calculating the output of the hidden layer neuron, wherein the calculation formula is as follows:
wherein,represents the output of the jth neuron of the 1 st hidden layer,represents the weight, x, between the kth neuron of the input layer and the jth neuron of the 1 st hidden layerkRepresents the input of the kth neuron of the input layer of the neural network,represents the weight between the kth neuron of the i-1 th hidden layer and the jth neuron of the i hidden layer,represents the output of the jth neuron of the ith hidden layer,represents the output of the kth neuron of the i-1 hidden layer, n0Number of neurons of the input layer, ni-1Is the nerve of the i-1 st hidden layerThe number of elements, f, is the activation function.
And calculating an output vector of the 1 st hidden layer by the following formula:
wherein n is1Number of neurons in hidden layer 1, h1Represented as the output vector of the 1 st hidden layer,represents the output of the 1 st neuron of the 1 st hidden layer,represents the output of the jth neuron of the 1 st hidden layer,n-th of a 1 st hidden layer1The output of each of the plurality of neurons,representing the weight between the kth neuron of the input layer and the 1 st neuron of the 1 st hidden layer,representing the weight between the kth neuron of the input layer and the jth neuron of the 1 st hidden layer,representing the kth neuron of the input layer with the nth of the 1 st hidden layer1Weights between individual neurons.
And calculating an output vector of the ith hidden layer, wherein the calculation formula is as follows:
wherein h isiIs the output vector of the ith hidden layer, hi-1Is the output vector of the i-1 th hidden layer, ui-1Represents the ith-1 hidden layer and the ith hidden layer output parameter values,representing the weight between the kth neuron of the i-1 th hidden layer and the 1 st neuron of the i hidden layer,represents the weight between the kth neuron of the i-1 th hidden layer and the jth neuron of the i hidden layer,representing the kth neuron of the i-1 th hidden layer and the nth of the i hidden layeriWeight between individual neurons, niThe number of neurons in the ith hidden layer.
And calculating an output vector of the Nth hidden layer, wherein the calculation formula is as follows:
uN-1representing the output parameter value of the Nth hidden layer, N being the number of layers of the hidden layer, hNNeuron output vector, h, representing the Nth hidden layerN-1Representing the neuron output vector of the N-1 hidden layer,represents the output of the kth neuron of the nth-1 hidden layer,representing the weight between the kth neuron of the N-1 th hidden layer and the 1 st neuron of the N hidden layer,representing the weight between the kth neuron of the nth-1 hidden layer and the jth neuron of the nth hidden layer,representing the kth neuron of the N-1 th hidden layer and the nth of the N hidden layerNWeight between individual neurons, nNNumber of neurons representing the Nth hidden layer, NN-1The number of neurons in the N-1 th hidden layer is shown.
The calculation formula of the step of calculating the output of the output layer neurons according to the output vector of the nth hidden layer is as follows:
wherein,representing the weight between the kth neuron of the nth hidden layer and the jth neuron of the output layer,an output representing a kth neuron of an Nth hidden layer; y isjRepresenting the output of the jth neuron of the output layer.
According to the output of the output layer neuron, the calculation formula of the step of calculating the output vector of the output layer is as follows:
f (U, W, X) denotes an output layer output vector, X denotes an input vector of an input layer, W denotes a neural network weight matrix, U denotes a hidden layer output parameter value matrix, and U ═1,...,ui,...,uN-1]，u1Representing values of the 2 nd hidden layer output parameter, uiRepresenting the value of the i +1 th hidden layer output parameterWherein u is1、uiAnd uN-1Is 0 or 1; y is1Representing the output of the 1 st neuron of the output layer, y2Represents the output of the 1 st neuron of the output layer, nN+1Indicates the number of output layer neurons.
The calculation formula of the step of calculating the objective function is as follows:
R＝||F(U,X,W)-d||2；
wherein R is the objective function and d is the expected output vector of the neural network.
According to the objective function, the calculation formula of the step of calculating the fitness of the individual is as follows:
Fit＝M-R；
where Fit is the fitness of the individual, and M is a number greater than 0.
The hidden layer output parameter value of 0 indicates that the outputs of the two hidden layers are the same, that is, the outputs of the two adjacent hidden layers are not changed, so that the two hidden layers can be merged, that is, the number of layers of the hidden layers is reduced, and the purpose of updating the number of layers of the hidden layers is achieved. The output of two hidden layers is different as indicated by 1 in the hidden layer output parameter values, and the output of the next hidden layer needs to be calculated through the weight and the hidden layer output, which indicates that the two hidden layers cannot be combined and the number of the hidden layers is unchanged. If the hidden layer output parameter value matrix U contains b zeros, the number of hidden layer layers of the neural network is b +1, and the number of hidden layer layers is updated by updating the hidden layer output parameter values.
And updating the individuals, namely updating the weight of the neural network and the connection parameters between the hidden layers, so that the weight of the neural network and the number of the hidden layers are updated, then carrying out the next round of fitness calculation on new individuals inherited into the next population, and repeating the steps until the individuals meeting the stop condition are found or the iteration times reach the maximum value.
The fitness of an individual is a parameter used for measuring the degree of goodness and badness of the individual in the population, and depends on the objective function value of the individual, the greater the fitness of the individual is, the greater the adaptive capacity is, the greater the probability of being inherited into the next generation population is, the greater the fitness is, the smaller the objective function value is, and the smaller the prediction error of the neural network is.
In one embodiment, the excitation function f is specifically:
where x is the argument of the excitation function f.
Referring to fig. 2, in one embodiment, the step S410 of selecting an individual includes the following steps:
s4101: m is initialized to 0.
S4102: the probability of each individual being inherited into the next generation population is calculated.
The calculation formula is specifically as follows:
wherein p iszThe probability of the z th individual being inherited to the next generation population is shown, T is the number of individuals, Fit (T) is the fitness of the T th individual, Fit (z) is the fitness of the z th individual, z is a positive integer which is greater than or equal to 1 and less than or equal to T, and T is a positive integer which is greater than or equal to 1 and less than or equal to T.
S4103: the cumulative probability for each individual is calculated.
The calculation formula is specifically as follows:
wherein q istIs the cumulative probability of the tth individual.
S4104: and judging whether m is larger than a preset value.
If so, step S420 of randomly interleaving the selected individuals is performed.
If the value is less than the preset value, the following steps are executed:
s4105: randomly generating a random number r between 0 and 1;
s4106: judging whether r is smaller than the cumulative probability q of the 1 st individual1。
If the value is less than the preset value, the following steps are executed:
s4107: and selecting the 1 st individual, increasing the selected times of the 1 st individual by 1, increasing m by 1, and returning to the step S4104 of continuously judging whether m is larger than a preset value.
If the r is judged to be more than or equal to the cumulative probability q of the 1 st individual1Then, the following steps are executed:
s4108: when q iss-1<r≤qsAnd if so, selecting the S-th individual, increasing the selected times of the S-th individual by 1, increasing m by 1, and returning to the step of continuously judging whether m is greater than the preset value S4104.
Wherein s is a positive integer of 2 or more and T or less.
m represents the number of rounds of selection of individuals, for example, if 5 rounds of selection are performed on an individual, the value of m is 5, and the selection of the individual is performed in order to select a good individual from the population and make the good individual have an opportunity to be a parent generation as a next generation for breeding offspring, and the individual is selected according to the fitness by adopting the roulette method, so that the better individual in the current population is selected. The more times an individual is selected, the higher the fitness of the individual, and the greater the probability that the individual will be inherited to the next generation.
In one embodiment, the step of randomly crossing the selected individuals specifically comprises:
and randomly crossing the individuals with the selected times larger than the preset times.
After the individuals are selected, the number of times of selecting each individual is obtained, some individuals can be selected for multiple times, and some individuals can not be selected once, so that when the number of times of selecting the individuals exceeds a preset number, the individuals are taken as the individuals capable of being crossed, and finally all the individuals with the number of times of selecting the individuals exceeding the preset number are crossed.
In one embodiment, the step S420 of randomly crossing the selected individuals comprises the following steps:
randomly pairing the individuals;
randomly setting the positions of the cross points;
one individual of the paired individuals is divided into a first part and a second part by taking a cross point as a boundary point;
the other individual of the paired individuals is divided into a third part and a fourth part by a dividing point corresponding to the position of the cross point;
exchanging the first portion with the third portion, or the second portion with the fourth portion;
and obtaining the crossed individuals.
In one embodiment, the step S430 of mutating the crossed individuals includes the following steps:
selecting variant locus points of individuals;
and (5) negating the value of the variation position point of the individual.
After the weight value and the hidden layer output parameter value of the neural network are optimally selected by using the neural network optimization method, the number of the hidden layer is obtained by using the hidden layer output parameter value, the neural network is initialized according to the weight value and the hidden layer number of the neural network, and the neural network is used for carrying out network prediction to obtain a prediction result. The performance of the neural network is evaluated by using a Root Mean Square Error (RMSE) which is calculated by the following formula:
wherein G is the number of samples used in the test experiment, YgA prediction output representing the prediction of the g-th sample using a neural network, dgRepresenting the expected output of the g-th sample.
The experiment was performed using the neural network optimization method described above, and the experimental results obtained are shown in table 1.
TABLE 1
As can be seen from the data in table 1, by setting different maximum network iteration times, the number of the optimal hidden layers selected by using the neural network optimization method is reduced from the number of the hidden layers initially selected, the actual iteration times are far less than the set maximum network iteration times, and the average value of the root mean square error RMSE of the predicted output of the neural network obtained by performing experiments with different network iteration times is less than 1.
The above examples only show some embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A neural network optimization method is characterized by comprising the following steps:
encoding the weight value of the neural network and the output parameter value of the hidden layer to form a genetic algorithm individual;
randomly generating an initial population comprising a plurality of said individuals;
calculating the fitness of each individual in the population;
judging whether a stop condition is met;
if the judgment result does not meet the requirement, the following steps are executed:
selecting the individual;
randomly crossing the selected individuals;
performing mutation on the individuals after crossing;
generating a new population of new individuals;
counting the number of iterations, and returning to continue to execute the step of calculating the fitness of each individual in the population;
if the stop condition is judged to be met, the following steps are executed:
selecting the individual with the greatest fitness;
decoding the individuals to obtain weights of the neural network and hidden layer output parameters;
and constructing a neural network according to the weight and the hidden layer output parameter value.
2. The neural network optimization method of claim 1, wherein the stopping condition comprises the number of iterations exceeding a maximum number of iterations or the fitness of the individual exceeding a threshold.
3. The neural network optimization method of claim 1, wherein the step of constructing the neural network according to the weights and the hidden layer output parameter values comprises the following steps:
and predicting through the neural network, and outputting a prediction result.
4. The neural network optimization method of claim 1, wherein said step of calculating the fitness of each of said individuals in said population comprises the steps of:
respectively calculating output vectors of the 1 st hidden layer to the Nth hidden layer;
calculating the output of the neuron of the output layer according to the output vector of the Nth hidden layer;
calculating an output vector of an output layer according to the output of the neuron of the output layer;
calculating an objective function;
calculating the fitness of the individual according to the objective function;
and N is the number of hidden layers.
5. The neural network optimization method of claim 4,
the step of calculating the output vectors of the 1 st hidden layer to the Nth hidden layer respectively comprises the following steps:
and calculating the output of the hidden layer neuron, wherein the calculation formula is as follows:
wherein, theRepresents the output of the jth neuron of the 1 st hidden layer, saidRepresents the weight between the kth neuron of the input layer and the jth neuron of the 1 st hidden layer, xkRepresents the input of the kth neuron of the input layer of the neural network, saidRepresents a weight between a k-th neuron of an i-1 th hidden layer and a j-th neuron of the i-th hidden layerAn output of a jth neuron representing an ith hidden layer, theRepresents the output of the kth neuron of the i-1 hidden layer, n0Is the number of neurons of the input layer, ni-1The number of the neurons of the i-1 hidden layer is shown, and f is an activation function;
and calculating an output vector of the 1 st hidden layer by the following formula:
wherein, said n1Represents the number of neurons of the 1 st hidden layer, h1An output vector represented as a layer 1 hidden neuron, theAn output of a 1 st neuron representing a 1 st hidden layer, saidRepresents the output of the jth neuron of the 1 st hidden layer, saidN-th of a 1 st hidden layer1An output of a neuron, theRepresenting a weight between a kth neuron of an input layer and a 1 st neuron of a 1 st hidden layer, theRepresents the weight between the kth neuron of the input layer and the jth neuron of the 1 st hidden layer, wk 0 ,niRepresenting the kth neuron of the input layer with the nth of the 1 st hidden layer1Weights between individual neurons;
and calculating an output vector of the ith hidden layer, wherein the calculation formula is as follows:
wherein, the hiIs the output vector of the ith hidden layer, the hi-1Is the output vector of the i-1 th hidden layer, ui-1Represents the value of an ith hidden layer output parameter, saidRepresenting a weight between a kth neuron of an i-1 th hidden layer and a 1 st neuron of an i hidden layerRepresents a weight between a k-th neuron of an i-1 th hidden layer and a j-th neuron of the i-th hidden layerRepresenting the kth neuron of the i-1 th hidden layer and the nth of the i hidden layeriWeight between neurons, niThe number of neurons of the ith hidden layer;
and calculating an output vector of the Nth hidden layer, wherein the calculation formula is as follows:
said u isN-1Representing the output parameter value of the Nth hidden layer, wherein N is the number of layers of the hidden layer, and hNAn output vector representing an Nth hidden layer, hN-1An output vector representing an N-1 hidden layer, saidAn output of a k-th neuron representing an N-1 hidden layer, theRepresenting a weight between a kth neuron of an N-1 th hidden layer and a 1 st neuron of an N hidden layer, theRepresenting a weight between a kth neuron of an N-1 th hidden layer and a jth neuron of an N hidden layerRepresenting the kth neuron of the N-1 th hidden layer and the nth of the N hidden layerNWeight between neurons, nNRepresenting the number of neurons of the Nth hidden layer, NN-1Representing the number of neurons of the N-1 hidden layer;
the calculation formula of the step of calculating the output of the output layer neuron according to the output vector of the Nth hidden layer is as follows:
wherein, theRepresenting a weight between a k neuron of an Nth hidden layer and a j neuron of an output layer, theAn output representing a kth neuron of an Nth hidden layer; said yjAn output representing a jth neuron of the output layer;
the calculation formula of the step of calculating the output vector of the output layer according to the output of the neuron of the output layer is as follows:
the F (U, W, X) represents an output vector of an output layer, the X is an input vector of an input layer, the W represents a neural network weight matrix, the U represents a hidden layer output parameter value matrix, and U ═ U [ [ U ═ U { [ U { [1,…,ui,…,uN-1]U of said1Representing a value of a 2 nd hidden layer output parameter, uiRepresenting the value of the i +1 th hidden layer output parameter, where u1、uiAnd uN-1Is 0 or 1; said y1Represents the output of the 1 st neuron of the output layer, y2Represents the output of the 1 st neuron of the output layer, ynN+1Indicating the n-th output layerN+1Output of a neuron, said nN+1Representing the number of output layer neurons;
the calculation formula of the step of calculating the objective function is as follows:
R＝||F(U,X,W)-d||2；
wherein R is the objective function, and d is an expected output vector of the neural network;
the calculation formula of the step of calculating the fitness of the individual according to the objective function is as follows:
Fit＝M-R；
wherein Fit is the fitness of the individual, and M is a number greater than 0.
6. The neural network optimization method according to claim 5, wherein the excitation function f is specifically:
wherein x is an argument of the excitation function f.
7. The neural network optimization method of claim 5, wherein the step of selecting the individuals comprises the steps of:
initializing m to be 0;
calculating the probability of each individual being inherited into the next generation population according to the following formula:
wherein, the p iszRepresenting the probability that the z th individual is inherited to the next generation population, wherein T is the number of individuals, Fit (T) is the fitness of the T th individual, Fit (z) is the fitness of the z th individual, z is a positive integer greater than or equal to 1 and less than or equal to T, and T is a positive integer greater than or equal to 1 and less than or equal to T;
calculating the cumulative probability of each individual by the formula:
wherein q ist(ii) a cumulative probability for the tth said individual;
judging whether m is larger than a preset value;
if so, executing the step of randomly crossing the selected individuals;
if the value is less than the preset value, the following steps are executed:
randomly generating a random number r between 0 and 1;
judging whether the r is smaller than the cumulative probability q of the 1 st individual1；
If the r is judged to be smaller than the cumulative probability q of the 1 st individual1Then, the following steps are executed:
selecting the 1 st individual, increasing the selected times of the 1 st individual by 1, increasing the m by 1, and returning to continuously execute the step of judging whether the m is greater than the preset value;
if the r is judged to be more than or equal to the cumulative probability q of the 1 st individual1Then, the following steps are executed:
when q iss-1<r≤qsThen, selecting the s-th individual, wherein the number of times the s-th individual is selected is increased by 1; increasing m by 1, and returning to continue to execute the step of judging whether m is larger than a preset value;
wherein s is a positive integer greater than or equal to 2 and less than or equal to T.
8. The neural network optimization method according to claim 7, wherein the step of randomly crossing the selected individuals specifically comprises:
and randomly crossing the individuals with the selected times larger than the preset times.
9. The neural network optimization method of claim 8, wherein the step of randomly crossing the selected individuals comprises the steps of:
randomly pairing the individuals;
randomly setting the positions of the cross points;
one individual of the paired individuals is divided into a first part and a second part by taking the intersection point as a boundary point;
the other individual of the paired individuals is divided into a third part and a fourth part by a dividing point corresponding to the position of the intersection point;
exchanging the first portion with the third portion, or the second portion with the fourth portion;
and obtaining the crossed individuals.
10. The neural network optimization method of claim 9, wherein the step of mutating the individuals after the crossover comprises the steps of:
selecting a variant locus point for the individual;
negating the value of the variant locus point for the individual.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410843528.5A CN104504442A (en) | 2014-12-30 | 2014-12-30 | Neural network optimization method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410843528.5A CN104504442A (en) | 2014-12-30 | 2014-12-30 | Neural network optimization method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104504442A true CN104504442A (en) | 2015-04-08 |
Family
ID=52945837
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410843528.5A Pending CN104504442A (en) | 2014-12-30 | 2014-12-30 | Neural network optimization method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104504442A (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105279555A (en) * | 2015-10-28 | 2016-01-27 | 清华大学 | Self-adaptive learning neural network implementation method based on evolutionary algorithm |
| CN107063349A (en) * | 2017-04-17 | 2017-08-18 | 云南电网有限责任公司电力科学研究院 | A kind of method and device of Fault Diagnosis Method of Power Transformer |
| CN107836000A (en) * | 2015-07-07 | 2018-03-23 | 触摸式有限公司 | For Language Modeling and the improved artificial neural network of prediction |
| WO2018068421A1 (en) * | 2016-10-11 | 2018-04-19 | 广州视源电子科技股份有限公司 | Method and device for optimizing neural network |
| CN109106384A (en) * | 2018-07-24 | 2019-01-01 | 安庆师范大学 | A kind of psychological pressure condition predicting method and system |
| CN109783857A (en) * | 2018-12-12 | 2019-05-21 | 珠海博雅科技有限公司 | A kind of quick charge pump design method and device |
| CN110059595A (en) * | 2019-04-03 | 2019-07-26 | 哈尔滨理工大学 | A kind of adaptive algorithm for image recognition |
| CN110595535A (en) * | 2019-08-19 | 2019-12-20 | 湖南强智科技发展有限公司 | Monitoring method, device and storage medium |
| CN111767718A (en) * | 2020-07-03 | 2020-10-13 | 北京邮电大学 | Chinese grammar error correction method based on weakened grammar error feature representation |
| US11205110B2 (en) | 2016-10-24 | 2021-12-21 | Microsoft Technology Licensing, Llc | Device/server deployment of neural network data entry system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6470261B1 (en) * | 1998-07-31 | 2002-10-22 | Cet Technologies Pte Ltd | Automatic freeway incident detection system and method using artificial neural network and genetic algorithms |
| CN1622129A (en) * | 2004-12-13 | 2005-06-01 | 中国科学院长春光学精密机械与物理研究所 | Optimization method for artificial neural network |
| CN103971162A (en) * | 2014-04-04 | 2014-08-06 | 华南理工大学 | Method for improving BP (back propagation) neutral network and based on genetic algorithm |
-
2014
- 2014-12-30 CN CN201410843528.5A patent/CN104504442A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6470261B1 (en) * | 1998-07-31 | 2002-10-22 | Cet Technologies Pte Ltd | Automatic freeway incident detection system and method using artificial neural network and genetic algorithms |
| CN1622129A (en) * | 2004-12-13 | 2005-06-01 | 中国科学院长春光学精密机械与物理研究所 | Optimization method for artificial neural network |
| CN103971162A (en) * | 2014-04-04 | 2014-08-06 | 华南理工大学 | Method for improving BP (back propagation) neutral network and based on genetic algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 杨国军等: "遗传算法在神经网络控制中的应用与实现", 《系统仿真学报》 * |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107836000A (en) * | 2015-07-07 | 2018-03-23 | 触摸式有限公司 | For Language Modeling and the improved artificial neural network of prediction |
| US11797822B2 (en) | 2015-07-07 | 2023-10-24 | Microsoft Technology Licensing, Llc | Neural network having input and hidden layers of equal units |
| CN107836000B (en) * | 2015-07-07 | 2022-04-01 | 微软技术许可有限责任公司 | Improved artificial neural network method and electronic device for language modeling and prediction |
| CN105279555B (en) * | 2015-10-28 | 2017-10-17 | 清华大学 | A kind of adaptive learning neural network implementation method based on evolution algorithm |
| CN105279555A (en) * | 2015-10-28 | 2016-01-27 | 清华大学 | Self-adaptive learning neural network implementation method based on evolutionary algorithm |
| WO2018068421A1 (en) * | 2016-10-11 | 2018-04-19 | 广州视源电子科技股份有限公司 | Method and device for optimizing neural network |
| US11205110B2 (en) | 2016-10-24 | 2021-12-21 | Microsoft Technology Licensing, Llc | Device/server deployment of neural network data entry system |
| CN107063349A (en) * | 2017-04-17 | 2017-08-18 | 云南电网有限责任公司电力科学研究院 | A kind of method and device of Fault Diagnosis Method of Power Transformer |
| CN109106384A (en) * | 2018-07-24 | 2019-01-01 | 安庆师范大学 | A kind of psychological pressure condition predicting method and system |
| CN109783857A (en) * | 2018-12-12 | 2019-05-21 | 珠海博雅科技有限公司 | A kind of quick charge pump design method and device |
| CN110059595A (en) * | 2019-04-03 | 2019-07-26 | 哈尔滨理工大学 | A kind of adaptive algorithm for image recognition |
| CN110595535A (en) * | 2019-08-19 | 2019-12-20 | 湖南强智科技发展有限公司 | Monitoring method, device and storage medium |
| CN111767718B (en) * | 2020-07-03 | 2021-12-07 | 北京邮电大学 | Chinese grammar error correction method based on weakened grammar error feature representation |
| CN111767718A (en) * | 2020-07-03 | 2020-10-13 | 北京邮电大学 | Chinese grammar error correction method based on weakened grammar error feature representation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104504442A (en) | Neural network optimization method | |
| CN109816095B (en) | Network flow prediction method based on improved gated cyclic neural network | |
| CN110263227A (en) | Clique based on figure neural network finds method and system | |
| CN109241291A (en) | Knowledge mapping optimal path inquiry system and method based on deeply study | |
| CN108985335B (en) | Integrated learning prediction method for irradiation swelling of nuclear reactor cladding material | |
| CN107688849A (en) | A kind of dynamic strategy fixed point training method and device | |
| Liu et al. | Multiobjective criteria for neural network structure selection and identification of nonlinear systems using genetic algorithms | |
| CN111898689A (en) | Image classification method based on neural network architecture search | |
| CN106610867B (en) | On-chip network task scheduling method and device | |
| CN116542382A (en) | Sewage treatment dissolved oxygen concentration prediction method based on mixed optimization algorithm | |
| CN113128432B (en) | Machine vision multitask neural network architecture searching method based on evolution calculation | |
| CN110738362A (en) | method for constructing prediction model based on improved multivariate cosmic algorithm | |
| CN107145943A (en) | Method based on the detection echo state network small-signal for improving teaching optimized algorithm in Chaotic Background | |
| CN104732067A (en) | Industrial process modeling forecasting method oriented at flow object | |
| CN115908909A (en) | Evolutionary neural architecture searching method and system based on Bayes convolutional neural network | |
| CN111126560A (en) | Method for optimizing BP neural network based on cloud genetic algorithm | |
| CN115033878A (en) | Rapid self-game reinforcement learning method and device, computer equipment and storage medium | |
| CN108537663A (en) | One B shareB trend forecasting method | |
| CN117273080A (en) | Neural network architecture based on evolutionary algorithm | |
| CN116881683A (en) | GA-AM-GRU-based flow industrial energy consumption prediction method | |
| CN116611504A (en) | Neural architecture searching method based on evolution | |
| CN114863508A (en) | Expression recognition model generation method, medium and device of adaptive attention mechanism | |
| Kathirvel et al. | Hybrid imperialistic competitive algorithm incorporated with hopfield neural network for robust 3 satisfiability logic programming | |
| Mei | Internet Abnormal Traffic Detection Based on WOA-BiLSTM-Attention | |
| CN110910164A (en) | Product sales forecasting method, system, computer device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150408 |
|
| RJ01 | Rejection of invention patent application after publication |