CN106897744A - A kind of self adaptation sets the method and system of depth confidence network parameter - Google Patents
A kind of self adaptation sets the method and system of depth confidence network parameter Download PDFInfo
- Publication number
- CN106897744A CN106897744A CN201710108331.0A CN201710108331A CN106897744A CN 106897744 A CN106897744 A CN 106897744A CN 201710108331 A CN201710108331 A CN 201710108331A CN 106897744 A CN106897744 A CN 106897744A
- Authority
- CN
- China
- Prior art keywords
- depth confidence
- confidence network
- fitness
- parameter
- genetic algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
Abstract
The invention discloses the method that the self adaptation based on genetic algorithm sets depth confidence network parameter, including：The initial value of depth confidence network parameter to be optimized is set, the fitness function that depth confidence network parameter is set is set up according to initial value；According to fitness function, optimum individual data are obtained using genetic algorithm；Optimum individual data are carried out into Gray code and obtains optimal depth confidence network parameter；The parameter setting mode inefficiency of manual formula is solved, and is often unable to reach the problem of optimal parameter setting；Accuracy rate can be improved using genetic algorithm, while the initial parameter of depth confidence network can automatically be determined according to input sample, and then optimal network topological structure is obtained；Using the parameter after optimization can learning sample data exactly advanced feature, depth confidence network is obtained more preferable recognition result.The invention also discloses the system that the self adaptation based on genetic algorithm sets depth confidence network parameter, with above-mentioned beneficial effect.
Description
Technical field
The present invention relates to technical field of data processing, more particularly to a kind of self adaptation sets the side of depth confidence network parameter
Method and system.
Background technology
At present, deep learning (Deep Learning) is a research topic for hot topic in field of target recognition.Depth
The concept of habit is that Hinton et al. was proposed in 2006, mainly simulates the National People's Congress by neutral net (Neural Network)
The learning process of brain, according to the achievement for recognizing brain things process study, it is intended to replace human brain using node one by one
Neuron, and the purpose of reconstruction of objects is reached by the Weight Training of node, its most outstanding feature is the extraction of feature
Process reduces artificial intervention as little as possible.Deep learning is substantially a kind of greedy algorithm, and it is a kind of neural network
Structure, however in general Internet can have multilayer, unlike artificial neural network only three layers.In deep learning mould
The bottom of type, that is, sample input layer, initial sample directly can be added into network, network can automatically detach spy
Levy, it is not necessary to which artificial goes extraction feature, this is deep learning and the maximum difference of traditional mode identification.
The framework of deep learning can have a variety of models, and conventional has sparse own coding, limited Boltzmann machine, convolution
Neutral net etc..Wherein the depth network based on limited Boltzmann machine is referred to as depth confidence network (Deep Belief
Networks, DBN), depth confidence network is a model generated by Bayesian probability, by a series of limited Boltzmann
Machine (Restricted Boltzmann Machine, RBM) stacking is formed.Depth confidence network is compared to traditional target identification
Method (template matches, Model Matching, SVMs, principal component analysis etc.) has advantages below：1) hand-designed is reduced
The huge workload of feature；2) depth confidence network can automatically carry out the extraction of feature and detach；3) depth confidence network
By Input transformation to higher dimensional space, there is better expression to input data, not only recognition effect can be more preferable, and makes
Use and be also more convenient.
But, the selection of current depth confidence network initial parameter is passed through mostly or using the method for artificial selection
Test of many times chooses the way of empirical value.This mode, due to according to sample self adaptation arrange parameter, thus cannot hardly result in
Optimal network parameter, cannot also obtain optimal recognition effect.
The content of the invention
It is an object of the invention to provide a kind of method that self adaptation based on genetic algorithm sets depth confidence network parameter
And system, by genetic algorithm come the parameter of Automatic adjusument depth confidence network, so as to obtain more preferable recognition effect.
In order to solve the above technical problems, the present invention provides a kind of self adaptation based on genetic algorithm sets depth confidence network
The method of parameter, methods described includes：
The initial value of depth confidence network parameter to be optimized is set, and setting depth confidence is set up according to the initial value
The fitness function of network parameter；
According to the fitness function, optimum individual data are obtained using genetic algorithm；
The optimum individual data are carried out into Gray code and obtains optimal depth confidence network parameter.
Optionally, the initial value of depth confidence network parameter to be optimized is set, and setting is set up according to the initial value
The fitness function of depth confidence network parameter, including：
The initial value of depth confidence network parameter to be optimized is set；
Binary coding is carried out to the initial value of the depth confidence network parameter using genetic algorithm, construction is initial to plant
Group, and as training sample；
The training sample is identified after determining the topological structure of depth confidence network, obtains the training sample
Discrimination；
Discrimination average is calculated according to the discrimination, and the discrimination average is joined as depth confidence network is set
Several fitness functions.
Optionally, according to the fitness function, optimum individual data are obtained using genetic algorithm, including：
Calculate initial population fitness；
New generation population is calculated after carrying out selection operation, crossover operation and mutation operation according to the initial population fitness
Fitness；
Judge whether the new generation population's fitness meets pre-conditioned；
Using the new generation population's fitness as optimum individual data if meeting；
Selection operation, crossover operation and variation behaviour are carried out according to the new generation population's fitness again if being unsatisfactory for
Make, until the population's fitness for newly obtaining meets pre-conditioned, untill obtaining optimum individual data.
Optionally, it is described it is pre-conditioned be specially：The number of times that circulation performs selection operation, crossover operation and mutation operation is expired
The fluctuation situation of the fitness numerical value obtained in sufficient cyclic algebra or continuous pre-determined number cyclic process meets preset range.
Optionally, binary coding is carried out to the depth confidence network parameter using genetic algorithm, including：
Binary coding is carried out to the depth confidence network parameter using the coding function in the tool box of genetic algorithm.
The present invention also provides the system that a kind of self adaptation based on genetic algorithm sets depth confidence network parameter, including：
Initialization module, the initial value for setting depth confidence network parameter to be optimized, and according to the initial value
Set up the fitness function that depth confidence network parameter is set；
Genetic algorithm module, for according to the fitness function, optimum individual data being obtained using genetic algorithm；
Parameter determination module, optimal depth confidence network ginseng is obtained for the optimum individual data to be carried out into Gray code
Number.
Optionally, the initialization module includes：
Initial value setup unit, the initial value for setting depth confidence network parameter to be optimized；
Coding unit, for carrying out binary system volume to the initial value of the depth confidence network parameter using genetic algorithm
Code, constructs initial population, and as training sample；
Recognition unit, for being identified to the training sample after the topological structure for determining depth confidence network, obtains
The discrimination of the training sample；
Fitness function determining unit for calculating discrimination average and the discrimination is equal according to the discrimination
It is worth as the fitness function for setting depth confidence network parameter.
Optionally, the genetic algorithm module includes：
First computing unit, for calculating initial population fitness；
Second computing unit, for carrying out selection operation, crossover operation and variation behaviour according to the initial population fitness
New generation population's fitness is calculated after work；
Judging unit, it is pre-conditioned for judging newly to produce population's fitness whether to meet；
As a result output unit, if during for meeting pre-conditioned, will newly produce population's fitness as optimum individual data；
3rd computing unit, if during for being unsatisfactory for pre-conditioned, being carried out again according to the new generation population's fitness
New generation population's fitness is calculated after selection operation, crossover operation and mutation operation, and triggers the judging unit.
Optionally, the coding unit includes：
Coded sub-units, it is initial to the depth confidence network for the coding function in the tool box using genetic algorithm
Parameter carries out binary coding.
The method that self adaptation based on genetic algorithm provided by the present invention sets depth confidence network parameter, including：If
The initial value of depth confidence network parameter to be optimized is put, the fitness that depth confidence network parameter is set is set up according to initial value
Function；According to fitness function, optimum individual data are obtained using genetic algorithm；Optimum individual data are carried out into Gray code acquisition
Optimal depth confidence network parameter；
It can be seen that, the method can solve the problem that the parameter setting mode inefficiency of manual formula, and often be unable to reach optimal
The problem of parameter setting；It uses the genetic algorithm can to improve accuracy rate, while depth can automatically be determined according to input sample
The initial parameter of confidence network is spent, and then obtains optimal network topological structure；Can be learnt exactly using the parameter after optimization
The advanced feature of sample data, makes depth confidence network obtain more preferable recognition result.Calculated present invention also offers based on heredity
The system that the self adaptation of method sets depth confidence network parameter, with above-mentioned beneficial effect, will not be repeated here.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
The accompanying drawing to be used needed for having technology description is briefly described, it should be apparent that, drawings in the following description are only this
Inventive embodiment, for those of ordinary skill in the art, on the premise of not paying creative work, can also basis
The accompanying drawing of offer obtains other accompanying drawings.
A kind of solution schematic diagram of genetic algorithm that Fig. 1 is provided by the embodiment of the present invention；
The side of the setting depth confidence network parameter of the self adaptation based on genetic algorithm that Fig. 2 is provided by the embodiment of the present invention
The flow chart of method；
Fig. 3 sets depth confidence network parameter by a kind of self adaptation based on genetic algorithm that the embodiment of the present invention is provided
Schematic diagram；
Fig. 4 is by what self adaptation based on genetic algorithm that the embodiment of the present invention is provided set depth confidence network parameter
The structured flowchart of system.
Specific embodiment
Core of the invention is to provide a kind of method that self adaptation based on genetic algorithm sets depth confidence network parameter
And system, by genetic algorithm come the parameter of Automatic adjusument depth confidence network, so as to obtain more preferable recognition effect.
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In accompanying drawing, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
A part of embodiment of the present invention, rather than whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art
The every other embodiment obtained under the premise of creative work is not made, belongs to the scope of protection of the invention.
The concept that be will appear from embodiment is explained below：
The depth confidence network (Deep belief Network, DBN) of standard：
DBN networks are a kind of models in deep learning, and the probabilistic model comprising multiple hidden layers, hidden layer is trained to come
Capture the higher order dependencies of visual layer data.The adjacent two layers of depth confidence network can regard an independent limited Bohr as
Hereby graceful machine, is in the RBM of low layer as the RBM for entering data to train next high level, you can greedy using each
One group of RBM is practised, therefore depth confidence network can be indicated to lower probability model：
P (V, H1, H2..., HN)=P (V | H1)P(H1|H2)…P(HN-2|HN-1)P(HN-1|HN)
In the beginning of training, the training of DBN only learns iteration phase every time by non-supervisory greedy successively training method
The network parameter of the RBM of adjacent two-layer, and the arameter optimization of whole DBN is completed by such mode, then by tape label
Data are finely adjusted to whole model.
Input sample is imported visual layers V by one each layer of DBN networks all comprising many nodes of knowing clearly, training the
One layer of RBM can obtain parameter θ1={ W1,B1,A1, then by θ1It is fixed, by P (H1| V)=P (H1|V,θ1) can be obtained
One layer of RBM hidden layer node data H1；Then, by H1Regard the input data of second layer RBM as, and then obtain the net of second layer RBM
Network parameter θ2={ W2,B2,A2And second layer RBM implicit layer data H2, by that analogy, each layer can be calculated with recursive
The node data of network parameter and hidden layer.
After training successively completes whole RBM, then the original training data with label is imported, by error
The BP algorithm of back-propagating, arameter optimization is carried out with gradient descent method to whole DBN networks.
DBN network parameters：
When DBN network models are built, in order to preferably carry out the training of sample data, results of learning are improved, had
Necessary initial value to some of which parameter etc. is configured.
In general, the parameter of DBN networks has following several：
(1) number of plies of hidden layer
The number of hidden layer in depth confidence network.Similar to the hidden layer of artificial neural network, but ANN
Network only has one layer of hidden layer.In general, the hidden layer of depth confidence network is not above 7 layers.
(2) number of layer unit is implied
The number of each hidden layer interior joint in depth confidence network.In general, one sample of DBN network representations
Required byte number, is multiplied by the number of input sample, then reduce the conduct implicit node of the layer that an order of magnitude can be approximate
Number.
(3) learning rate
In the renewal rule of RBM parameters, the speed that parameter updates is represented, in general, learning rate sets excessive, it will
Cause model reconstruction error to increase, be unfavorable for the extraction of high-order feature；Set too small, the reduction of parameter renewal speed, training time
Increase, so as to cause the generation of over-fitting.
(4) size of data of batch study
In the renewal process that the interlayer connection weight and interlayer of DBN networks are biased, the side that training sample is imported completely
Formula, its amount of calculation is very big.Therefore, before training, input sample is first usually resolved into multiple comprising tens or several
Hundred sample datas, this mode is referred to as the mode of batch processing, and the sample number included per batch similarly affects model
Training.
(5) iterations of RBM
During using to sdpecific dispersion Algorithm Learning RBM, it is often necessary to carry out successive ignition, can just obtain preferably
RBM network parameters.
Genetic algorithm (Genetic Algorithm, GA)：
Genetic algorithm represents individual one by one in population by binary coding string so that genetic manipulation become it is simple easily
OK.Adaptability is low in using for reference principle (represented here as size of fitness function value) the elimination genetic process of the survival of the fittest simultaneously
Individuality, in order to ensure global search, genetic algorithm produces more new explanations using the mutation operation of gene, prevents optimization from asking
Topic is absorbed in local optimum.The characteristics of genetic algorithm has following：
1st, in essentially all of Practical Project problem, being related to Parametric optimization problem can be solved using heredity.
2nd, algorithm has concurrency, self adaptation and has good robustness, it is not necessary to other auxiliary informations, it is only necessary to press
According to the facts suitable its fitness function of border problem definition one.
3rd, in function optimization, various types of objective optimization functions are gone for.
4th, by the coding of parameter so that the details adjustment of line parameter can be entered, while being conducive to generation with a greater variety new
Solution.Genetic algorithm can in solution space by intersect, variation and selection operation constantly produce new solution, it is ensured that the diversity of solution from
And reach global optimum.
Genetic algorithm has three basic operations：Selection, intersection, variation.These three basic operations are used to ensure that generation is new
Optional solution, the workflow of basic genetic algorithmic is as shown in Figure 1.
The present embodiment realizes automatic acquisition optimal depth confidence network parameter using genetic algorithm, specifically refer to Fig. 2, schemes
2 flows that the method for depth confidence network parameter is set by the self adaptation based on genetic algorithm that the embodiment of the present invention is provided
Figure；The method can include：
S100, the initial value that depth confidence network parameter to be optimized is set, and it is deep to set up setting according to the initial value
Spend the fitness function of confidence network parameter；
Specifically, the step is mainly for determination fitness function.Global search is carried out by genetic algorithm, is obtained most
Excellent fitness numerical value, and then determine corresponding optimum individual data, and obtain optimal depth confidence network ginseng according to the data
Number.Automatic Optimal depth confidence network parameter is realized, and then improves the purpose of depth confidence Network Recognition efficiency.
Here it is to be optimized be depth confidence network initial parameter, 5 optimizations such as listed in above-mentioned DBN network parameters
Parameter.Binary system initial code is carried out firstly the need of to this 5 parameters, then according to initial 5 network parameter after coding
It is added in depth confidence network and is once recognized, obtains the discrimination of training sample.Finally, by the discrimination of training sample
Average as genetic algorithm fitness function.This fitness function, be with the discrimination average of training sample be from become
The function of amount, and 5 above-mentioned parameters to be optimized affect discrimination, therefore, it can indirectly fitness function
Independent variable, regards this 5 parameters to be optimized as.When optimal fitness function is obtained, that is, obtain optimal discrimination when
Wait, just obtained the parameter of optimal depth confidence network.
Optionally, the initial value of depth confidence network parameter to be optimized is set, and setting is set up according to the initial value
The fitness function of depth confidence network parameter, can include：
The initial value of depth confidence network parameter to be optimized is set；
Binary coding is carried out to the initial value of the depth confidence network parameter using genetic algorithm, construction is initial to plant
Group, and as training sample；
The training sample is identified after determining the topological structure of depth confidence network, obtains the training sample
Discrimination；
Discrimination average is calculated according to the discrimination, and the discrimination average is joined as depth confidence network is set
Several fitness functions.
Specifically, setting the initial value of depth confidence network parameter to be optimized first, that is, determine that depth confidence network needs
The parameter of setting is optimized, and provides each parameter minimum value, maximum in an iterative process.Such as first, imply
The number of plies number of layer：Here it is 1 to be set to minimum hidden layer, and maximum is 5.Second, the maximum and minimum of each hidden layer
Value；Such as the first hidden layer：The minimum value of the first hidden layer size, the maximum of the first hidden layer size；Second hidden layer：The
The minimum value of two hidden layer sizes, the maximum of the second hidden layer size；Until kth hidden layer：Kth hidden layer size is most
Small value, the maximum of kth hidden layer size.3rd, the size of the learning rate of depth confidence network：Learning rate is minimum
Value, learning rate maximum.4th, the size of the batch training data of depth confidence network：Initial batch learning data
The minimum value of size, the maximum of the size of initial batch learning data.5th, the RBM iterationses of depth confidence network：
The minimum value of RBM primary iteration number of times, the maximum of RBM primary iteration number of times.
Here binary coding be genetic algorithm coding according to standard genetic algorithm step by above-mentioned parameter, carry out
Binary coding, may be referred to the operation of standard genetic algorithm.Here can also be standard genetic algorithm tool box in
Coding function.One Population in Genetic Algorithms of construction, each individuality is exactly a coding of above-mentioned parameter.Here initial network knot
Structure parameter is generated at random according to the maximum and minimum value of each parameter.
Initial value in the present embodiment according to depth confidence network parameter builds a depth confidence network, can adopt here
With the Toolbox structure depth confidence network of depth confidence network.Training sample is added into depth confidence network to be identified.
Obtain the discrimination of training sample.
S110, according to the fitness function, obtain optimum individual data using genetic algorithm；
Specifically, the detailed process of genetic algorithm can be as follows in the present embodiment：
Calculate initial population fitness；
New generation population is calculated after carrying out selection operation, crossover operation and mutation operation according to the initial population fitness
Fitness；
Judge whether the new generation population's fitness meets pre-conditioned；
Using the new generation population's fitness as optimum individual data if meeting；
Selection operation, crossover operation and variation behaviour are carried out according to the new generation population's fitness again if being unsatisfactory for
Make, until the population's fitness for newly obtaining meets pre-conditioned, untill obtaining optimum individual data.
Wherein, fitness function is good during selection operation is specially selected population, i.e. discrimination those individualities high are made
It is the seed of heredity of future generation.Crossover operation is specially according to crossover probability, and the individual encoded radio for selecting is carried out into single-point friendship
Fork or multiple-spot detection, produce new individuality.Mutation operation is specially according to mutation probability, and the individual encoded radio for selecting is entered
Row variation, produces new individuality.
Whether when genetic algorithm is exporting optimum individual data, to see meet iterated conditional i.e. heredity circulation exit criteria
Namely it is above-mentioned described pre-conditioned.The present embodiment is not defined to pre-conditioned, and user can be according to available accuracy and hard
Part capability requirement is selected.It is optionally pre-conditioned to be specially：Circulation performs selection operation, crossover operation and variation behaviour
The number of times of work meet cyclic algebra (here refer to once the overall work for completing selection operation, crossover operation and mutation operation
For once) or continuous pre-determined number cyclic process in the fluctuation situation of fitness numerical value that obtains meet preset range.
If for example number of cycles is more than in default cyclic algebra, or continuous 10 cyclic processes, fitness function
Value fluctuated in the range of a very little, then exit circulation.Optimum individual Gray code is obtained into corresponding parameter, it is as optimal
Depth confidence network structural parameters.Otherwise continue cycling through iteration.
S120, the optimum individual data are carried out Gray code obtain optimal depth confidence network parameter.
Specifically, the binary coding process and Gray code process of genetic algorithm inverse process each other here.
The optimal depth confidence network parameter obtained using above-described embodiment can obtain optimal depth confidence network, to this
Network is trained, it is possible to carry out the identification of test sample, obtains the recognition result of test sample.Fig. 3 is specifically refer to, is schemed
3 give one kind implements schematic diagram.Said process can be divided into genetic algorithm part and depth confidence network portion.
Based on above-described embodiment, the self adaptation based on genetic algorithm that the embodiment of the present invention is carried sets depth confidence network ginseng
Several methods, blending inheritance algorithm to depth confidence network improve and has made it possible to automatically adjusting parameter, realizes parameter
Self-optimizing, that is, can solve the problem that the parameter setting mode inefficiency of manual formula, and often be unable to reach optimal parameter setting
Problem；It uses the genetic algorithm can to improve accuracy rate, while depth confidence network can automatically be determined according to input sample
Initial parameter, and then obtain optimal network topological structure；Can learning sample data exactly using the parameter after optimization
Advanced feature, makes depth confidence network obtain more preferable recognition result.
It is to what the self adaptation based on genetic algorithm provided in an embodiment of the present invention set depth confidence network parameter below
System is introduced, and the self adaptation based on genetic algorithm described below sets the system of depth confidence network parameter and is described above
Self adaptation based on genetic algorithm the method for depth confidence network parameter is set can be mutually to should refer to.
Fig. 4 is refer to, Fig. 4 sets depth confidence net by the self adaptation based on genetic algorithm that the embodiment of the present invention is provided
The structured flowchart of the system of network parameter；The system can include：
Initialization module 100, the initial value for setting depth confidence network parameter to be optimized, and according to described initial
Value sets up the fitness function for setting depth confidence network parameter；
Genetic algorithm module 200, for according to the fitness function, optimum individual data being obtained using genetic algorithm；
Parameter determination module 300, optimal depth confidence network is obtained for the optimum individual data to be carried out into Gray code
Parameter.
Based on above-described embodiment, the initialization module 100 can include：
Initial value setup unit, the initial value for setting depth confidence network parameter to be optimized；
Coding unit, for carrying out binary system volume to the initial value of the depth confidence network parameter using genetic algorithm
Code, constructs initial population, and as training sample；
Recognition unit, for being identified to the training sample after the topological structure for determining depth confidence network, obtains
The discrimination of the training sample；
Fitness function determining unit for calculating discrimination average and the discrimination is equal according to the discrimination
It is worth as the fitness function for setting depth confidence network parameter.
Based on above-described embodiment, the genetic algorithm module 200 can include：
First computing unit, for calculating initial population fitness；
Second computing unit, for carrying out selection operation, crossover operation and variation behaviour according to the initial population fitness
New generation population's fitness is calculated after work；
Judging unit, it is pre-conditioned for judging newly to produce population's fitness whether to meet；
As a result output unit, if during for meeting pre-conditioned, will newly produce population's fitness as optimum individual data；
3rd computing unit, if during for being unsatisfactory for pre-conditioned, being carried out again according to the new generation population's fitness
New generation population's fitness is calculated after selection operation, crossover operation and mutation operation, and triggers the judging unit.
Based on above-described embodiment, the coding unit includes：
Coded sub-units, it is initial to the depth confidence network for the coding function in the tool box using genetic algorithm
Parameter carries out binary coding.
Based on above-described embodiment, the self adaptation based on genetic algorithm that the embodiment of the present invention is carried sets depth confidence network ginseng
Several systems, blending inheritance algorithm to depth confidence network improve and has made it possible to automatically adjusting parameter, realizes parameter
Self-optimizing, that is, solve the parameter setting mode inefficiency of manual formula, and is often unable to reach optimal parameter setting
Problem.Depth confidence network is improved using genetic algorithm, makes it possible to adaptively carry out initial parameter tuning, improved
Discrimination；Accuracy rate higher can be obtained using genetic algorithm, while depth can automatically be determined according to input sample
The initial parameter of confidence network, and then obtain network topology structure；Parameter after optimization can learning sample data exactly
Advanced feature, makes depth confidence network obtain more preferable recognition result.
Each embodiment is described by the way of progressive in specification, and what each embodiment was stressed is and other realities
Apply the difference of example, between each embodiment identical similar portion mutually referring to.For device disclosed in embodiment
Speech, because it is corresponded to the method disclosed in Example, so description is fairly simple, related part is referring to method part illustration
.
Professional further appreciates that, with reference to the unit of each example of the embodiments described herein description
And algorithm steps, can be realized with electronic hardware, computer software or the combination of the two, in order to clearly demonstrate hardware and
The interchangeability of software, generally describes the composition and step of each example according to function in the above description.These
Function is performed with hardware or software mode actually, depending on the application-specific and design constraint of technical scheme.Specialty
Technical staff can realize described function to each specific application using distinct methods, but this realization should not
Think beyond the scope of this invention.
The step of method or algorithm for being described with reference to the embodiments described herein, directly can be held with hardware, processor
Capable software module, or the two combination is implemented.Software module can be placed in random access memory (RAM), internal memory, read-only deposit
Reservoir (ROM), electrically programmable ROM, electrically erasable ROM, register, hard disk, moveable magnetic disc, CD-ROM or technology
In field in known any other form of storage medium.
The side of depth confidence network parameter is set to a kind of self adaptation based on genetic algorithm provided by the present invention above
Method and system are described in detail.Specific case used herein is explained principle of the invention and implementation method
State, the explanation of above example is only intended to help and understands the method for the present invention and its core concept.It should be pointed out that for this skill
For the those of ordinary skill in art field, under the premise without departing from the principles of the invention, some changing can also be carried out to the present invention
Enter and modify, these are improved and modification is also fallen into the protection domain of the claims in the present invention.
Claims (9)
1. a kind of method that self adaptation based on genetic algorithm sets depth confidence network parameter, it is characterised in that methods described
Including：
The initial value of depth confidence network parameter to be optimized is set, and setting depth confidence network is set up according to the initial value
The fitness function of parameter；
According to the fitness function, optimum individual data are obtained using genetic algorithm；
The optimum individual data are carried out into Gray code and obtains optimal depth confidence network parameter.
2. method according to claim 1, it is characterised in that the initial of depth confidence network parameter to be optimized is set
Value, and the fitness function that depth confidence network parameter is set is set up according to the initial value, including：
The initial value of depth confidence network parameter to be optimized is set；
Binary coding is carried out to the initial value of the depth confidence network parameter using genetic algorithm, initial population is constructed, and
As training sample；
The training sample is identified after determining the topological structure of depth confidence network, obtains the identification of the training sample
Rate；
Discrimination average is calculated according to the discrimination, and using the discrimination average as setting depth confidence network parameter
Fitness function.
3. method according to claim 2, it is characterised in that according to the fitness function, obtained using genetic algorithm
Optimum individual data, including：
Calculate initial population fitness；
New generation Population adaptation is calculated after carrying out selection operation, crossover operation and mutation operation according to the initial population fitness
Degree；
Judge whether the new generation population's fitness meets pre-conditioned；
Using the new generation population's fitness as optimum individual data if meeting；
Selection operation, crossover operation and mutation operation are carried out according to the new generation population's fitness again if being unsatisfactory for, directly
Meet pre-conditioned to the population's fitness for newly obtaining, untill obtaining optimum individual data.
4. method according to claim 3, it is characterised in that described pre-conditioned to be specially：Circulation execution selection operation,
The number of times of crossover operation and mutation operation meets the fitness numerical value obtained in cyclic algebra or continuous pre-determined number cyclic process
Fluctuation situation meet preset range.
5. method according to claim 4, it is characterised in that entered to the depth confidence network parameter using genetic algorithm
Row binary coding, including：
Binary coding is carried out to the depth confidence network parameter using the coding function in the tool box of genetic algorithm.
6. the system that a kind of self adaptation based on genetic algorithm sets depth confidence network parameter, it is characterised in that including：
Initialization module, the initial value for setting depth confidence network parameter to be optimized, and set up according to the initial value
The fitness function of depth confidence network parameter is set；
Genetic algorithm module, for according to the fitness function, optimum individual data being obtained using genetic algorithm；
Parameter determination module, optimal depth confidence network parameter is obtained for the optimum individual data to be carried out into Gray code.
7. system according to claim 6, it is characterised in that the initialization module includes：
Initial value setup unit, the initial value for setting depth confidence network parameter to be optimized；
Coding unit, for carrying out binary coding, structure to the initial value of the depth confidence network parameter using genetic algorithm
Initial population is made, and as training sample；
Recognition unit, for being identified to the training sample after the topological structure for determining depth confidence network, obtains described
The discrimination of training sample；
Fitness function determining unit, for calculating discrimination average according to the discrimination, and the discrimination average is made
To set the fitness function of depth confidence network parameter.
8. system according to claim 7, it is characterised in that the genetic algorithm module includes：
First computing unit, for calculating initial population fitness；
Second computing unit, after carrying out selection operation, crossover operation and mutation operation according to the initial population fitness
Calculate new generation population's fitness；
Judging unit, it is pre-conditioned for judging newly to produce population's fitness whether to meet；
As a result output unit, if during for meeting pre-conditioned, will newly produce population's fitness as optimum individual data；
3rd computing unit, if during for being unsatisfactory for pre-conditioned, being selected again according to the new generation population's fitness
New generation population's fitness is calculated after operation, crossover operation and mutation operation, and triggers the judging unit.
9. system according to claim 8, it is characterised in that the coding unit includes：
Coded sub-units, for the coding function in the tool box using genetic algorithm to the depth confidence network initial parameter
Carry out binary coding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710108331.0A CN106897744A (en) | 2017-02-27 | 2017-02-27 | A kind of self adaptation sets the method and system of depth confidence network parameter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710108331.0A CN106897744A (en) | 2017-02-27 | 2017-02-27 | A kind of self adaptation sets the method and system of depth confidence network parameter |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106897744A true CN106897744A (en) | 2017-06-27 |
Family
ID=59184234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710108331.0A Pending CN106897744A (en) | 2017-02-27 | 2017-02-27 | A kind of self adaptation sets the method and system of depth confidence network parameter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106897744A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609648A (en) * | 2017-07-21 | 2018-01-19 | 哈尔滨工程大学 | A kind of combination stacks the genetic algorithm of the sparse autocoder of noise reduction |
| CN108280451A (en) * | 2018-01-19 | 2018-07-13 | 北京市商汤科技开发有限公司 | Semantic segmentation and network training method and device, equipment, medium, program |
| CN108446534A (en) * | 2018-03-13 | 2018-08-24 | 腾讯科技（深圳）有限公司 | Select the method, apparatus and computer readable storage medium of neural network hyper parameter |
| CN108921602A (en) * | 2018-06-21 | 2018-11-30 | 华南理工大学 | A kind of user's buying behavior prediction technique based on integrated neural network |
| CN109086817A (en) * | 2018-07-25 | 2018-12-25 | 西安工程大学 | A kind of Fault Diagnosis for HV Circuit Breakers method based on deepness belief network |
| CN109800763A (en) * | 2019-02-09 | 2019-05-24 | 鲁东大学 | A kind of handwritten Chinese recognition methods based on deep learning |
| CN113723536A (en) * | 2021-09-02 | 2021-11-30 | 国网智能科技股份有限公司 | Power inspection target identification method and system |
-
2017
- 2017-02-27 CN CN201710108331.0A patent/CN106897744A/en active Pending
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107609648A (en) * | 2017-07-21 | 2018-01-19 | 哈尔滨工程大学 | A kind of combination stacks the genetic algorithm of the sparse autocoder of noise reduction |
| CN107609648B (en) * | 2017-07-21 | 2021-02-12 | 哈尔滨工程大学 | Genetic algorithm combined with stacking noise reduction sparse automatic encoder |
| CN108280451A (en) * | 2018-01-19 | 2018-07-13 | 北京市商汤科技开发有限公司 | Semantic segmentation and network training method and device, equipment, medium, program |
| CN108280451B (en) * | 2018-01-19 | 2020-12-29 | 北京市商汤科技开发有限公司 | Semantic segmentation and network training method and device, equipment and medium |
| CN108446534A (en) * | 2018-03-13 | 2018-08-24 | 腾讯科技（深圳）有限公司 | Select the method, apparatus and computer readable storage medium of neural network hyper parameter |
| CN108921602A (en) * | 2018-06-21 | 2018-11-30 | 华南理工大学 | A kind of user's buying behavior prediction technique based on integrated neural network |
| CN108921602B (en) * | 2018-06-21 | 2021-12-21 | 华南理工大学 | User purchasing behavior prediction method based on integrated neural network |
| CN109086817A (en) * | 2018-07-25 | 2018-12-25 | 西安工程大学 | A kind of Fault Diagnosis for HV Circuit Breakers method based on deepness belief network |
| CN109086817B (en) * | 2018-07-25 | 2022-04-26 | 西安工程大学 | High-voltage circuit breaker fault diagnosis method based on deep belief network |
| CN109800763A (en) * | 2019-02-09 | 2019-05-24 | 鲁东大学 | A kind of handwritten Chinese recognition methods based on deep learning |
| CN113723536A (en) * | 2021-09-02 | 2021-11-30 | 国网智能科技股份有限公司 | Power inspection target identification method and system |
| CN113723536B (en) * | 2021-09-02 | 2024-01-26 | 国网智能科技股份有限公司 | Electric power inspection target identification method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106897744A (en) | A kind of self adaptation sets the method and system of depth confidence network parameter | |
| CN103049792B (en) | Deep-neural-network distinguish pre-training | |
| CN109992779B (en) | Emotion analysis method, device, equipment and storage medium based on CNN | |
| CN106203625B (en) | A kind of deep-neural-network training method based on multiple pre-training | |
| CN104751228B (en) | Construction method and system for the deep neural network of speech recognition | |
| CN102622418B (en) | Prediction device and equipment based on BP (Back Propagation) nerve network | |
| CN110245741A (en) | Optimization and methods for using them, device and the storage medium of multilayer neural network model | |
| CN113905391B (en) | Integrated learning network traffic prediction method, system, equipment, terminal and medium | |
| CN108182260B (en) | Multivariate time sequence classification method based on semantic selection | |
| CN108197294A (en) | A kind of text automatic generation method based on deep learning | |
| CN112000772B (en) | Sentence-to-semantic matching method based on semantic feature cube and oriented to intelligent question and answer | |
| CN106709565A (en) | Optimization method and device for neural network | |
| CN106411896A (en) | APDE-RBF neural network based network security situation prediction method | |
| CN110020667A (en) | Searching method, system, storage medium and the equipment of neural network structure | |
| CN103593703A (en) | Neural network optimizing system and method based on genetic algorithms | |
| CN109726676B (en) | Planning method for automatic driving system | |
| CN103324954A (en) | Image classification method based on tree structure and system using same | |
| CN102622515A (en) | Weather prediction method | |
| CN112000770A (en) | Intelligent question and answer oriented sentence-to-sentence matching method based on semantic feature map | |
| CN113190688A (en) | Complex network link prediction method and system based on logical reasoning and graph convolution | |
| CN108009527A (en) | A kind of intelligent characteristic recognition methods towards STEP-NC2.5D manufacturing features | |
| CN109933659A (en) | A kind of vehicle-mounted more wheel dialogue methods towards trip field | |
| CN111353534B (en) | Graph data category prediction method based on adaptive fractional order gradient | |
| CN110033089A (en) | Deep neural network parameter optimization method and system based on Distributed fusion algorithm | |
| CN113537365A (en) | Multitask learning self-adaptive balancing method based on information entropy dynamic weighting |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination |