CN109284820A - A kind of search structure method and device of deep neural network - Google Patents
A kind of search structure method and device of deep neural network Download PDFInfo
- Publication number
- CN109284820A CN109284820A CN201811259033.2A CN201811259033A CN109284820A CN 109284820 A CN109284820 A CN 109284820A CN 201811259033 A CN201811259033 A CN 201811259033A CN 109284820 A CN109284820 A CN 109284820A
- Authority
- CN
- China
- Prior art keywords
- neural network
- computing unit
- module
- search
- information flow
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of search structure method and devices of deep neural network, are related to field of artificial intelligence.Method includes: every layer of computing unit structure in each module for obtaining in pre-set search space and being sequentially connected in series in deep neural network；Each computing unit is attached using default connection type in each module, obtains the information flow in each module；According to the connection of the computing unit in module and each module, initial neural network is obtained；Sparse scaling operator is arranged to the information flow in initial neural network, wherein sparse scaling operator is for zooming in and out information flow；The weight of initial neural network and the sparse scaling operator of information flow are trained using preset training sample data, obtain intermediate nerve network；The information flow that scaling operator sparse in intermediate nerve network is zero is deleted, the search result neural network in search space is obtained.The present invention can save the time of network structure search.
Description
Technical field
The present invention relates to field of artificial intelligence more particularly to the search structure methods and dress of a kind of deep neural network
It sets.
Background technique
In recent years, deep neural network achieved huge success in numerous areas, such as computer vision, natural language
Speech processing etc..Deep neural network is converted the feature of traditional hand-designed in order to end-to-end by powerful characterization ability
Study.However, structure is complicated for deep neural network at present, the computing units node such as convolution, pond is numerous, so that such as
Where in numerous computing unit nodes search obtain one compact-sized, the speed of service is very fast, effect again good model structure at
For a difficult point.
Currently available technology generally using search space is first defined, then searches for optimal network knot in search space
Structure.Network structure search can be carried out using the heuristic searched for based on the network structure of controller under normal circumstances,
Or network structure search is carried out using evolution algorithm.However, controller is needed to be trained or use in the prior art
Evolution algorithm carries out network structure search, needs in search process to convergence to come the sub-network training in complete or collected works to subnet
Network is assessed, so that the time of network structure search and calculation amount are very big, for biggish data set, are adopted this method and is searched
The process of rope to optimum network structure is cumbersome and slow.
Summary of the invention
The embodiment of the present invention provides a kind of search structure method and device of deep neural network, to solve the prior art
In network structure search time and calculation amount it is very big, for biggish data set, search the process of optimum network structure
Cumbersome and slow problem.
In order to achieve the above objectives, the present invention adopts the following technical scheme:
On the one hand, the present invention provides a kind of search structure method of deep neural network, comprising:
Every layer of meter in each module being sequentially connected in series in deep neural network is obtained in pre-set search space
Calculate cellular construction；Every layer of computing unit structure includes at least one computing unit；
Each computing unit is attached using default connection type in each module, obtains the information in each module
Stream；Wherein, between the computing unit in same layer computing unit structure without connection, each computing unit can with and its
The computing unit of different layers in the module and outputting and inputting for module where it be attached；
According to the connection of the computing unit in module and each module, initial neural network is obtained；
Sparse scaling operator is arranged to the information flow in the initial neural network, wherein the sparse scaling operator is used for
The information flow is zoomed in and out；
Using preset training sample data to the weight of the initial neural network and the sparse scaling operator of information flow
It is trained, obtains intermediate nerve network；
The information flow that scaling operator sparse in the intermediate nerve network is zero is deleted, the search in search space is obtained
As a result neural network.
On the other hand, the present invention provides a kind of object detection method, comprising:
The sample data for obtaining pending target detection is input to the search structure side using above-mentioned deep neural network
In the search result neural network that method obtains, using the output of described search result neural network as object detection results.
On the other hand, the present invention provides a kind of semantic segmentation method, comprising:
The sample data for obtaining pending semantic segmentation is input to the search structure side using above-mentioned deep neural network
In the search result neural network that method obtains, using the output of described search result neural network as semantic segmentation result.
Another aspect, the present invention provide a kind of search structure device of deep neural network, comprising:
Computing unit structure obtaining unit, for being obtained in deep neural network successively in pre-set search space
Every layer of computing unit structure in each module of concatenation；Every layer of computing unit structure includes at least one computing unit；
Information flow obtaining unit, for each computing unit to be attached using default connection type in each module,
Obtain the information flow in each module；Wherein, without connection between the computing unit in same layer computing unit structure, often
A computing unit can with its different layers in the module computing unit and module where it output and input into
Row connection；
Initial neural network obtaining unit is obtained for the connection according to the computing unit in module and each module
To initial neural network；
Sparse scaling operator setting unit is calculated for sparse scaling to be arranged to the information flow in the initial neural network
Son, wherein the sparse scaling operator is for zooming in and out the information flow；
Weight and operator training unit, for the weight using preset training sample data to the initial neural network
It is trained with the sparse scaling operator of information flow, obtains intermediate nerve network；
Search result obtaining unit, for by the intermediate nerve network it is sparse scaling operator be zero information flow delete
It removes, obtains the search result neural network in search space.
In another aspect, the present invention provides a kind of computer readable storage medium, it is stored thereon with computer program, feature
It is, which realizes the search structure method of above-mentioned deep neural network when being executed by processor.
In another aspect, the present invention provides a kind of computer equipment, including memory, processor and to be stored in storage upper and can
The computer program run on a processor, the processor realize the knot of above-mentioned deep neural network when executing described program
Structure searching method.
The search structure method and device of a kind of deep neural network provided in an embodiment of the present invention, firstly, being set in advance
Every layer of computing unit structure in each module being sequentially connected in series in deep neural network is obtained in the search space set；Every layer of meter
Calculating cellular construction includes at least one computing unit；Later, use default connection type by each computing unit in each module
It is attached, obtains the information flow in each module；Wherein, between the computing unit in same layer computing unit structure not into
Row connection, each computing unit can with its institute's different layers in the module computing unit and its place module it is defeated
Enter and output is attached；Then, according to the connection of the computing unit in module and each module, initial nerve net is obtained
Network；Sparse scaling operator is arranged to the information flow in initial neural network, wherein sparse scaling operator is used to carry out information flow
Scaling；The weight of initial neural network and the sparse scaling operator of information flow are instructed using preset training sample data
Practice, obtains intermediate nerve network；In turn, the information flow that scaling operator sparse in intermediate nerve network is zero is deleted, is searched
Search result neural network in rope space.The present invention and important network structure is directly searched for from search space in the prior art
Difference, the present invention can delete unessential information flow by sparse scaling operator to realize the search of network structure.The present invention exists
In the search process of network structure, without being trained to controller, without complicated evolution algorithm is used, antithetical phrase is not needed
Network is trained for a long time, search result only can be obtained by the training to weight and sparse scaling operator, so that net
The time of network search structure greatly reduces, and searches for especially for the network structure on large-scale dataset, more saving network
The time of search structure.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention without any creative labor, may be used also for those of ordinary skill in the art
To obtain other drawings based on these drawings.
Fig. 1 is a kind of flow chart one of the search structure method of deep neural network provided in an embodiment of the present invention；
Fig. 2 is the network structure signal in the search space in deep neural network involved in the embodiment of the present invention
Figure；
Fig. 3 is the example schematic that the embodiment of the present invention is applied to the web search of double-layer structure；
Fig. 4 is a kind of structural schematic diagram of the search structure device of deep neural network provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
To facilitate the understanding of the present invention, technical term according to the present invention is explained below:
DNN: deep neural network (Deep Neural Network).
Computing unit: the cell node for being used to carry out the calculating such as convolution, pond in neural network.
Network structure search: the process of optimum network structure is searched in neural network.
During realizing the embodiment of the present invention, inventor has found that the prior art generally uses the network based on controller
The heuristic of search structure, it may be assumed that
According to priori knowledge and deep neural network structure (specific structures such as neuron, network layer, mould group, module) come structure
Build some network structures to be searched；Then controller is set for network structure to be searched, in such a way that distribution solves,
I.e. for each controller, the parallel computation of multiple network structures to be searched is carried out, the accuracy rate of each network structure is obtained
For carrying out gradient descent algorithm to controller, to obtain optimum network structure.As it can be seen that for using the net based on controller
The heuristic of network search structure needs to be trained big amount controller, and distributed solution, and process is relatively complicated slow
Slowly.
In order to solve above-mentioned the problems of the prior art, as shown in Figure 1, the embodiment of the present invention provides a kind of depth nerve net
The search structure method of network, comprising:
Step 101 is obtained in pre-set search space in each module being sequentially connected in series in deep neural network
Every layer of computing unit structure.
Wherein, every layer of computing unit structure includes at least one computing unit.
Step 102 uses default connection type to be attached each computing unit in each module, obtains each module
In information flow.
Wherein, without connection between the computing unit in same layer computing unit structure, each computing unit can
With with its different layers in the module computing unit and outputting and inputting for module where it be attached.
Step 103, according to the connection of the computing unit in module and each module, obtain initial neural network.
Sparse scaling operator is arranged to the information flow in initial neural network in step 104, wherein sparse scaling operator is used for
Information flow is zoomed in and out.
Step 105, using preset training sample data to the weight of initial neural network and the sparse scaling of information flow
Operator is trained, and obtains intermediate nerve network.
Step 106 deletes the information flow that scaling operator sparse in intermediate nerve network is zero, obtains in search space
Search result neural network.
It is worth noting that the pre-set search space of institute can be as shown in Fig. 2, wherein in deep neural network
It may include multiple modules 21, modules 21 are sequentially connected in series, i.e., the output of a upper module is the input of next module；Each
Module 21 (can be considered as directed acyclic graph) may include multilayer computing unit structure 22, and every layer of computing unit structure 22 includes at least
One computing unit 23 (each computing unit can be considered as the node in directed acyclic graph), in this every layer computing unit structure 22
Computing unit 23 generally may include at least one of convolutional calculation unit and pond computing unit.The convolutional calculation unit
It can also be expansion convolutional calculation unit or group convolutional calculation unit etc..
Preferably, above-mentioned steps 102 can be accomplished in that
In each module 21, each computing unit 23 is attached using full connection type, i.e., as shown in Fig. 2, it will
Each computing unit 23 with and the different layers where it in module 21 computing unit 23 and its place module 21 input and
Output is attached；It is available in this way to be input to every layer of computing unit structure 22, from every layer of computing unit knot from module 21
Structure 22 (can be considered between directed acyclic graph interior joint to the information flow between the output and each computing unit 23 of module 21
Side).(any network structure in the search space can be considered as the complete or collected works of network structure in search space available in this way
The subgraph of above-mentioned directed acyclic graph).For example, in a module 21, i-th of computing unit F(i)(x) output h (i), is equal to
The sum of output h (j) of computing unit before all passes through computing unit F(i)(x) calculate as a result, can be formulated
Are as follows:
In this way, according to above-mentioned structure shown in Fig. 2, initial neural network can be obtained in above-mentioned steps 103.
Further, after above-mentioned steps 103, the weight of initial neural network can be configured, with initialization
The weight of initial neural network.Or, it is preferable that after above-mentioned steps 103, preset pre-training sample data can be used
Pre-training is carried out to the weight of the initial neural network, the initial neural network after obtaining pre-training, after such pre-training,
The weight of obtained initial neural network is preferable.It is initial refreshing in order to obtain for being configured or carried out herein pre-training to weight
Weight initial value through network, in order to the setting and training of subsequent sparse scaling operator.
Later in above-mentioned steps 104, need that sparse scaling operator, i.e. example is arranged to the information flow in initial neural network
Such as it is above-mentioned it is all before computing unit output h (j) at increase sparse scaling operatorFor indicating that j-th of calculating is single
Sparse scaling operator of the member to information flow between i-th of computing unit.Then above-mentioned formula (1) is after increasing sparse scaling operator,
It should indicate are as follows:
Herein, the value of each sparse scaling operator is more than or equal to 0.For example, the above-mentioned weight to initial neural network into
Row configures, and after the weight to initialize initial neural network, value interval of the sparse scaling operator can be [0,1], sparse
Scaling operator needs not be equal to 1.And it is carried out above-mentioned using weight of the preset pre-training sample data to initial neural network
After pre-training, the value of the sparse scaling operator is generally taken as 1.
It is illustrated below with the search of a convolution neural network structure, in convolutional neural networks structure, computing unit
As convolutional calculation unit and pond computing unit, information flow are the characteristic pattern in network.In the convolutional neural networks structure
In, several modules are contained, each module includes several layers computing unit structure, if each layer of computing unit structure includes again
Different computing units is done (for example, 1 × 1 convolutional calculation, 3 × 3 convolutional calculation, 5 × 5 convolutional calculation, pondization calculate
Deng being not only limited to above-mentioned these types).Modules are sequentially connected in series, i.e., the output of a upper module is the defeated of next module
Enter, each computing unit with and its computing unit of different layers in the module and outputting and inputting for its place module
It is attached.In this way, the output of each computing unit can be represented, for example, in convolutional neural networks structure, b
The output of i-th layer of j-th of computing unit of a module can indicate are as follows:
Wherein, F(b,i,j)(x) calculating of i-th layer of j-th of computing unit of b-th of module is indicated；N indicates one layer of calculating
The computing unit sum that cellular construction is included；Indicate m layers of n-th of computing unit of b-th of module to b-th
The sparse scaling operator of information flow between i-th layer of j-th of computing unit of module；H (b, m, n) indicates the m of b-th of module
The output of n-th of computing unit of layer；O (b-1) indicates the output of the b-1 module, i.e., the input of b-th module；Table
Show the sparse scaling of information flow between j-th of computing unit of the input O (b-1) to i-th layer of b-th of module of b-th of module
Operator.Herein, if input of h (b, 0, the 0)=O (b-1) as b-th of module, if h (b, M+1,0)=O (b) is used as b-th of mould
The output of block, wherein M indicates the layer sum that b-th of module is included.It can determine that the computing unit positioned at m layers shares in this way
(m-1) N+1 input.
Herein, it should be noted that in embodiments of the present invention, company of each computing unit to module outlet chamber where it
Connect is also that can train study.For example, the output O (b) of b-th of module can be by this in above-mentioned convolutional neural networks
The output of all computing units is spliced in module, and reusing the convolution that convolution kernel size is 1 reduces the port number of characteristic pattern
Keep port number constant, shown in following formula:
Wherein, h (b, m, n) is indicated in b-th of module, the output of n-th of computing unit in m layers,It indicates in b-th of module, the information flow of n-th of computing unit and b-th of module output connection in m layers
Scaling operator, O (b-1) indicate the b-1 module output, i.e., the input of b-th module.The splicing of R (x) expression characteristic pattern
The convolutional calculation for being 1 with convolution kernel size for fusion feature figure and guarantees that the port number of module output is constant.
For above-mentioned steps 105, can realize in the following way:
Step S1, construct the corresponding objective function of initial neural network, the objective function include loss function, weight just
Then function and sparse regular function.The objective function can be such as formula:
Wherein, W is weight, and λ is sparse scaling operator vector, and K is the quantity of sample data, L (yi,Net(xi, W, λ)) be
Neural network is in sample data xiOn loss, yiFor sample label, Net (xi, W, λ) be neural network output,For
Weight regular function is denoted as R (W), parameter decaying weight of the δ for weight W, γ | | λ | |1For sparse regular function, it is denoted as Rs
(λ).In addition, sparse regular function γ herein | | λ | |1It can also be substituted by more complicated sparse constraint, such as non-convex dilute
Dredge constraint.
Step S2, training is iterated to the initial neural network using the training sample data.
Step S3, it when repetitive exercise number reaches threshold value or the objective function meets the preset condition of convergence, obtains
To the intermediate nerve network.
Preferably, abovementioned steps S2 specific implementation can by initial neural network carry out repeatedly repetitive exercise below,
To be described for the iterative process (hereinafter referred to as current iteration training) of primary non-iteration for the first time and non-tail time iteration, once
Repetitive exercise includes the following steps C1~step C3:
Step C1, the sparse scaling operator for obtaining a preceding repetitive exercise is as the constant of the objective function, by institute
Variable of the weight as the objective function is stated, the objective function is optimized using the first optimization algorithm, obtains this
The weight of repetitive exercise；
Step C2, using the weight of current iteration training as the constant of the objective function, using sparse scaling operator as
The variable of the objective function optimizes the objective function using the second optimization algorithm, obtains current iteration training
Sparse scaling operator；
Step C3, weight and sparse scaling operator based on current iteration training carry out next iteration training.
In addition, repetitive exercise process is as follows for the first time:, will using initial sparse scaling operator as the constant of the objective function
Variable of the weight as the objective function optimizes the objective function using the first optimization algorithm, obtains this
The weight of secondary repetitive exercise；Using the weight of current iteration training as the constant of the objective function, sparse scaling operator is made
For the variable of the objective function, the objective function is optimized using the second optimization algorithm, obtains current iteration training
Sparse scaling operator；Weight and sparse scaling operator based on current iteration training carry out second of repetitive exercise.
In addition, tail time repetitive exercise process is as follows: the sparse scaling operator that a preceding repetitive exercise is obtained is as described in
The constant of objective function, using the weight as the variable of the objective function, using the first optimization algorithm to the target letter
Number optimizes, and obtains the weight of current iteration training；Using the weight of current iteration training as the constant of the objective function,
Using sparse scaling operator as the variable of the objective function, the objective function is optimized using the second optimization algorithm,
Obtain the sparse scaling operator of current iteration training；By the mind of sparse the scaling operator and weight that are obtained comprising current iteration training
Through network as intermediate nerve network.
Herein, in embodiments of the present invention, which can be, but not limited to as any one following algorithm: with
Machine gradient descent algorithm, the mutation algorithm for introducing momentum.
Herein, in embodiments of the present invention, which can be, but not limited to as any one following algorithm: add
Fast proximal end gradient descent algorithm, proximal end gradient descent algorithm or alternating direction Multiplier Algorithm.
For further the W and λ that how are solved in objective function in the embodiment of the present invention are described in detail, below
By taking objective function is above-mentioned formula (5) as an example, an iteration training optimization object function is solved to obtain W and λ is described.It willIt is denoted as g (λ), Rs (λ) is denoted as H (λ).
Using λ as constant, using W as variable, then objective function, which changes, switchs toUsing stochastic gradient
Descent algorithm can solve to obtain the value of W, and detailed process is not described in detail.
Using W as constant, using λ as variable, then objective function, which changes, switchs toUsing acceleration proximal end
Gradient descent algorithm solves the value of λ, can specifically pass through but be not limited only to following methods and obtains:
Mode 1 obtains λ using formula (6)~formula (8):
λt=prox ηtH(zt) formula (8)
Wherein η t indicates the step-length that gradient declines in the t times repetitive exercise,
For soft-threshold operator, it is defined as follows Sα(z)i=sign (zi)(|zi|-α)+。
Mode 2 is solved λ due to foregoing manner 1 and needs additional forward-backward algorithm to calculate to obtainThe algorithm is straight
It scoops out and uses existing deep learning frame and have a difficulty.Therefore, mode 2 is updated the formula of foregoing manner 1, obtains formula
(9) λ is calculated according to formula (9)~formula (11) in~formula (11):
λt=λt-1+vtFormula (11)
Mode 3, the present invention can also use variable substitution method, i.e., λ is calculated using following formula (12)~(14):
Wherein λ 't-1=λt-1+μt-1vt-1, μ is preset fixed value, and come more in the form of batch stochastic gradient descent
New W and λ.
Later, in above-mentioned steps 106, the information flow that scaling operator sparse in intermediate nerve network is zero can be deleted
It removes, obtains the search result neural network in search space.Also, in information flow corresponding with the connection of a computing unit quilt
After deletion, then the computing unit can then delete the computing unit to subsequent calculating without effect.
For example, as shown in figure 3, the embodiment of the present invention is applied in picture classification task.Contain in setting basic network
Double-layer structure Level1 and Level2, the connection there are two different computing unit OP1 and OP2 in every layer, between computing unit
As shown in the leftmost side of Fig. 3.After above-mentioned steps 101 to step 105, it can train to obtain shown in the centre in Fig. 3
The sparse scaling operator of dotted line is 0.And then as shown in the rightmost side of Fig. 3, after these dotted lines are deleted, the meter of Level1 layers of confirmation
The connectionless corresponding information flow of unit OP1 is calculated, then is also deleted, finally obtains search result neural network.
It is worth noting that Fig. 3 exemplifications set out is only a concrete application of the embodiment of the present invention, and not all
Using.The embodiment of the present invention is located at the sparse of network disparate modules in addition to applying other than individual module search structure in the present invention
Scaling operator can be enabled disparate modules to search for training and be obtained more flexible network structure with independently updated.
In addition, the embodiment of the present invention also provides a kind of object detection method, comprising:
The sample data for obtaining pending target detection, the structure for being input to the corresponding deep neural network of above-mentioned Fig. 1 are searched
In the search result neural network that Suo Fangfa is obtained, using the output of described search result neural network as object detection results.
In addition, the embodiment of the present invention also provides a kind of semantic segmentation method, comprising:
The sample data for obtaining pending semantic segmentation, the structure for being input to the corresponding deep neural network of above-mentioned Fig. 1 are searched
In the search result neural network that Suo Fangfa is obtained, using the output of described search result neural network as semantic segmentation result.
The search structure method of the corresponding deep neural network of Fig. 1 is not limited solely to apply in target detection and semantic point
It cuts in task, can be also used for will not enumerate herein in other different tasks.
In addition, as shown in figure 4, the embodiment of the present invention also provides a kind of search structure device of deep neural network, it is special
Sign is, comprising:
Computing unit structure obtaining unit 31, in pre-set search space obtain deep neural network according to
Every layer of computing unit structure in each module of secondary concatenation；Every layer of computing unit structure includes that at least one calculates list
Member.
Information flow obtaining unit 32, for being connected each computing unit using default connection type in each module
It connects, obtains the information flow in each module；Wherein, without even between the computing unit in same layer computing unit structure
Connect, each computing unit can with its different layers in the module computing unit and the input of module where it and
Output is attached.
Initial neural network obtaining unit 33, for the connection according to the computing unit in module and each module,
Obtain initial neural network.
Sparse scaling operator setting unit 34 is calculated for sparse scaling to be arranged to the information flow in the initial neural network
Son, wherein the sparse scaling operator is for zooming in and out the information flow.
Weight and operator training unit 35, for the power using preset training sample data to the initial neural network
The sparse scaling operator of weight and information flow is trained, and obtains intermediate nerve network.
Search result obtaining unit 36, for by the intermediate nerve network it is sparse scaling operator be zero information flow delete
It removes, obtains the search result neural network in search space.
In addition, the embodiment of the present invention also provides a kind of computer readable storage medium, it is stored thereon with computer program,
It is characterized in that, which realizes the search structure method of the corresponding deep neural network of above-mentioned Fig. 1 when being executed by processor.
In addition, the embodiment of the present invention also provides a kind of computer equipment, including memory, processor and it is stored in storage
And the computer program that can be run on a processor, the processor realize the corresponding depth of above-mentioned Fig. 1 when executing described program
The search structure method of neural network.
In conclusion a kind of search structure method and device of deep neural network provided in an embodiment of the present invention, firstly,
Every layer of computing unit knot in each module being sequentially connected in series in deep neural network is obtained in pre-set search space
Structure；Every layer of computing unit structure includes at least one computing unit；It later, will be each using default connection type in each module
Computing unit is attached, and obtains the information flow in each module；Wherein, the computing unit in same layer computing unit structure
Between without connection, each computing unit can with its institute's different layers in the module computing unit and its place
Outputting and inputting for module is attached；Then, it according to the connection of the computing unit in module and each module, obtains just
Beginning neural network；Sparse scaling operator is arranged to the information flow in initial neural network, wherein sparse scaling operator is used for letter
Breath stream zooms in and out；Using preset training sample data to the weight of initial neural network and the sparse scaling operator of information flow
It is trained, obtains intermediate nerve network；In turn, the information flow that scaling operator sparse in intermediate nerve network is zero is deleted,
Obtain the search result neural network in search space.The present invention and important net is directly searched for from search space in the prior art
Network structure is different, and the present invention can delete unessential information flow by sparse scaling operator to realize the search of network structure.This
Invention is in the search process of network structure, without being trained to controller, without complicated evolution algorithm is used, is not required to
Sub-network is trained for a long time, search result only can be obtained by the training to weight and sparse scaling operator,
So that the time of network structure search greatly reduces, searches for especially for the network structure on large-scale dataset, more save
Save the time of network structure search.
It should be understood by those skilled in the art that, the embodiment of the present invention can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the present invention
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the present invention, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The present invention be referring to according to the method for the embodiment of the present invention, the process of equipment (system) and computer program product
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
Specific embodiment is applied in the present invention, and principle and implementation of the present invention are described, above embodiments
Explanation be merely used to help understand method and its core concept of the invention；At the same time, for those skilled in the art,
According to the thought of the present invention, there will be changes in the specific implementation manner and application range, in conclusion in this specification
Appearance should not be construed as limiting the invention.
Claims (15)
1. a kind of search structure method of deep neural network characterized by comprising
The every layer of calculating obtained in each module being sequentially connected in series in deep neural network in pre-set search space is single
Meta structure；Every layer of computing unit structure includes at least one computing unit；
Each computing unit is attached using default connection type in each module, obtains the information flow in each module；
Wherein, between the computing unit in same layer computing unit structure without connection, each computing unit can with its institute
The computing unit of different layers in the module and outputting and inputting for module where it are attached；
According to the connection of the computing unit in module and each module, initial neural network is obtained；
Sparse scaling operator is arranged to the information flow in the initial neural network, wherein the sparse scaling operator is used for institute
Information flow is stated to zoom in and out；
The weight of the initial neural network and the sparse scaling operator of information flow are carried out using preset training sample data
Training, obtains intermediate nerve network；
The information flow that scaling operator sparse in the intermediate nerve network is zero is deleted, the search result in search space is obtained
Neural network.
2. the search structure method of deep neural network according to claim 1, which is characterized in that every layer of calculating is single
The computing unit of meta structure includes at least one of convolutional calculation unit and pond computing unit.
3. the search structure method of deep neural network according to claim 1, which is characterized in that adopted in each module
Each computing unit is attached with default connection type, obtains the information flow in each module, comprising:
In each module, by each computing unit with and its different layers in the module computing unit and its place
Outputting and inputting for module is attached；It obtains being input to every layer of computing unit structure, from every layer of computing unit knot from module
Structure is to the information flow between the output and each computing unit of module.
4. the search structure method of deep neural network according to claim 1, which is characterized in that according to module and often
The connection of computing unit in a module, after obtaining initial neural network, further includes:
The weight of initial neural network is configured, to initialize the weight of initial neural network.
5. the search structure method of deep neural network according to claim 1, which is characterized in that according to module and often
The connection of computing unit in a module, after obtaining initial neural network, further includes:
Pre-training is carried out using weight of the preset pre-training sample data to the initial neural network, after obtaining pre-training
Initial neural network.
6. the search structure method of deep neural network according to claim 1, which is characterized in that will the intermediate mind
After being deleted through the information flow that scaling operator sparse in network is zero, further includes:
After information flow corresponding with the connection of a computing unit is deleted, which is deleted.
7. the search structure method of deep neural network according to claim 1, which is characterized in that described using preset
Training sample data are trained the weight of the initial neural network and the sparse scaling operator of information flow, obtain intermediate mind
Through network, comprising:
Construct the corresponding objective function of initial neural network, the objective function includes loss function, weight regular function and dilute
Dredge regular function；
Training is iterated to the initial neural network using the training sample data；
When repetitive exercise number reaches threshold value or the objective function meets the preset condition of convergence, the intermediate mind is obtained
Through network.
8. the search structure method of deep neural network according to claim 7, which is characterized in that described to use the instruction
Practice sample data and training be iterated to the initial neural network, specifically includes:
Repeatedly repetitive exercise below is carried out to the initial neural network:
The sparse scaling operator that a preceding repetitive exercise is obtained is as the constant of the objective function, using the weight as institute
The variable for stating objective function optimizes the objective function using the first optimization algorithm, obtains the power of current iteration training
Weight；
Using the weight of current iteration training as the constant of the objective function, using sparse scaling operator as the objective function
Variable, the objective function is optimized using the second optimization algorithm, obtain current iteration training sparse scaling operator；
Weight and sparse scaling operator based on current iteration training carry out next iteration training.
9. the search structure method of deep neural network according to claim 8, which is characterized in that second optimization is calculated
Method is to accelerate proximal end gradient descent algorithm, proximal end gradient descent algorithm or alternating direction Multiplier Algorithm.
10. the method according to the description of claim 7 is characterized in that the objective function are as follows:
Wherein, W is weight, and λ is sparse scaling operator vector, and K is the quantity of sample data, L (yi,Net(xi, W, λ)) it is nerve
Network is in sample data xiOn loss, yiFor sample label, Net (xi, W, λ) be neural network output,For weight
Regular function, parameter decaying weight of the δ for weight W, γ | | λ | |1For sparse regular function.
11. a kind of object detection method characterized by comprising
The sample data for obtaining pending target detection is input to using the described in any item depth nerves of claims 1 to 10
In the search result neural network that the search structure method of network obtains, using the output of described search result neural network as mesh
Mark testing result.
12. a kind of semantic segmentation method characterized by comprising
The sample data for obtaining pending semantic segmentation is input to using the described in any item depth nerves of claims 1 to 10
In the search result neural network that the search structure method of network obtains, using the output of described search result neural network as language
Adopted segmentation result.
13. a kind of search structure device of deep neural network characterized by comprising
Computing unit structure obtaining unit is sequentially connected in series for obtaining in deep neural network in pre-set search space
Each module in every layer of computing unit structure；Every layer of computing unit structure includes at least one computing unit；
Information flow obtaining unit is obtained for being attached each computing unit using default connection type in each module
Information flow in each module；Wherein, without connection, Mei Geji between the computing unit in same layer computing unit structure
Calculating unit can be with the company of outputting and inputting of computing unit and its place module with its institute's different layers in the module
It connects；
Initial neural network obtaining unit obtains just for the connection according to the computing unit in module and each module
Beginning neural network；
Sparse scaling operator setting unit, for sparse scaling operator to be arranged to the information flow in the initial neural network,
Described in sparse scaling operator for being zoomed in and out to the information flow；
Weight and operator training unit, for the weight and letter using preset training sample data to the initial neural network
The sparse scaling operator of breath stream is trained, and obtains intermediate nerve network；
Search result obtaining unit is obtained for the information flow deletion for being zero by scaling operator sparse in the intermediate nerve network
Search result neural network in search space.
14. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
Claims 1 to 10 described in any item methods are realized when execution.
15. a kind of computer equipment including memory, processor and is stored in the calculating that storage is upper and can run on a processor
Machine program, which is characterized in that the processor realizes the described in any item methods of claims 1 to 10 when executing described program.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811259033.2A CN109284820A (en) | 2018-10-26 | 2018-10-26 | A kind of search structure method and device of deep neural network |
| PCT/CN2019/077049 WO2020082663A1 (en) | 2018-10-26 | 2019-03-05 | Structural search method and apparatus for deep neural network |
| CN201911007284.6A CN110717586A (en) | 2018-10-26 | 2019-10-22 | Structure search method and device for deep neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811259033.2A CN109284820A (en) | 2018-10-26 | 2018-10-26 | A kind of search structure method and device of deep neural network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109284820A true CN109284820A (en) | 2019-01-29 |
Family
ID=65177420
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811259033.2A Pending CN109284820A (en) | 2018-10-26 | 2018-10-26 | A kind of search structure method and device of deep neural network |
| CN201911007284.6A Pending CN110717586A (en) | 2018-10-26 | 2019-10-22 | Structure search method and device for deep neural network |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911007284.6A Pending CN110717586A (en) | 2018-10-26 | 2019-10-22 | Structure search method and device for deep neural network |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN109284820A (en) |
| WO (1) | WO2020082663A1 (en) |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919304A (en) * | 2019-03-04 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Neural network searching method, device, readable storage medium storing program for executing and computer equipment |
| CN109948795A (en) * | 2019-03-11 | 2019-06-28 | 驭势科技（北京）有限公司 | A kind of method and apparatus of determining network structure precision and delay Optimization point |
| CN109978142A (en) * | 2019-03-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | The compression method and device of neural network model |
| CN110197258A (en) * | 2019-05-29 | 2019-09-03 | 北京市商汤科技开发有限公司 | Neural network searching method, image processing method and device, equipment and medium |
| CN110276442A (en) * | 2019-05-24 | 2019-09-24 | 西安电子科技大学 | A kind of searching method and device of neural network framework |
| CN110428046A (en) * | 2019-08-28 | 2019-11-08 | 腾讯科技（深圳）有限公司 | Acquisition methods and device, the storage medium of neural network structure |
| CN110473195A (en) * | 2019-08-13 | 2019-11-19 | 中山大学 | It is a kind of can automatic customization medicine lesion detection framework and method |
| CN110490323A (en) * | 2019-08-20 | 2019-11-22 | 腾讯科技（深圳）有限公司 | Network model compression method, device, storage medium and computer equipment |
| CN110751267A (en) * | 2019-09-30 | 2020-02-04 | 京东城市(北京)数字科技有限公司 | Neural network structure searching method, training method, device and storage medium |
| CN110826696A (en) * | 2019-10-30 | 2020-02-21 | 北京百度网讯科技有限公司 | Search space construction method and device of hyper network and electronic equipment |
| WO2020082663A1 (en) * | 2018-10-26 | 2020-04-30 | 北京图森未来科技有限公司 | Structural search method and apparatus for deep neural network |
| CN111090673A (en) * | 2019-12-20 | 2020-05-01 | 北京百度网讯科技有限公司 | Cache unit searching method and related equipment |
| CN111160515A (en) * | 2019-12-09 | 2020-05-15 | 中山大学 | Running time prediction method, model search method and system |
| CN111191785A (en) * | 2019-12-20 | 2020-05-22 | 沈阳雅译网络技术有限公司 | Structure searching method based on expanded search space |
| CN111401516A (en) * | 2020-02-21 | 2020-07-10 | 华为技术有限公司 | Neural network channel parameter searching method and related equipment |
| CN111667057A (en) * | 2020-06-05 | 2020-09-15 | 北京百度网讯科技有限公司 | Method and apparatus for searching model structure |
| CN111684472A (en) * | 2019-05-31 | 2020-09-18 | 深圳市大疆创新科技有限公司 | Method and apparatus for network structure search, computer storage medium, and computer program product |
| CN111684471A (en) * | 2019-05-31 | 2020-09-18 | 深圳市大疆创新科技有限公司 | Method and apparatus for network structure search, computer storage medium, and computer program product |
| CN111950710A (en) * | 2020-08-12 | 2020-11-17 | 深圳市商汤科技有限公司 | Neural network optimization method and device, electronic equipment and storage medium |
| CN112052865A (en) * | 2019-06-05 | 2020-12-08 | 北京京东尚科信息技术有限公司 | Method and apparatus for generating neural network model |
| WO2021008206A1 (en) * | 2019-07-12 | 2021-01-21 | 华为技术有限公司 | Neural architecture search method, and image processing method and device |
| CN112464930A (en) * | 2019-09-09 | 2021-03-09 | 华为技术有限公司 | Target detection network construction method, target detection method, device and storage medium |
| WO2021057690A1 (en) * | 2019-09-24 | 2021-04-01 | 华为技术有限公司 | Neural network building method and device, and image processing method and device |
| CN113379034A (en) * | 2021-06-15 | 2021-09-10 | 南京大学 | Neural network structure optimization method based on network structure search technology |
| WO2021253671A1 (en) * | 2020-06-18 | 2021-12-23 | 中国科学院深圳先进技术研究院 | Magnetic resonance cine imaging method and apparatus, and imaging device and storage medium |
| WO2021253938A1 (en) * | 2020-06-19 | 2021-12-23 | 深圳市商汤科技有限公司 | Neural network training method and apparatus, and video recognition method and apparatus |
| CN112308200B (en) * | 2019-07-30 | 2024-04-26 | 华为技术有限公司 | Searching method and device for neural network |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113361680B (en) * | 2020-03-05 | 2024-04-12 | 华为云计算技术有限公司 | Neural network architecture searching method, device, equipment and medium |
| CN111753953A (en) * | 2020-05-13 | 2020-10-09 | 北京迈格威科技有限公司 | Method and device for acquiring neural network architecture |
| CN111797983A (en) * | 2020-05-25 | 2020-10-20 | 华为技术有限公司 | Neural network construction method and device |
| CN113743168B (en) * | 2020-05-29 | 2023-10-13 | 北京机械设备研究所 | Urban flyer identification method based on micro-depth neural network search |
| CN111738418A (en) * | 2020-06-19 | 2020-10-02 | 北京百度网讯科技有限公司 | Training method and device for hyper network |
| CN113902088A (en) * | 2020-06-22 | 2022-01-07 | 华为技术有限公司 | Method, device and system for searching neural network structure |
| CN111753964A (en) * | 2020-06-29 | 2020-10-09 | 北京百度网讯科技有限公司 | Neural network training method and device |
| CN112100466A (en) * | 2020-09-25 | 2020-12-18 | 北京百度网讯科技有限公司 | Method, device and equipment for generating search space and storage medium |
| CN112528123A (en) * | 2020-12-18 | 2021-03-19 | 北京百度网讯科技有限公司 | Model searching method, model searching apparatus, electronic device, storage medium, and program product |
| CN112560985B (en) * | 2020-12-25 | 2024-01-12 | 北京百度网讯科技有限公司 | Neural network searching method and device and electronic equipment |
| CN112668702B (en) * | 2021-01-15 | 2023-09-19 | 北京格灵深瞳信息技术股份有限公司 | Fixed-point parameter optimization method, system, terminal and storage medium |
| CN112966812A (en) * | 2021-02-25 | 2021-06-15 | 中国人民解放军战略支援部队航天工程大学 | Automatic neural network structure searching method for communication signal modulation recognition |
| CN113762026A (en) * | 2021-03-04 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Combined processing method, equipment and product for human body key point detection and position analysis |
| CN113326922B (en) * | 2021-05-31 | 2023-06-13 | 北京市商汤科技开发有限公司 | Neural network generation method and device, electronic equipment and storage medium |
| CN113469010B (en) * | 2021-06-25 | 2024-04-02 | 中国科学技术大学 | NOx concentration real-time estimation method based on black smoke image of diesel vehicle and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106372721B (en) * | 2016-08-29 | 2018-08-21 | 中国传媒大学 | The 3D method for visualizing of Large Scale Neural Networks |
| CN111178520B (en) * | 2017-06-15 | 2024-06-07 | 北京图森智途科技有限公司 | Method and device for constructing neural network |
| CN107316079A (en) * | 2017-08-08 | 2017-11-03 | 珠海习悦信息技术有限公司 | Processing method, device, storage medium and the processor of terminal convolutional neural networks |
| CN107480774A (en) * | 2017-08-11 | 2017-12-15 | 山东师范大学 | Dynamic neural network model training method and device based on integrated study |
| CN109284820A (en) * | 2018-10-26 | 2019-01-29 | 北京图森未来科技有限公司 | A kind of search structure method and device of deep neural network |
-
2018
- 2018-10-26 CN CN201811259033.2A patent/CN109284820A/en active Pending
-
2019
- 2019-03-05 WO PCT/CN2019/077049 patent/WO2020082663A1/en active Application Filing
- 2019-10-22 CN CN201911007284.6A patent/CN110717586A/en active Pending
Cited By (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020082663A1 (en) * | 2018-10-26 | 2020-04-30 | 北京图森未来科技有限公司 | Structural search method and apparatus for deep neural network |
| CN109919304B (en) * | 2019-03-04 | 2021-07-02 | 腾讯科技（深圳）有限公司 | Image processing method, image processing device, readable storage medium and computer equipment |
| CN109919304A (en) * | 2019-03-04 | 2019-06-21 | 腾讯科技（深圳）有限公司 | Neural network searching method, device, readable storage medium storing program for executing and computer equipment |
| CN109948795A (en) * | 2019-03-11 | 2019-06-28 | 驭势科技（北京）有限公司 | A kind of method and apparatus of determining network structure precision and delay Optimization point |
| CN109948795B (en) * | 2019-03-11 | 2021-12-14 | 驭势科技（北京）有限公司 | Method and device for determining network structure precision and delay optimization point |
| CN109978142A (en) * | 2019-03-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | The compression method and device of neural network model |
| CN109978142B (en) * | 2019-03-29 | 2022-11-29 | 腾讯科技（深圳）有限公司 | Neural network model compression method and device |
| CN110276442A (en) * | 2019-05-24 | 2019-09-24 | 西安电子科技大学 | A kind of searching method and device of neural network framework |
| CN110276442B (en) * | 2019-05-24 | 2022-05-17 | 西安电子科技大学 | Searching method and device of neural network architecture |
| CN110197258B (en) * | 2019-05-29 | 2021-10-29 | 北京市商汤科技开发有限公司 | Neural network searching method, image processing device, neural network searching apparatus, image processing apparatus, and recording medium |
| CN110197258A (en) * | 2019-05-29 | 2019-09-03 | 北京市商汤科技开发有限公司 | Neural network searching method, image processing method and device, equipment and medium |
| CN111684472A (en) * | 2019-05-31 | 2020-09-18 | 深圳市大疆创新科技有限公司 | Method and apparatus for network structure search, computer storage medium, and computer program product |
| WO2020237687A1 (en) * | 2019-05-31 | 2020-12-03 | 深圳市大疆创新科技有限公司 | Network architecture search method and apparatus, computer storage medium and computer program product |
| WO2020237688A1 (en) * | 2019-05-31 | 2020-12-03 | 深圳市大疆创新科技有限公司 | Method and device for searching network structure, computer storage medium and computer program product |
| CN111684471A (en) * | 2019-05-31 | 2020-09-18 | 深圳市大疆创新科技有限公司 | Method and apparatus for network structure search, computer storage medium, and computer program product |
| CN112052865A (en) * | 2019-06-05 | 2020-12-08 | 北京京东尚科信息技术有限公司 | Method and apparatus for generating neural network model |
| WO2021008206A1 (en) * | 2019-07-12 | 2021-01-21 | 华为技术有限公司 | Neural architecture search method, and image processing method and device |
| CN112215332B (en) * | 2019-07-12 | 2024-05-14 | 华为技术有限公司 | Searching method, image processing method and device for neural network structure |
| US12026938B2 (en) | 2019-07-12 | 2024-07-02 | Huawei Technologies Co., Ltd. | Neural architecture search method and image processing method and apparatus |
| CN112308200B (en) * | 2019-07-30 | 2024-04-26 | 华为技术有限公司 | Searching method and device for neural network |
| CN110473195B (en) * | 2019-08-13 | 2023-04-18 | 中山大学 | Medical focus detection framework and method capable of being customized automatically |
| CN110473195A (en) * | 2019-08-13 | 2019-11-19 | 中山大学 | It is a kind of can automatic customization medicine lesion detection framework and method |
| CN110490323A (en) * | 2019-08-20 | 2019-11-22 | 腾讯科技（深圳）有限公司 | Network model compression method, device, storage medium and computer equipment |
| CN110428046A (en) * | 2019-08-28 | 2019-11-08 | 腾讯科技（深圳）有限公司 | Acquisition methods and device, the storage medium of neural network structure |
| CN110428046B (en) * | 2019-08-28 | 2023-12-15 | 腾讯科技（深圳）有限公司 | Method and device for acquiring neural network structure and storage medium |
| CN112464930A (en) * | 2019-09-09 | 2021-03-09 | 华为技术有限公司 | Target detection network construction method, target detection method, device and storage medium |
| WO2021057690A1 (en) * | 2019-09-24 | 2021-04-01 | 华为技术有限公司 | Neural network building method and device, and image processing method and device |
| CN110751267B (en) * | 2019-09-30 | 2021-03-30 | 京东城市(北京)数字科技有限公司 | Neural network structure searching method, training method, device and storage medium |
| CN110751267A (en) * | 2019-09-30 | 2020-02-04 | 京东城市(北京)数字科技有限公司 | Neural network structure searching method, training method, device and storage medium |
| CN110826696B (en) * | 2019-10-30 | 2023-06-27 | 北京百度网讯科技有限公司 | Super-network search space construction method and device and electronic equipment |
| CN110826696A (en) * | 2019-10-30 | 2020-02-21 | 北京百度网讯科技有限公司 | Search space construction method and device of hyper network and electronic equipment |
| CN111160515A (en) * | 2019-12-09 | 2020-05-15 | 中山大学 | Running time prediction method, model search method and system |
| CN111160515B (en) * | 2019-12-09 | 2023-03-21 | 中山大学 | Running time prediction method, model search method and system |
| CN111191785A (en) * | 2019-12-20 | 2020-05-22 | 沈阳雅译网络技术有限公司 | Structure searching method based on expanded search space |
| CN111090673B (en) * | 2019-12-20 | 2023-04-18 | 北京百度网讯科技有限公司 | Cache unit searching method and related equipment |
| CN111090673A (en) * | 2019-12-20 | 2020-05-01 | 北京百度网讯科技有限公司 | Cache unit searching method and related equipment |
| CN111401516A (en) * | 2020-02-21 | 2020-07-10 | 华为技术有限公司 | Neural network channel parameter searching method and related equipment |
| CN111401516B (en) * | 2020-02-21 | 2024-04-26 | 华为云计算技术有限公司 | Searching method for neural network channel parameters and related equipment |
| CN111667057A (en) * | 2020-06-05 | 2020-09-15 | 北京百度网讯科技有限公司 | Method and apparatus for searching model structure |
| CN111667057B (en) * | 2020-06-05 | 2023-10-20 | 北京百度网讯科技有限公司 | Method and apparatus for searching model structures |
| WO2021253671A1 (en) * | 2020-06-18 | 2021-12-23 | 中国科学院深圳先进技术研究院 | Magnetic resonance cine imaging method and apparatus, and imaging device and storage medium |
| WO2021253938A1 (en) * | 2020-06-19 | 2021-12-23 | 深圳市商汤科技有限公司 | Neural network training method and apparatus, and video recognition method and apparatus |
| CN111950710A (en) * | 2020-08-12 | 2020-11-17 | 深圳市商汤科技有限公司 | Neural network optimization method and device, electronic equipment and storage medium |
| CN113379034B (en) * | 2021-06-15 | 2023-10-20 | 南京大学 | Neural network structure optimization method based on network structure search technology |
| CN113379034A (en) * | 2021-06-15 | 2021-09-10 | 南京大学 | Neural network structure optimization method based on network structure search technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110717586A (en) | 2020-01-21 |
| WO2020082663A1 (en) | 2020-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109284820A (en) | A kind of search structure method and device of deep neural network | |
| CN108122032B (en) | Neural network model training method, device, chip and system | |
| CN109347697B (en) | Opportunistic network link prediction method, apparatus and readable storage medium storing program for executing | |
| Prusinkiewicz et al. | Animation of plant development | |
| Xiao et al. | Fast deep learning training through intelligently freezing layers | |
| CN106650725A (en) | Full convolutional neural network-based candidate text box generation and text detection method | |
| CN110717627B (en) | Full traffic prediction method based on dual graph framework | |
| CN109598332A (en) | Neural network generation method and device, electronic equipment and storage medium | |
| CN107247991A (en) | A kind of method and device for building neutral net | |
| CN109962688A (en) | The quick predict and reverse geometry design method of all dielectric Meta Materials filter transfer characteristic based on deep learning neural network | |
| CN114418085B (en) | Personalized collaborative learning method and device based on neural network model pruning | |
| CN106022531A (en) | Searching method of shortest path passing by necessary peak points | |
| CN111737535A (en) | Network characterization learning method based on element structure and graph neural network | |
| CN107169031A (en) | It is a kind of to recommend method based on the picture materials that depthmeter reaches | |
| CN109409261A (en) | A kind of Crop classification method and system | |
| CN108923983A (en) | Prediction technique, device and the readable storage medium storing program for executing of opportunistic network link | |
| Addanki et al. | Placeto: Efficient progressive device placement optimization | |
| CN114172820A (en) | Cross-domain SFC dynamic deployment method, device, computer equipment and storage medium | |
| CN111737826B (en) | Rail transit automatic simulation modeling method and device based on reinforcement learning | |
| Xing et al. | Solve traveling salesman problem by Monte Carlo tree search and deep neural network | |
| CN107945534A (en) | A kind of special bus method for predicting based on GMDH neutral nets | |
| CN113065443A (en) | Training method, recognition method, system, device and medium of image recognition model | |
| Gupta et al. | Differential evolution-driven traffic light scheduling for vehicle-pedestrian mixed-flow networks | |
| Wei et al. | Multi-agent deep reinforcement learning for traffic signal control with Nash Equilibrium | |
| Zhang et al. | Solving multi-class traffic assignment problem with genetic algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190129 |