CN110969211A - Automatic classification method of convolutional neural network constructed based on incremental branch growth - Google Patents

Automatic classification method of convolutional neural network constructed based on incremental branch growth Download PDF

Info

Publication number
CN110969211A
CN110969211A CN201911214838.XA CN201911214838A CN110969211A CN 110969211 A CN110969211 A CN 110969211A CN 201911214838 A CN201911214838 A CN 201911214838A CN 110969211 A CN110969211 A CN 110969211A
Authority
CN
China
Prior art keywords
layer
branch
incremental
neural network
convolutional
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911214838.XA
Other languages
Chinese (zh)
Inventor
黄琳
杨铁军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guilin University of Technology
Original Assignee
Guilin University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guilin University of Technology filed Critical Guilin University of Technology
Priority to CN201911214838.XA priority Critical patent/CN110969211A/en
Publication of CN110969211A publication Critical patent/CN110969211A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses an automatic classification method of a convolutional neural network constructed based on incremental branch growth, which comprises the following steps of S1: building and initializing convolutional neural network model Gm(ii) a S2: for convolution neural network model GmTraining is carried out to obtain a model G 'after training'm(ii) a S3: for convolution neural network model GmCarrying out next generation network growth to obtain a next generation convolution neural network model Gm+1(ii) a S4: for convolution neural network model Gm+1Training is carried out to obtain a model G 'after training'm+1(ii) a S5: if model G'mAnd model G'm+1Is less than a preset threshold value, then model G 'is used'm+1Finishing a classification task; if model G'mAnd model G'm+1Classification test ofWhen the difference in accuracy is not less than the preset threshold value, m ← m +1, and the process returns to step S3. The invention can avoid the fussy parameter adjusting process in the process of building the neural network and automatically build the deep convolution neural network model suitable for the specific classification task more efficiently.

Description

Automatic classification method of convolutional neural network constructed based on incremental branch growth
Technical Field
The invention relates to the field of neural networks, in particular to an automatic classification method of a convolutional neural network constructed based on incremental branch growth.
Background
With the development of artificial intelligence and computer technology, the application of convolutional neural networks in reality is more and more common. The task of utilizing the convolutional neural network to perform automatic classification is one application of the convolutional neural network, in practical application, the convolutional neural network needs to be learned, and target features in the same classification are identified by manually adjusting parameters of the neural network, but the process needs a complicated parameter adjusting process, so that the efficiency of the task is greatly reduced.
Disclosure of Invention
The invention provides an automatic classification method of a convolutional neural network constructed based on incremental branch growth, which can more efficiently and automatically construct a deep convolutional neural network model suitable for a specific classification task to finish the classification task.
In order to solve the technical problems, the technical scheme of the invention is as follows:
an automatic classification method of a convolutional neural network constructed based on incremental branch growth comprises the following steps:
s1: building and initializing convolutional neural network model Gm；
S2: for convolution neural network model GmTraining is carried out to obtain a model G 'after training'm；
S3: for convolution neural network model GmCarrying out next generation network growth to obtain a next generation convolution neural network model Gm+1；
S4: for convolution neural network model Gm+1Training is carried out to obtain a model G 'after training'm+1；
S5: if model G'mAnd model G'm+1Is less than a preset threshold value, then model G 'is used'm+1Finishing a classification task; if model G'mAnd model G'm+1M ← m +1, making model G not less than preset threshold valuem+1Replacement model GmThe flow returns to step S3.
Preferably, the neural network model G is convolved in step S1mComprises an incremental branch generation layer (IBG) including nAnd each sublayer comprises a plurality of Increment Branches (IB) and convolution layers, the last sublayer of the increment branch generation layer has only one increment branch, the increment branches are output to a Conv + GAP layer of the convolution neural network model, and finally, the GAP layer outputs a classification result.
Preferably, the incremental branch generation layer includes n sublayers, where n is an integer not less than 3.
Preferably, each incremental branch exists independently or accesses other convolutional layers, and the incremental branch accessing the convolutional layer and the convolutional layer belong to the same network layer.
Preferably, the incremental branch comprises a convolutional layer, a branch access layer, a pooling layer and a partial response normalization layer, wherein the input of the convolutional layer is the input of the incremental branch, and is the input of the data layer or the output of other incremental branches, the output of the convolutional layer is the input of the branch access layer, the output of the branch access layer is the input of the pooling layer, the output of the pooling layer is the input of the partial response normalization layer, and the output of the partial response normalization layer is the output of the incremental branch.
Preferably, the branch access layer may further be accessed with other convolutional layers, when there is no other convolutional layer, the output of the branch convolutional layer is equal to the input, when there is another convolutional layer, the branch access layer performs concat operation on the output of the convolutional layer in the same incremental branch and the output of the accessed other convolutional layer, and then performs two-way input fusion and output dimension normalization through one 1 × 1 convolutional layer.
Preferably, convolutional layers in the same incremental branch of the branch access layer are consistent with convolutional kernel parameters of other convolutional layers accessed.
Preferably, the convolutional neural network model G is applied in step S3mCarrying out next generation network growth, specifically comprising the following steps:
s3.1: IBG from the first sublayer of the growing layer of incremental branches1Beginning to last sub-layer IBGnSearching for an accessible incremental branch;
S3.2：if in the sub-layer IBGiFinding an accessible incremental branch, the IBG is in the sub-layeriThe method is characterized in that a convolution layer is additionally arranged in the method, and specifically comprises the following steps:
if i>1, in the sub-layer IBGi-1Adding an increment branch, the output of which is sub-layer IBGiIn the newly added convolutional layer or the input of the delta branch, if the sub-layer IBGi-1Is the first sublayer, a convolution layer is newly added and the newly added incremental branch is accessed, and the step S3.3 is entered;
if until the last sub-layer IBGnIf the accessible incremental branch can not be found, a sub-layer IBG is addedn+1The method specifically comprises the following steps:
in sub-layer IBGn+1Adding an increment branch with the input of a sub-layer IBGnAnd outputting the output of the unique increment branch to a Conv + GAP layer, n ← n +1, and returning to the step S3.1;
s3.3: obtaining a next generation convolutional neural network model Gm+1。
Preferably, the classification test precision in step S5 is calculated by using the average precision ratio mean value.
Compared with the prior art, the technical scheme of the invention has the beneficial effects that:
the invention firstly establishes an initial network based on the incremental branch and trains the initial network, and then leads the network to grow continuously through the iterative process of branch access-incremental training. The training of the next generation network in the growth process is completed based on the training result of the previous generation network, and the searching process of the neural network model is greatly accelerated. By the method, a complex parameter adjusting process in the process of constructing the neural network can be avoided, and the deep convolutional neural network model suitable for the specific classification task can be automatically constructed more efficiently.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention.
Fig. 2 is a schematic diagram of an incremental branching structure of the present invention.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent;
for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product;
it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The technical solution of the present invention is further described below with reference to the accompanying drawings and examples.
Example 1
The embodiment provides an automatic classification method of a convolutional neural network constructed based on incremental branch growth, as shown in fig. 1, including the following steps:
s1: building and initializing convolutional neural network model Gm；
S2: for convolution neural network model GmTraining is carried out to obtain a model G 'after training'm；
S3: for convolution neural network model GmCarrying out next generation network growth to obtain a next generation convolution neural network model Gm+1；
S4: for convolution neural network model Gm+1Training is carried out to obtain a model G 'after training'm+1；
S5: if model G'mAnd model G'm+1Is less than a preset threshold value, then model G 'is used'm+1Finishing a classification task; if model G'mAnd model G'm+1Is not less than the preset threshold value, m ← m +1, and the process returns to step S3.
Convolution neural network model G in step S1mThe method comprises an incremental branch generation layer IBG, wherein the incremental branch generation layer IBG comprises n sublayers, each sublayer comprises a plurality of incremental branches and convolution layers, the last sublayer of the incremental branch generation layer comprises one and only one incremental branch, the incremental branch is output to a Conv + GAP layer of a convolutional neural network model, and finally, a classification result is output by the GAP layer.
The incremental branch generation layer includes n sublayers, where n is an integer not less than 3.
Each increment branch independently exists or is accessed into other convolution layers, and the increment branch accessed into the convolution layers and the convolution layers belong to the same network layer.
The incremental branch is shown in fig. 2 and comprises a convolutional layer, a branch access layer, a pooling layer and a partial response normalization layer, wherein the input of the convolutional layer is the input of the incremental branch, the input of the convolutional layer is the input of the data layer or the output of other incremental branches, the output of the convolutional layer is the input of the branch access layer, the output of the branch access layer is the input of the pooling layer, the output of the pooling layer is the input of the partial response normalization layer, and the output of the partial response normalization layer is the output of the incremental branch.
The branch access layer can be accessed with other convolution layers, when no other convolution layer is accessed, the output of the branch convolution layer is equal to the input, when other convolution layers are accessed, the branch access layer carries out concat operation on the output of the convolution layer in the same incremental branch and the output of the accessed other convolution layers, and then carries out two-way input fusion and output dimension normalization through one convolution layer of 1 x 1.
Convolution layers in the same incremental branch of the branch access layer have the same convolution kernel parameters with other accessed convolution layers.
Convolution neural network model G in step S3mCarrying out next generation network growth, specifically comprising the following steps:
s3.1: IBG from the first sublayer of the growing layer of incremental branches1Beginning to last sub-layer IBGnSearching for an accessible incremental branch;
s3.2: if in the sub-layer IBGiFinding an accessible incremental branch, the IBG is in the sub-layeriThe method is characterized in that a convolution layer is additionally arranged in the method, and specifically comprises the following steps:
if i>1, in the sub-layer IBGi-1Adding an increment branch, the output of which is sub-layer IBGiIn the newly added convolutional layer or the input of the delta branch, if the sub-layer IBGi-1Is the first sublayer, adds a convolutional layer and accesses the added incremental branch, and entersStep S3.3;
if until the last sub-layer IBGnIf the accessible incremental branch can not be found, a sub-layer IBG is addedn+1The method specifically comprises the following steps:
in sub-layer IBGn+1Adding an increment branch with the input of a sub-layer IBGnAnd outputting the output of the unique increment branch to a Conv + GAP layer, n ← n +1, and returning to the step S3.1;
s3.3: obtaining a next generation convolutional neural network model Gm+1。
In step S5, the classification test precision is calculated by using the average precision ratio mean value.
The same or similar reference numerals correspond to the same or similar parts;
the terms describing positional relationships in the drawings are for illustrative purposes only and are not to be construed as limiting the patent;
it should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.

Claims (9)

1. An automatic classification method of a convolutional neural network constructed based on incremental branch growth is characterized by comprising the following steps:
s1: building and initializing convolutional neural network model Gm；
S2: for convolution neural network model GmTraining is carried out to obtain a model G 'after training'm；
S3: for convolution neural network model GmCarrying out next generation network growth to obtain a next generation convolution neural network model Gm+1；
S4: for convolution neural network model Gm+1Training is carried out to obtain a model G 'after training'm+1；
S5: if model G'mAnd model G'm+1Is less than a preset threshold value, then model G 'is used'm+1Finishing a classification task; if model G'mAnd model G'm+1Is not less than the preset threshold value, m ← m +1, and the process returns to step S3.
2. The method for automatically classifying convolutional neural networks based on incremental branch growth as claimed in claim 1, wherein the convolutional neural network model G in step S1mThe method comprises an incremental branch generation layer IBG, wherein the incremental branch generation layer IBG comprises n sublayers, each sublayer comprises a plurality of incremental branches and convolution layers, the last sublayer of the incremental branch generation layer comprises one and only one incremental branch, the incremental branch is output to a Conv + GAP layer of a convolutional neural network model, and finally, a classification result is output by the GAP layer.
3. The method of claim 2, wherein the incremental branch generation layer comprises n sublayers, wherein n is an integer no less than 3.
4. The method of claim 3, wherein each incremental branch independently exists or is connected to other convolutional layers, and the incremental branch connected to a convolutional layer and the convolutional layer belong to the same network layer.
5. The method according to any one of claims 2 to 4, wherein the incremental branch comprises a convolutional layer, a branch access layer, a pooling layer and a partial response normalization layer, wherein the convolutional layer has an input of the incremental branch, an input of the data layer or an output of other incremental branches, an output of the convolutional layer is an input of the branch access layer, an output of the branch access layer is an input of the pooling layer, an output of the pooling layer is an input of the partial response normalization layer, and an output of the partial response normalization layer is an output of the incremental branch.
6. The method according to claim 5, wherein the branch access layer is further accessible to other convolutional layers, when there is no other convolutional layer, the output of the branch convolutional layer is equal to the input, when there is other convolutional layer, the branch access layer performs concat operation on the output of the convolutional layer in the same incremental branch and the output of the other convolutional layer, and then performs two-way input fusion and output dimension normalization through one convolutional layer of 1 x 1.
7. The method of claim 6, wherein convolutional layers in the same incremental branch of the branch access layer are consistent with convolutional kernel parameters of other convolutional layers accessed.
8. The method for automatically classifying convolutional neural network constructed based on incremental branch growth as claimed in claim 7, wherein step S3 is performed on convolutional neural network model GmCarrying out next generation network growth, specifically comprising the following steps:
s3.1: IBG from the first sublayer of the growing layer of incremental branches1Beginning to last sub-layer IBGnSearching for an accessible incremental branch;
s3.2: if in the sub-layer IBGiFinding an accessible incremental branch, the IBG is in the sub-layeriThe method is characterized in that a convolution layer is additionally arranged in the method, and specifically comprises the following steps:
if i>1, in the sub-layer IBGi-1Adding an increment branch, the output of which is sub-layer IBGiThe input of the incremental branch or convolutional layer, if any, inLayer IBGi-1Is the first sublayer, a convolution layer is newly added and the newly added incremental branch is accessed, and the step S3.3 is entered;
if until the last sub-layer IBGnIf the accessible incremental branch can not be found, a sub-layer IBG is addedn+1The method specifically comprises the following steps:
in sub-layer IBGn+1Adding an increment branch with the input of a sub-layer IBGnAnd outputting the output of the unique increment branch to a Conv + GAP layer, n ← n +1, and returning to the step S3.1;
s3.3: obtaining a next generation convolutional neural network model Gm+1。
9. The method for automatically classifying the convolutional neural network constructed based on incremental branch growth as claimed in claim 8, wherein the classification test precision in step S5 is calculated by using an average precision rate mean.
CN201911214838.XA 2019-12-02 2019-12-02 Automatic classification method of convolutional neural network constructed based on incremental branch growth Pending CN110969211A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911214838.XA CN110969211A (en) 2019-12-02 2019-12-02 Automatic classification method of convolutional neural network constructed based on incremental branch growth

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911214838.XA CN110969211A (en) 2019-12-02 2019-12-02 Automatic classification method of convolutional neural network constructed based on incremental branch growth

Publications (1)

Publication Number Publication Date
CN110969211A true CN110969211A (en) 2020-04-07

Family

ID=70032594

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911214838.XA Pending CN110969211A (en) 2019-12-02 2019-12-02 Automatic classification method of convolutional neural network constructed based on incremental branch growth

Country Status (1)

Country Link
CN (1) CN110969211A (en)

Similar Documents

Publication Publication Date Title
US11625614B2 (en) Small-world nets for fast neural network training and execution
CN111243045B (en) Image generation method based on Gaussian mixture model prior variation self-encoder
CN109948029A (en) Based on the adaptive depth hashing image searching method of neural network
CN113761221B (en) Knowledge graph entity alignment method based on graph neural network
CN110705029A (en) Flow field prediction method of oscillating flapping wing energy acquisition system based on transfer learning
CN114283320B (en) Branch-free structure target detection method based on full convolution
CN115437795B (en) Video memory recalculation optimization method and system for heterogeneous GPU cluster load perception
CN114372438A (en) Chip macro-unit layout method and system based on lightweight deep reinforcement learning
CN105550368A (en) Approximate nearest neighbor searching method and system of high dimensional data
CN109754122A (en) A kind of Numerical Predicting Method of the BP neural network based on random forest feature extraction
CN114943345A (en) Federal learning global model training method based on active learning and model compression
CN110689113A (en) Deep neural network compression method based on brain consensus initiative
CN111488981A (en) Method for selecting sparse threshold of depth network parameter based on Gaussian distribution estimation
CN116450954A (en) Collaborative filtering recommendation method based on graph convolution network
CN115983105B (en) Occam inversion Lagrange multiplier optimization method based on deep learning weighted decision
Wang et al. [Retracted] Architectural and Landscape Garden Planning Integrated with Artificial Intelligence Parametric Analysis
CN114399018B (en) Efficient ientNet ceramic fragment classification method based on sparrow optimization of rotary control strategy
CN109034372B (en) Neural network pruning method based on probability
CN110969211A (en) Automatic classification method of convolutional neural network constructed based on incremental branch growth
CN110321799B (en) Scene number selection method based on SBR and average inter-class distance
CN112766687A (en) Airplane general assembly material distribution priority distribution method based on deep CNN
CN111651683A (en) Parameter-adjustable scale-free network generation method
CN117151998A (en) Image illumination correction method based on support vector regression
CN113011589B (en) Co-evolution-based hyperspectral image band selection method and system
CN113722951B (en) Scatterer three-dimensional finite element grid optimization method based on neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination