CN108764317B - Residual convolutional neural network image classification method based on multipath feature weighting - Google Patents
Residual convolutional neural network image classification method based on multipath feature weighting Download PDFInfo
- Publication number
- CN108764317B CN108764317B CN201810485738.XA CN201810485738A CN108764317B CN 108764317 B CN108764317 B CN 108764317B CN 201810485738 A CN201810485738 A CN 201810485738A CN 108764317 B CN108764317 B CN 108764317B
- Authority
- CN
- China
- Prior art keywords
- image
- convolution
- size
- feature
- residual error
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
A residual convolutional neural network image classification method based on multipath feature weighting comprises the following steps: 1) the input image of the model is a preprocessed original image, and the preprocessed image is cut into a fixed size; 2) performing convolution operation and pooling operation of larger size on the image; 3) sending the features output in the step 2) to a first multipath feature weighted residual error module; 4) continuously sending the output of the multi-path characteristic weighted residual error module in the step 3) to a next multi-path characteristic weighted residual error module, gradually reducing the size of the output characteristic image after passing through the multi-path characteristic weighted residual error modules until the size of the output characteristic image is changed into a smaller size, and finally reducing the size of the output characteristic image into a characteristic point after passing through an average pooling layer; the obtained feature points are directly sent to a classification layer for classification or classified after passing through a full connection layer. The method is applied to a complex image classification task, enriches the feature expression and avoids the problem of gradient disappearance caused by the increase of the depth of the neural network.
Description
Technical Field
The invention relates to the field of computer vision, in particular to an image classification method, which is one of deep learning technologies and is mainly used for training a deep image classification model.
Background
In recent years, with the exponential growth of computer computing power and the emergence of new neural network architectures, deep learning techniques have begun to be highlighted in the fields of computer vision, speech recognition, natural language processing, and the like. In the field of computer vision, the performance of a computer in image segmentation and image recognition tasks is greatly improved due to the occurrence of a convolutional neural network, and the recognition accuracy is far higher than that of a traditional machine learning algorithm. At present, image recognition technology based on convolutional neural networks has been widely used.
The conventional convolutional neural network comprises a convolutional layer, a pooling layer, a full-link layer and a classification layer, and the basic structure of the convolutional neural network is a series combination of the units. Convolutional layers are used to learn low-level and high-level features in an image, and pooling layers can extract these features and continually reduce the size of the feature map. And the full connection layer and the classification layer are positioned at the tail of the whole neural network and are used for classifying the finally extracted high-level features. In a specific image classification task, the recognized objects in the image have more feature types and more complex feature structures. To cope with this situation, the conventional convolutional neural network needs to be boosted in both width and depth. First, a single convolutional layer often cannot learn all valid features, so the existing network structure solves the problem by increasing the network width, that is, increasing the number of output features of a certain layer in the network. However, the increase of the number of channels may cause the network to learn repeated or useless features, so that the model is redundant, the classification is not facilitated, and the useless calculation amount is increased. In addition, in order to make the neural network learn more complex features, the depth of the network is designed to be deeper. However, as the depth of the network increases, the problems of gradient disappearance and gradient explosion occur more easily in the network training process, so that the learning process cannot be converged. To address this problem, the research team at microsoft asian institute proposed a convolutional neural network based on a residual structure, the basic idea of which is to introduce a fast connection channel between low-level and high-level features. According to the structure, on one hand, low-level features are introduced into high-level features, the diversity of feature extraction is increased, and on the other hand, the problem of gradient disappearance in the backward propagation process is avoided due to the additionally-added forward channel.
The two schemes respectively optimize the conventional convolutional neural network in the aspects of network width and depth. But to better address the complexities of the image classification problem, convolutional neural network models need to learn more complex feature expressions. Meanwhile, the redundancy problem caused by the increase of the network width still needs to be solved.
Disclosure of Invention
In order to overcome the defect of poor performance when the existing image classification method is applied to complex images, the invention provides a residual convolutional neural network image classification method based on multipath feature weighting. The method solves the fusion of the neural network width and depth method. On the basis of increasing the width of the neural network, the multi-path features are fused in a weighting mode, so that the expression of key features is enhanced, redundant features are removed, and the problem of model redundancy caused by output of repeated and useless features is solved. Meanwhile, the network structure also integrates a residual error structure for solving the depth problem. The addition of the residual structure increases more forward channels, further enriches the expression of features and avoids the gradient problem.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a residual convolutional neural network image classification method based on multipath feature weighting, comprising the following steps:
1) firstly, an input image of a model is a preprocessed original image, and the preprocessed image is cut into a fixed size;
2) performing convolution operation and pooling operation of larger size on the image;
3) sending the features output in step 2) to a first multipath feature weighted residual error module, wherein the multipath feature weighted residual error module is composed of a plurality of convolution groups and a weighted combination module: each convolution group comprises one or more convolution layers, and the configuration of the convolution layers among the convolution groups is different so as to ensure the diversity of the sizes of convolution kernels; meanwhile, when the feature images in the network pass through certain multi-path feature weighting residual error modules, the sizes of the feature images are correspondingly reduced, so that in order to ensure that the sizes of the output feature images of each convolution group are kept consistent, some convolution groups also contain pooling layers;
the outputs of all the convolution groups are sent to a weighting combination module for integration, and the specific weighting combination method is to multiply the output characteristic of each convolution group by a different parameter k initialized randomly and output the weighted convolution groups in a characteristic channel for splicing; the spliced combined features are added with the input of the multi-path feature weighted residual error module to be used as the output of the whole multi-path feature weighted residual error module;
4) and continuously sending the output of the multi-path characteristic weighted residual error module in the step 3) to a next multi-path characteristic weighted residual error module. After passing through a plurality of multipath feature weighted residual error modules, the size of the output feature image is gradually reduced until the size of the output feature image is reduced to a smaller size within 10 multiplied by 10, and finally the output feature image is reduced to feature points through an average pooling layer; the obtained feature points are directly sent to a classification layer for classification or classified after passing through a full connection layer.
Further, the method comprises the following steps:
5) in the initial stage of model training, the weighting coefficient k and the coefficient in the convolution kernel are initialized randomly in various modes, and finally, the network parameters for classification are continuously optimized in the back propagation process until an optimal value is obtained;
still further, in step 2), the convolution kernel size may be set to 7 × 7 with a step size of 2 and maximum pooling with a size of 3 × 3 with a step size of 2; and modifying the size according to the input image cut in the step 1). The goal of choosing a larger convolution kernel size is to extract the underlying features with a larger field of view and to prevent loss of detail features as much as possible. Meanwhile, the size of the characteristic image can be further reduced by selecting a larger step length in the convolution and pooling processes so as to reduce the calculation amount;
further, in the step 3), the size of the convolution kernel in the convolution group is selected in a manner that: one is to select as many combinations of convolution kernel sizes as computing resources allow to cover all possible feature types; the other scheme is that according to the image characteristics needing to be classified, more favorable convolution kernel size parameters are designed manually; a combination of the two approaches may also be used.
The invention discloses a residual convolutional neural network image classification method based on multipath feature weighting, which is mainly characterized by fusing and improving a width lifting scheme and a depth lifting scheme in a convolutional neural network. The basic structure of the system comprises a convolution layer, a pooling layer, a full-link layer and a multi-path characteristic weighting residual error module. The specific structure of the method is that a convolution layer, a pooling layer and a plurality of multipath feature weighted residual error modules are used in series, and finally a final classification result is obtained through a full connection layer and a classification layer. The main role of the convolutional layer and the pooling layer is to extract low-level features while reducing the size of the feature image. The core of the network is a multipath feature weighted residual error module, which comprises a plurality of convolution groups connected in parallel, wherein each convolution group comprises one or more convolution layers. The number of convolution layers in a convolution group should be adjusted according to the total number of neural network layers to be learned, and the size of convolution kernel should be selected to ensure that the sizes of convolution kernels in each convolution group are different from each other. The outputs of all convolution groups are sent to weighted combination processing, and the specific processing scheme is to multiply the output of each convolution block by a different learnable parameter k respectively and then splice the weighted outputs in a feature channel. By assigning a learnable weight to the output of each convolution group, the network can enhance the expression of important features and remove redundant or duplicated features, thereby avoiding model redundancy.
The invention has the following beneficial effects: the convolutional neural network can be promoted in width and depth at the same time, so that a better characteristic weight-improving effect is obtained, and the performance of an image classification task when the convolutional neural network is applied to a complex image is improved.
Drawings
Fig. 1 is a general flow chart of a residual convolutional neural network based on multipath feature weighting.
Fig. 2 is a diagram of an internal structure of a multi-path feature weighted residual error module.
Detailed Description
The present invention is further described below with reference to the flow chart.
Referring to fig. 1 and 2, a residual convolutional neural network image classification method based on multipath feature weighting includes the following steps:
1) firstly, the input image of the model is a preprocessed original image, the preprocessed image must be completely cut to a fixed size, and in order to facilitate the model training, the length and width of the fixed size are preferably kept consistent, and the specific size is determined by the specific application of the model and the size of the model. Common input image sizes are: 512. 299, 224, etc.
2) The image is subjected to convolution and pooling operations of larger size, e.g. convolution kernel size 7 x 7 with step size 2 and maximal pooling with size 3 x 3 with step size 2. The significance of choosing a larger convolution kernel size is to extract the underlying features with a larger field of view and to prevent loss of detail features as much as possible. Meanwhile, the size of the characteristic image can be further reduced by selecting a larger step size in the convolution and pooling processes.
3) Sending the features output in the step 2) to a first multipath feature weighted residual error module. The detailed structure of the multipath feature weighted residual module is shown in fig. 2: the module contains a plurality of convolution groups, each convolution group contains one or more convolution layers, and the size parameters of the convolution layers in each convolution group should be different from each other so as to ensure the diversity of feature extraction. For example: the convolution group 1 is configured as one convolution layer of 7 × 1 plus 1 × 7, the convolution group 2 is configured as a convolution layer of 3 × 3 plus 1 × 1, and the convolution group 3 is configured as a convolution of 5 × 5 plus 1 × 1. The specific size design can select more size combinations under the permission of computing resources, and can also manually design the size of the convolution kernel according to a specific classification task. In addition, the image size should be reduced as the feature image passes through some multi-way feature weighted residual modules, so some convolution groups also contain pooling layers for compressing the feature image size.
4) The outputs of all convolution groups in step 3) are sent to a weighted combination module for feature fusion. The specific implementation of the weighted combination module in fig. 2 is as follows: defining a random initialized coefficient value k for the output of each convolution group, multiplying the output of each convolution group by the respective coefficient value, and splicing the weighted features in the feature channel;
and adding the features fused in the step 4) and the input of the multi-path feature weighted residual error module in the step 3) to obtain the output of the whole multi-path feature weighted residual error module.
And continuously sending the output of the multi-path characteristic weighted residual error module in the step 3) to a next multi-path characteristic weighted residual error module. After passing through a plurality of multipath feature weighted residual error modules, the feature image size is continuously reduced to a smaller size within 10 multiplied by 10, and is reduced to feature points through an average pooling layer; the obtained feature points can be directly sent to a classification layer for classification or classified after passing through a full connection layer.
5) In the initial stage of model training, the weighting parameter k and the convolution kernel parameter can be initialized randomly in various ways. Finally the parameters for classification will be continuously optimized in the back propagation process until the optimal values are obtained.
Claims (4)
1. A residual convolutional neural network image classification method based on multipath feature weighting is characterized by comprising the following steps:
1) firstly, an input image of a model is a preprocessed original image, and the preprocessed image is cut into a fixed size;
2) performing large-size convolution operation and pooling operation on the image;
3) sending the features output in step 2) to a first multipath feature weighted residual error module, wherein the multipath feature weighted residual error module is composed of a plurality of convolution groups and a weighted combination module: each convolution group comprises one or more convolution layers, and the configuration of the convolution layers among the convolution groups is different so as to ensure the diversity of the sizes of convolution kernels; meanwhile, when the feature images in the network pass through the multipath feature weighting residual error module, the sizes of the feature images are correspondingly reduced, so that in order to ensure that the sizes of the output feature images of each convolution group are kept consistent, the convolution groups also contain a pooling layer;
the outputs of all the convolution groups are sent to a weighting combination module for integration, and the specific weighting combination method is to multiply the output characteristic of each convolution group by a different parameter k initialized randomly and output the weighted convolution groups in a characteristic channel for splicing; the spliced combined features are added with the input of the multi-path feature weighted residual error module to be used as the output of the whole multi-path feature weighted residual error module;
4) continuously sending the output of the multi-path characteristic weighted residual error module in the step 3) to a next multi-path characteristic weighted residual error module, gradually reducing the size of the output characteristic image after passing through the multi-path characteristic weighted residual error modules until the size of the output characteristic image is changed into a small size within 10 multiplied by 10, and finally reducing the size of the output characteristic image into a characteristic point through an average pooling layer; the obtained feature points are directly sent to a classification layer for classification or classified after passing through a full connection layer.
2. The method for classifying images based on multipath feature weighting residual convolutional neural network as claimed in claim 1, wherein the method further comprises the steps of:
5) in the initial stage of model training, the weighting coefficient k and the coefficient in the convolution kernel are initialized randomly in various modes, and finally, the network parameters for classification are continuously optimized in the back propagation process until the optimal values are obtained.
3. The method for classifying the image of the residual convolutional neural network based on the multipath feature weighting as claimed in claim 1 or 2, wherein in the step 2), the convolution kernel size is set to be 7 x 7, the convolution step size is 2, and the maximum pooling is 3 x 3 and the convolution step size is 2; and modifying the size according to the input image cut in the step 1).
4. The method for classifying images of residual convolutional neural network based on multipath feature weighting as claimed in claim 1 or 2, wherein in the step 3), the convolutional kernel size in the convolutional group is selected in a manner that: one is to select a combination of convolution kernel sizes to cover all feature types, if computational resources allow; the other scheme is that convolution kernel size parameters are designed manually according to the image features needing to be classified; the two modes are combined.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810485738.XA CN108764317B (en) | 2018-05-21 | 2018-05-21 | Residual convolutional neural network image classification method based on multipath feature weighting |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810485738.XA CN108764317B (en) | 2018-05-21 | 2018-05-21 | Residual convolutional neural network image classification method based on multipath feature weighting |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108764317A CN108764317A (en) | 2018-11-06 |
| CN108764317B true CN108764317B (en) | 2021-11-23 |
Family
ID=64007225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810485738.XA Active CN108764317B (en) | 2018-05-21 | 2018-05-21 | Residual convolutional neural network image classification method based on multipath feature weighting |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764317B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109376792B (en) * | 2018-11-07 | 2022-02-18 | 河北工业大学 | Photovoltaic cell appearance defect classification method based on multi-channel residual error neural network |
| CN109886331A (en) * | 2019-02-17 | 2019-06-14 | 太原理工大学 | A method of based on convolutional neural networks structure extraction characteristics of image |
| CN110149541B (en) | 2019-04-23 | 2021-08-03 | 腾讯科技（深圳）有限公司 | Video recommendation method and device, computer equipment and storage medium |
| WO2021022543A1 (en) * | 2019-08-08 | 2021-02-11 | Boe Technology Group Co., Ltd. | Computer-implemented method, computer-implemented diagnosis method, apparatus for classifying image, and computer-program product |
| CN112446266B (en) * | 2019-09-04 | 2024-03-29 | 北京君正集成电路股份有限公司 | Face recognition network structure suitable for front end |
| CN112614483A (en) * | 2019-09-18 | 2021-04-06 | 珠海格力电器股份有限公司 | Modeling method based on residual convolutional network, voice recognition method and electronic equipment |
| CN110796167B (en) * | 2019-09-25 | 2022-07-22 | 武汉大学 | Image classification method based on deep neural network of lifting scheme |
| CN112883981A (en) * | 2019-11-29 | 2021-06-01 | 阿里巴巴集团控股有限公司 | Image processing method, device and storage medium |
| CN111368937A (en) * | 2020-03-19 | 2020-07-03 | 京东方科技集团股份有限公司 | Image classification method and device, and training method, device, equipment and medium thereof |
| CN111444957B (en) * | 2020-03-25 | 2023-11-07 | 腾讯科技（深圳）有限公司 | Image data processing method, device, computer equipment and storage medium |
| CN111680763A (en) * | 2020-08-12 | 2020-09-18 | 城云科技（中国）有限公司 | Sub-model decorrelation-based article stacking detection method and detection device |
| CN112329534B (en) * | 2020-10-07 | 2023-07-18 | 南京理工大学 | Radar target identification method based on two-dimensional weighted residual convolution neural network |
| CN113162908B (en) * | 2021-03-04 | 2022-11-15 | 中国科学院信息工程研究所 | Encrypted flow detection method and system based on deep learning |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
| CN106295694A (en) * | 2016-08-05 | 2017-01-04 | 浙江工业大学 | A kind of face identification method of iteration weight set of constraints rarefaction representation classification |
| CN107437096A (en) * | 2017-07-28 | 2017-12-05 | 北京大学 | Image classification method based on the efficient depth residual error network model of parameter |
-
2018
- 2018-05-21 CN CN201810485738.XA patent/CN108764317B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204467A (en) * | 2016-06-27 | 2016-12-07 | 深圳市未来媒体技术研究院 | A kind of image de-noising method based on cascade residual error neutral net |
| CN106295694A (en) * | 2016-08-05 | 2017-01-04 | 浙江工业大学 | A kind of face identification method of iteration weight set of constraints rarefaction representation classification |
| CN107437096A (en) * | 2017-07-28 | 2017-12-05 | 北京大学 | Image classification method based on the efficient depth residual error network model of parameter |
Non-Patent Citations (1)
| Title |
|---|
| 基于空间约束加权条件稀疏表示高光谱图像分类;陈善学 等;《系统工程与电子技术》;20160228;第442-449页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108764317A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764317B (en) | Residual convolutional neural network image classification method based on multipath feature weighting | |
| CN108985317B (en) | Image classification method based on separable convolution and attention mechanism | |
| CN107609587B (en) | Multi-class multi-view data generation method for generating countermeasure network based on deep convolution | |
| CN108664993B (en) | Dense weight connection convolutional neural network image classification method | |
| CN107506822B (en) | Deep neural network method based on space fusion pooling | |
| CN110570346B (en) | Method for performing style migration on calligraphy based on cyclic generation countermeasure network | |
| CN110889416B (en) | Salient object detection method based on cascade improved network | |
| CN110135460B (en) | Image information enhancement method based on VLAD convolution module | |
| CN112489164B (en) | Image coloring method based on improved depth separable convolutional neural network | |
| CN107392919A (en) | Gray threshold acquisition methods, image partition method based on self-adapted genetic algorithm | |
| CN107292458A (en) | A kind of Forecasting Methodology and prediction meanss applied to neural network chip | |
| CN111224905B (en) | Multi-user detection method based on convolution residual error network in large-scale Internet of things | |
| CN110009700B (en) | Convolutional neural network visual depth estimation method based on RGB (red, green and blue) graph and gradient graph | |
| CN110298394A (en) | A kind of image-recognizing method and relevant apparatus | |
| CN116580184A (en) | YOLOv 7-based lightweight model | |
| CN111523483A (en) | Chinese food dish image identification method and device | |
| CN113919401A (en) | Modulation type identification method and device based on constellation diagram characteristics and computer equipment | |
| CN113420651A (en) | Lightweight method and system of deep convolutional neural network and target detection method | |
| CN110414586B (en) | Anti-counterfeit label counterfeit checking method, device, equipment and medium based on deep learning | |
| KR102393761B1 (en) | Method and system of learning artificial neural network model for image processing | |
| CN110188692B (en) | Enhanced cyclic cascading method for effective target rapid identification | |
| CN113239949A (en) | Data reconstruction method based on 1D packet convolutional neural network | |
| CN111461144A (en) | Method for accelerating convolutional neural network | |
| Wang et al. | Exploring quantization in few-shot learning | |
| CN111223120B (en) | Point cloud semantic segmentation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |