CN109034249A - Based on convolution optimization method, device, terminal device and the computer readable storage medium for decomposing radial symmetric convolution kernel - Google Patents
Based on convolution optimization method, device, terminal device and the computer readable storage medium for decomposing radial symmetric convolution kernel Download PDFInfo
- Publication number
- CN109034249A CN109034249A CN201810852407.5A CN201810852407A CN109034249A CN 109034249 A CN109034249 A CN 109034249A CN 201810852407 A CN201810852407 A CN 201810852407A CN 109034249 A CN109034249 A CN 109034249A
- Authority
- CN
- China
- Prior art keywords
- convolution
- feature map
- isv
- convolution kernel
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000005457 optimization Methods 0.000 title claims abstract description 44
- 238000012545 processing Methods 0.000 claims description 31
- 239000013598 vector Substances 0.000 claims description 24
- 238000007781 pre-processing Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 abstract description 9
- 238000013528 artificial neural network Methods 0.000 description 18
- 230000008569 process Effects 0.000 description 14
- 238000013527 convolutional neural network Methods 0.000 description 10
- 241000282472 Canis lupus familiaris Species 0.000 description 5
- 241000282326 Felis catus Species 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000008034 disappearance Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 150000001875 compounds Chemical class 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses based on convolution optimization method, device, terminal device and the computer readable storage medium for decomposing radial symmetric convolution kernel, which comprises input images to be recognized, and the images to be recognized is pre-processed；It is utilized respectively the convolution kernel and (m-1)/2 1*m (m=2k+3 for decomposing 1 1*1 that m*m radial symmetric convolution kernel obtains in advance, k ∈ N) convolution kernel, convolution is carried out to by pretreated images to be recognized, obtain the fisrt feature figure of 1 1*1 and the second feature figure of (m-1)/2 1*m (m=2k+3, k ∈ N)；Convolution further is carried out to second feature figure, obtains third feature figure；To fisrt feature figure and (third feature figure is summed, and target signature is obtained, and exports the target signature.The present invention achievees the purpose that optimize convolution by reducing parameter amount on the basis of reducing radial symmetric convolution kernel calculation amount.
Description
Technical Field
The invention relates to the technical field of neural networks, in particular to a convolution optimization method and device based on a decomposed radial symmetric convolution kernel, terminal equipment and a computer readable storage medium.
Background
The Convolutional Neural Network (CNN) is a neural network which is most commonly used in the field of image processing and recognition in recent years, and has the advantages of good feature classification effect and easiness in high-dimensional data processing, but the convolutional neural network is easy to generate an overfitting phenomenon, and the robustness of the convolutional neural network is low, so that in the traditional convolutional neural network construction, an original training set picture is often subjected to mirroring and large-angle rotation processing to increase the robustness of the convolutional neural network, and the convolutional neural network can recognize pictures at any angle; however, this conventional method has a problem that the amount of data increases, and the training time increases.
In the prior art, in response to the problems of the conventional method, a convolution kernel with radial symmetry property is generally adopted, which can provide good robustness in the use of a convolutional neural network and can reduce the possibility of occurrence of an overfitting phenomenon. However, the calculation amount of the radial symmetric convolution kernel is too large in operation, and the conventional technical scheme for solving the problem of the too large calculation amount mainly adopts a convolution kernel clipping method and a multi-channel convolution optimization algorithm in a convolution neural network. However, when implementing the prior art solutions, the inventors have found that these optimization algorithms suffer from an excessive number of parameters in use.
Disclosure of Invention
The technical problem to be solved by the present invention is to provide a convolution optimization method, apparatus, terminal device and computer readable storage medium based on decomposed radial symmetric convolution kernel, so as to achieve the purpose of optimizing convolution by reducing parameters on the basis of reducing the calculation amount of radial symmetric convolution kernel.
In order to solve the above technical problem, an embodiment of the present invention provides a convolution optimization method based on a decomposed radially symmetric convolution kernel, which is suitable for being executed in a computing device, and includes the following steps:
inputting an image to be recognized, and preprocessing the image to be recognized;
respectively convolving the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m is 2k +3, k belongs to N) obtained by decomposing m × m radial symmetric convolution kernels in advance to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m is 2k +3, k belongs to N); then, convolving (m-1)/2 second feature maps of 1 m (m is 2k +3, k is N) by using convolution kernels of (m-1)/2 m (m is 2k +3, k is N) which are obtained by decomposing m radial symmetric convolution kernels in advance and correspond to the convolution kernels of (m-1)/2 m (m is 2k +3, k is N) one by one to obtain a third feature map of (m-1)/2 m 1(m is 2k +3, k is N);
and summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m is 2k +3, k is equal to N) to obtain a target feature map, and outputting the target feature map.
Further, the preprocessing the image to be recognized specifically includes:
according to preset parameters, randomly stretching and shading the image to be identified, and adding specific Gaussian noise;
and further, according to the requirement of convolution processing, rotating and cutting the image to be identified by an angle of 0-pi/2.
Further, the convolution kernel matrix a of each convolution kernel satisfies the following formula:
wherein,
further, each convolution kernel of m × 1(m ═ 2k +3, k ∈ N) and a corresponding convolution kernel of m × 1(m ═ 2k +3, k ∈ N) form an equal ratio symmetric vector group, i.e. ISV, specifically:
let anIs a parameter value, a1Constantly equal to 1, the length of the ISV is m (m ═ 2k +3, k ∈ N),
ISV＝(ISV_1,ISV_2)；
wherein, ISV _1 is a vector of 1 × m, and ISV _2 is a vector of m × 1;
further, after the summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N) to obtain the target feature map, and outputting the target feature map, the method further includes:
judging whether the direction of the image has influence on the recognition result;
if so, 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, k ∈ N) are subjected to global average pooling, and output 1+ (m-1)/2 values are subjected to softmax processing, so that the probability of the target feature map is obtained.
Further, the convolution optimization method based on the decomposed radial symmetric convolution kernel further includes:
and if not, performing spatial pyramid pooling and full-connection layer processing on 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, and k belongs to N), and performing softmax processing on the output 1+ (m-1)/2 values to obtain a judgment result of the target feature map.
An embodiment of the present invention further provides a convolution optimization apparatus based on decomposed radially symmetric convolution kernels, including:
the input module is used for inputting an image to be recognized and preprocessing the image to be recognized;
a convolution optimization module, configured to convolve the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m ═ 2k +3, k ∈ N), which are obtained by decomposing m × m radial symmetric convolution kernels in advance, to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m ═ 2k +3, k ∈ N); then, convolving (m-1)/2 second feature maps of 1 m (m is 2k +3, k is N) by using convolution kernels of (m-1)/2 m (m is 2k +3, k is N) which are obtained by decomposing m radial symmetric convolution kernels in advance and correspond to the convolution kernels of (m-1)/2 m (m is 2k +3, k is N) one by one to obtain a third feature map of (m-1)/2 m 1(m is 2k +3, k is N);
and the output module is used for summing the 1 first feature map with 1 x 1 and the third feature map with (m-1)/2 m x 1(m is 2k +3, k belongs to N) to obtain a target feature map and outputting the target feature map.
Further, each convolution kernel of m × 1(m ═ 2k +3, k ∈ N) and a corresponding convolution kernel of m × 1(m ═ 2k +3, k ∈ N) form an equal ratio symmetric vector group, i.e. ISV, specifically:
let anIs a parameter value, a1Constantly equal to 1, the length of the ISV is m (m ═ 2k +3, k ∈ N),
ISV＝(ISV_1,ISV_2)；
wherein, ISV _1 is a vector of 1 × m, and ISV _2 is a vector of m × 1;
an embodiment of the present invention further provides a convolution optimization terminal device based on a decomposed radial symmetric convolution kernel, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing a method of convolution optimization based on a decomposed radially symmetric convolution kernel as described above when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program runs, the apparatus on which the computer-readable storage medium is located is controlled to execute the above-mentioned convolution optimization method based on decomposed radial symmetric convolution kernel.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
the invention provides a convolution optimization method, a convolution optimization device, terminal equipment and a computer readable storage medium based on a decomposed radial symmetric convolution kernel, wherein the method comprises the following steps: inputting an image to be recognized, and preprocessing the image to be recognized; respectively convolving the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m is 2k +3, k belongs to N) obtained by decomposing m × m radial symmetric convolution kernels in advance to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m is 2k +3, k belongs to N); performing convolution on the (m-1)/2 second feature maps of 1 × m (m ═ 2k +3, k ∈ N) by using convolution kernels of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N) which are in one-to-one correspondence with the convolution kernels of (m-1)/2 m (m ═ 2k +3, k ∈ N), so as to obtain a third feature map of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N); and summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m is 2k +3, k is equal to N) to obtain a target feature map, and outputting the target feature map. The invention achieves the aim of optimizing the convolution by reducing the parameter quantity on the basis of reducing the calculation quantity of the radial symmetric convolution kernel.
Drawings
FIG. 1 is a flowchart illustrating a convolution optimization method based on decomposed radially symmetric convolution kernels according to a first embodiment of the present invention;
FIG. 2 is a schematic flow chart of the process of the decomposed 3 x 3 radially symmetric convolution kernel to check an image according to the first embodiment of the present invention;
FIG. 3 is a schematic flow chart of a process for processing a decomposed 5 x 5 radially symmetric convolution kernel on an image according to a first embodiment of the present invention;
FIG. 4 is a schematic flow chart of a process for processing a decomposed m x m radially symmetric convolution kernel on an image according to a first embodiment of the present invention;
FIG. 5 is a flowchart illustrating a procedure of classifying images of cats and dogs by a neural network including a decomposed symmetric convolution kernel according to a first embodiment of the present invention;
FIG. 6 is a flow chart illustrating the process of a neural network with decomposed symmetric convolution kernels to classify MNIST images according to a first embodiment of the present invention;
fig. 7 is a schematic structural diagram of a convolution optimization apparatus based on a decomposed radially symmetric convolution kernel according to a second embodiment of the present invention.
Detailed Description
In the following, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment of the present invention:
please refer to fig. 1-5.
As shown in fig. 1, the present embodiment provides a convolution optimization method based on a decomposed radial symmetric convolution kernel, where the convolution optimization method based on a decomposed radial symmetric convolution kernel is suitable for being executed in a computing device, and includes the following steps:
s101, inputting an image to be identified, and preprocessing the image to be identified;
s102, respectively utilizing 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m ═ 2k +3, k ∈ N) obtained by decomposing m × m radial symmetric convolution kernels in advance to convolve the preprocessed image to be recognized, and obtaining 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m ═ 2k +3, k ∈ N); then, convolving (m-1)/2 second feature maps of 1 m (m is 2k +3, k is N) by using convolution kernels of (m-1)/2 m (m is 2k +3, k is N) which are obtained by decomposing m radial symmetric convolution kernels in advance and correspond to the convolution kernels of (m-1)/2 m (m is 2k +3, k is N) one by one to obtain a third feature map of (m-1)/2 m 1(m is 2k +3, k is N);
and S103, summing the 1 first feature map of 1 x 1 and the third feature map of (m-1)/2 m x 1(m is 2k +3, k is equal to N) to obtain a target feature map, and outputting the target feature map.
The convolution optimization method based on the decomposed radial symmetric convolution kernel enhances robustness, reduces the number of parameters, accelerates the fitting speed of the neural network and prevents overfitting.
For step S101, the preprocessing the image to be recognized specifically includes:
according to preset parameters, randomly stretching and shading the image to be identified, and adding specific Gaussian noise;
and further, according to the requirement of convolution processing, rotating and cutting the image to be identified by an angle of 0-pi/2.
In the present embodiment, it can be understood that, in order to increase robustness, a slight gaussian noise is added to the image to be recognized to be input, and random stretching and shading change with small amplitude are performed. Due to the inherent advantages of the radial symmetric convolution kernel, the picture to be recognized does not need mirror image transformation, and meanwhile, the rotation step in the preprocessing only needs to rotate by an angle of 0-pi/2. Because the network only applies the convolutional neural network, the network has no full connection layer, and has no requirement on the size, and no processing is needed to be carried out on the picture size except for carrying out necessary cutting on the rotated picture.
For step S102, the convolution kernel matrix a of each convolution kernel satisfies the following formula:
wherein,
in this embodiment, two matrices are as follows:
taking into account that pictures are rotatedThe pixel values change at multiple angles, only the angle of rotation being taken into accountThe result of the convolution processing of the picture rotated by the angle theta is equal to the result obtained by directly processing the original picture, namely, the processed mirror image picture also generates the same result as the original picture.
It can be inferred that this is achieved by setting the convolution kernel to a matrix satisfying a when it is initialized.
In this embodiment, the convolution kernel, in addition to having good robustness, also reduces the large number of parameters that the conventional convolution kernel hasThe number of times of the total number of the parts,i.e. the method can reduce the maximumParameter (d) ofAmount of the compound (A). The convergence rate of the model is effectively accelerated, and overfitting is prevented.
The method has more obvious effect in a high-dimensional convolution network, and the kernel can be reduced at most in an x-dimensional convolution neural networkThe optimization effect is very obvious.
In this embodiment, each convolution kernel of m × 1(m ═ 2k +3, k ∈ N) and a corresponding convolution kernel of m × 1(m ═ 2k +3, k ∈ N) form an equal ratio symmetric vector group, that is, ISV, specifically:
let anIs a parameter value, a1Constantly equal to 1, the length of the ISV is m (m ═ 2k +3, k ∈ N),
ISV＝(ISV_1,ISV_2)；
wherein, ISV _1 is a vector of 1 × m, and ISV _2 is a vector of m × 1;
in the present embodiment, since a large number of repeated operations are involved in the conventional convolution operation, which consumes much unnecessary operation time, the present embodiment decomposes the length-width-3 radially symmetric convolution kernel into a pair of length-3 ISV vectors and 1 × 1 convolution kernel combinations, and the specific variables of the convolution kernels are defined as follows:
ISV _1 convolution kernel:
ISV _2 convolution kernel:
1 ＊ 1 convolution kernel:
a in the above convolution kernelnAnd b are parameter values.
As shown in fig. 2, the specific convolution process is as follows:
assuming that the original is P, firstly, an ISV _1 convolution kernel and a 1 ＊ 1 convolution kernel are respectively convolved on P to obtain feature maps P1 and P2, then, an ISV _2 convolution kernel is used for convolving P1 to obtain a feature map P3, and then, P2 and P3 are added to obtain an output feature map P _ output.
Through mathematical operation, the convolution process in the convolution process is equivalent to using convolution kernel shape
The radially symmetrical convolution kernel is used for convolution, in addition, only 7 times of multiplication operation and 5 times of addition operation are used for the same pixel point in the convolution process, 9 times of multiplication and 8 times of addition operation are required in the traditional convolution operation, and a lot of operation time is reduced.
In this embodiment, as shown in fig. 3, it is preferable that, if 5 × 5 radially symmetric convolution kernels are to be applied, they can be decomposed into two pairs of ISV vectors with lengths of 5, 3 and 1 × 1 convolution kernel combinations (a of each ISV pair)nAll different).
Further, as shown in fig. 4, to generalize, if a radially symmetric convolution kernel of m × m (m × 2k +3, k ∈ N) is to be applied, it can be decomposed into (m-1)/2 pairs of ISV vectors with lengths of 3,5,7, …, m and convolution kernel combinations of 1 ∈ 1, respectively.
In the present embodiment, this movement is in accordance with the above descriptionThe calculation mode can greatly reduce the operation amount when the convolution kernel size is larger, and when the convolution kernel size is m × m, the operation amount of multiplication is the original operation amountThe operation amount of addition is the original oneAnd (4) doubling.
For step S103, further, after the summing the 1 first feature map of 1 × 1 and the (m-1)/2 third feature maps of m × 1(m ═ 2k +3, k ∈ N) to obtain a target feature map, and outputting the target feature map, the method further includes:
judging whether the direction of the image has influence on the recognition result;
if so, 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, k ∈ N) are subjected to global average pooling, and output 1+ (m-1)/2 values are subjected to softmax processing, so that the probability of the target feature map is obtained.
In the embodiment, the convolution kernel splitting algorithm can obviously reduce the operation amount in the high-dimensional convolution kernel. As shown in fig. 5, cat and dog are classified and identified by using decomposed radial symmetric convolutional network.
In the construction of the neural network, 8 layers of 3-by-3 decomposed radial symmetric convolution network are used at the front end of the network, and residual layers are connected in the network, so that the gradient is prevented from disappearing.
Because the direction of the image has no influence on the recognition result, the last two feature maps are averaged and pooled by adopting a global average pooling method at the tail end of the neural network, and the probability of being a cat or a dog is obtained by performing softmax processing on the two output values
Further, the convolution optimization method based on the decomposed radial symmetric convolution kernel further includes:
and if not, performing spatial pyramid pooling and full-connection layer processing on 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, and k belongs to N), and performing softmax processing on the output 1+ (m-1)/2 values to obtain a judgment result of the target feature map.
In this embodiment, as shown in fig. 6, the MNIST dataset is identified using a decomposed radially symmetric convolutional network:
in the image processing stage, the preprocessing method described above is still used, except that no rotation is involved in the preprocessing. The main body of the neural network also adopts a radial symmetric convolution network of 3 x 3 decomposition using 8 layers at the front end of the network, and connects residual layers therein to prevent gradient disappearance. But the number of layers of the neural network is correspondingly reduced due to the reduction of data.
Since the direction of the image has an influence on the recognition result, if the recognition 6 and 9 are wrong due to uncertain direction, the spatial pyramid pooling and the full connection layer collocation softmax are adopted at the end of the neural network to output the result judged as each number.
It should be noted that these are just two simple examples in this embodiment, and the network can be used in a larger and more complex recognition system and achieve good results.
The convolution optimization method based on the decomposed radial symmetric convolution kernel provided by the embodiment comprises the following steps: inputting an image to be recognized, and preprocessing the image to be recognized; respectively convolving the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m is 2k +3, k belongs to N) obtained by decomposing m × m radial symmetric convolution kernels in advance to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m is 2k +3, k belongs to N); performing convolution on the (m-1)/2 second feature maps of 1 × m (m ═ 2k +3, k ∈ N) by using convolution kernels of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N) which are in one-to-one correspondence with the convolution kernels of (m-1)/2 m (m ═ 2k +3, k ∈ N), so as to obtain a third feature map of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N); and summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m is 2k +3, k is equal to N) to obtain a target feature map, and outputting the target feature map. The embodiment reduces the parameter quantity on the basis of reducing the calculation quantity of the radial symmetric convolution kernel, thereby achieving the purpose of optimizing the convolution.
Second embodiment of the invention:
please refer to fig. 7.
As shown in fig. 7, this embodiment further provides a convolution optimization apparatus based on decomposed radially symmetric convolution kernels, including:
the input module 201 is configured to input an image to be recognized and perform preprocessing on the image to be recognized.
The preprocessing is performed on the image to be identified, and specifically comprises the following steps:
according to preset parameters, randomly stretching and shading the image to be identified, and adding specific Gaussian noise;
and further, according to the requirement of convolution processing, rotating and cutting the image to be identified by an angle of 0-pi/2.
In the present embodiment, it can be understood that, in order to increase robustness, a slight gaussian noise is added to the image to be recognized to be input, and random stretching and shading change with small amplitude are performed. Due to the inherent advantages of the radial symmetric convolution kernel, the picture to be recognized does not need mirror image transformation, and meanwhile, the rotation step in the preprocessing only needs to rotate by an angle of 0-pi/2. Because the network only applies the convolutional neural network, the network has no full connection layer, and has no requirement on the size, and no processing is needed to be carried out on the picture size except for carrying out necessary cutting on the rotated picture.
A convolution optimization module 202, configured to convolve the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m ═ 2k +3, k ∈ N) obtained by decomposing m × m radial symmetric convolution kernels in advance, respectively, to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m ═ 2k +3, k ∈ N); and then (m-1)/2 convolution kernels of m-1 (m-2 k +3, k epsilon N) which are obtained by decomposing m-m radial symmetric convolution kernels in advance and correspond to the convolution kernels of (m-1)/2 m-1 (m-2 k +3, k epsilon N) in a one-to-one mode are utilized to perform convolution on the second feature map of (m-1)/2 m (m-2 k +3, k epsilon N) so as to obtain a third feature map of (m-1)/2 m-1 (m-2 k +3, k epsilon N).
Wherein, the convolution kernel matrix A of each convolution kernel satisfies the following formula:
wherein,
in this embodiment, two matrices are as follows:
taking into account that pictures are rotatedThe pixel values change at multiple angles, only the angle of rotation being taken into accountThe result of the convolution processing of the picture rotated by the angle theta is equal to the result obtained by directly processing the original picture, namely, the processed mirror image picture also generates the same result as the original picture.
It can be inferred that this is achieved by setting the convolution kernel to a matrix satisfying a when it is initialized.
In the bookIn an embodiment, the convolution kernel, in addition to having good robustness, also reduces the large number of parameters of the kernel that are of conventional convolution kernelsThe number of times of the total number of the parts,i.e. the method can reduce the maximumThe amount of the above-mentioned components. The convergence rate of the model is effectively accelerated, and overfitting is prevented.
The method has more obvious effect in a high-dimensional convolution network, and the kernel can be reduced at most in an x-dimensional convolution neural networkThe optimization effect is very obvious.
In this embodiment, each convolution kernel of m × 1(m ═ 2k +3, k ∈ N) and a corresponding convolution kernel of m × 1(m ═ 2k +3, k ∈ N) form an equal ratio symmetric vector group, that is, ISV, specifically:
let anIs a parameter value, a1Constantly equal to 1, the length of the ISV is m (m ═ 2k +3, k ∈ N),
ISV＝(ISV_1,ISV_2)；
wherein, ISV _1 is a vector of 1 × m, and ISV _2 is a vector of m × 1;
in the present embodiment, since a large number of repeated operations are involved in the conventional convolution operation, which consumes much unnecessary operation time, the present embodiment decomposes the length-width-3 radially symmetric convolution kernel into a pair of length-3 ISV vectors and 1 × 1 convolution kernel combinations, and the specific variables of the convolution kernels are defined as follows:
ISV _1 convolution kernel:
ISV _2 convolution kernel:
1 ＊ 1 convolution kernel:
a in the above convolution kernelnAnd b are parameter values.
As shown in fig. 2, the specific convolution process is as follows:
assuming that the original is P, firstly, an ISV _1 convolution kernel and a 1 ＊ 1 convolution kernel are respectively convolved on P to obtain feature maps P1 and P2, then, an ISV _2 convolution kernel is used for convolving P1 to obtain a feature map P3, and then, P2 and P3 are added to obtain an output feature map P _ output.
Through mathematical operation, the convolution process in the convolution process is equivalent to using convolution kernel shape
The radially symmetrical convolution kernel is used for convolution, in addition, only 7 times of multiplication operation and 5 times of addition operation are used for the same pixel point in the convolution process, 9 times of multiplication and 8 times of addition operation are required in the traditional convolution operation, and a lot of operation time is reduced.
In this implementationIn the example, as shown in fig. 3, it is preferred that, if a radially symmetric convolution kernel of 5 x 5 is to be applied, it can be decomposed into two pairs of ISV vectors of 5, 3 lengths and 1 x 1 convolution kernel combinations (a for each ISV pair)nAll different).
Further, as shown in fig. 4, to generalize, if a radially symmetric convolution kernel of m × m (m × 2k +3, k ∈ N) is to be applied, it can be decomposed into (m-1)/2 pairs of ISV vectors with lengths of 3,5,7, …, m and convolution kernel combinations of 1 ∈ 1, respectively.
In this embodiment, according to the above description, this operation method can greatly reduce the operation amount when the convolution kernel size is large, and when the convolution kernel size is m × m, the operation amount of multiplication is the original oneThe operation amount of addition is the original oneAnd (4) doubling.
And the output module 203 is configured to sum up 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, and k ∈ N) to obtain a target feature map, and output the target feature map.
Further, after the summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N) to obtain the target feature map, and outputting the target feature map, the method further includes:
judging whether the direction of the image has influence on the recognition result;
if so, 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, k ∈ N) are subjected to global average pooling, and output 1+ (m-1)/2 values are subjected to softmax processing, so that the probability of the target feature map is obtained.
In the embodiment, the convolution kernel splitting algorithm can obviously reduce the operation amount in the high-dimensional convolution kernel. As shown in fig. 5, cat and dog are classified and identified by using decomposed radial symmetric convolutional network.
In the construction of the neural network, 8 layers of 3-by-3 decomposed radial symmetric convolution network are used at the front end of the network, and residual layers are connected in the network, so that the gradient is prevented from disappearing.
Because the direction of the image has no influence on the recognition result, the last two feature maps are averaged and pooled by adopting a global average pooling method at the tail end of the neural network, and the probability of being a cat or a dog is obtained by performing softmax processing on the two output values
Further, the convolution optimization method based on the decomposed radial symmetric convolution kernel further includes:
and if not, performing spatial pyramid pooling and full-connection layer processing on 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, and k belongs to N), and performing softmax processing on the output 1+ (m-1)/2 values to obtain a judgment result of the target feature map.
In this embodiment, as shown in fig. 6, the MNIST dataset is identified using a decomposed radially symmetric convolutional network:
in the image processing stage, the preprocessing method described above is still used, except that no rotation is involved in the preprocessing. The main body of the neural network also adopts a radial symmetric convolution network of 3 x 3 decomposition using 8 layers at the front end of the network, and connects residual layers therein to prevent gradient disappearance. But the number of layers of the neural network is correspondingly reduced due to the reduction of data.
Since the direction of the image has an influence on the recognition result, if the recognition 6 and 9 are wrong due to uncertain direction, the spatial pyramid pooling and the full connection layer collocation softmax are adopted at the end of the neural network to output the result judged as each number.
It should be noted that these are just two simple examples in this embodiment, and the network can be used in a larger and more complex recognition system and achieve good results.
According to the convolution optimization device based on the decomposed radial symmetric convolution kernel, an image to be recognized is input, and the image to be recognized is preprocessed; respectively convolving the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m is 2k +3, k belongs to N) obtained by decomposing m × m radial symmetric convolution kernels in advance to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m is 2k +3, k belongs to N); performing convolution on the (m-1)/2 second feature maps of 1 × m (m ═ 2k +3, k ∈ N) by using convolution kernels of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N) which are in one-to-one correspondence with the convolution kernels of (m-1)/2 m (m ═ 2k +3, k ∈ N), so as to obtain a third feature map of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N); and summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m is 2k +3, k is equal to N) to obtain a target feature map, and outputting the target feature map. The embodiment reduces the parameter quantity on the basis of reducing the calculation quantity of the radial symmetric convolution kernel, thereby achieving the purpose of optimizing the convolution.
An embodiment of the present invention further provides a convolution optimization terminal device based on a decomposed radial symmetric convolution kernel, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing a method of convolution optimization based on a decomposed radially symmetric convolution kernel as described above when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program runs, the apparatus on which the computer-readable storage medium is located is controlled to execute the above-mentioned convolution optimization method based on decomposed radial symmetric convolution kernel.
Compared with the prior art, the embodiment of the invention has the beneficial effects that:
the invention provides a convolution optimization method, a convolution optimization device, terminal equipment and a computer readable storage medium based on a decomposed radial symmetric convolution kernel, wherein the method comprises the following steps: inputting an image to be recognized, and preprocessing the image to be recognized; respectively convolving the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m is 2k +3, k belongs to N) obtained by decomposing m × m radial symmetric convolution kernels in advance to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m is 2k +3, k belongs to N); performing convolution on the (m-1)/2 second feature maps of 1 × m (m ═ 2k +3, k ∈ N) by using convolution kernels of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N) which are in one-to-one correspondence with the convolution kernels of (m-1)/2 m (m ═ 2k +3, k ∈ N), so as to obtain a third feature map of (m-1)/2 m × 1(m ═ 2k +3, k ∈ N); and summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m is 2k +3, k is equal to N) to obtain a target feature map, and outputting the target feature map. The invention reduces the parameter quantity on the basis of reducing the calculation quantity of the radial symmetric convolution kernel, thereby achieving the aim of optimizing the convolution
While the foregoing is directed to the preferred embodiment of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Claims (10)
1. A convolution optimization method based on decomposed radially symmetric convolution kernels, adapted to be executed in a computing device, comprising the steps of:
inputting an image to be recognized, and preprocessing the image to be recognized;
respectively convolving the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m is 2k +3, k belongs to N) obtained by decomposing m × m radial symmetric convolution kernels in advance to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m is 2k +3, k belongs to N); then, convolving (m-1)/2 second feature maps of 1 m (m is 2k +3, k is N) by using convolution kernels of (m-1)/2 m (m is 2k +3, k is N) which are obtained by decomposing m radial symmetric convolution kernels in advance and correspond to the convolution kernels of (m-1)/2 m (m is 2k +3, k is N) one by one to obtain a third feature map of (m-1)/2 m 1(m is 2k +3, k is N);
and summing the 1 first feature map of 1 × 1 and the third feature map of (m-1)/2 m × 1(m is 2k +3, k is equal to N) to obtain a target feature map, and outputting the target feature map.
2. The method of claim 1, wherein the preprocessing the image to be recognized is specifically:
according to preset parameters, randomly stretching and shading the image to be identified, and adding specific Gaussian noise;
and further, according to the requirement of convolution processing, rotating and cutting the image to be identified by an angle of 0-pi/2.
3. A method of convolution optimization based on decomposed radially symmetric convolution kernels according to claim 1, characterized in that the convolution kernel matrix a of each convolution kernel satisfies the following formula:
wherein,
4. a convolution optimization method based on decomposed radially symmetric convolution kernels according to claim 1, wherein each convolution kernel of m × 1(m — 2k +3, k ∈ N) and a corresponding convolution kernel of m × 1(m — 2k +3, k ∈ N) form an equal ratio symmetric vector set, i.e. ISV, specifically:
let anAs the value of the parameter(s),a1constantly equal to 1, the length of the ISV is m (m ═ 2k +3, k ∈ N),
ISV＝(ISV_1,ISV_2)；
wherein, ISV _1 is a vector of 1 × m, and ISV _2 is a vector of m × 1;
5. the method of claim 1, wherein after summing the 1 first feature map of 1 x 1 and the third feature map of (m-1)/2 m x 1(m 2k +3, k e N) to obtain the target feature map and outputting the target feature map, the method further comprises:
judging whether the direction of the image has influence on the recognition result;
if so, 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, k ∈ N) are subjected to global average pooling, and output 1+ (m-1)/2 values are subjected to softmax processing, so that the probability of the target feature map is obtained.
6. The method of claim 5 for convolution optimization based on a decomposed radially symmetric convolution kernel, further comprising:
and if not, performing spatial pyramid pooling and full-connection layer processing on 1 first feature map of 1 × 1 and (m-1)/2 third feature maps of m × 1(m is 2k +3, and k belongs to N), and performing softmax processing on the output 1+ (m-1)/2 values to obtain a judgment result of the target feature map.
7. A convolution optimization apparatus based on decomposed radially symmetric convolution kernels, comprising:
the input module is used for inputting an image to be recognized and preprocessing the image to be recognized;
a convolution optimization module, configured to convolve the preprocessed image to be recognized by using 1 convolution kernel of 1 × 1 and (m-1)/2 convolution kernels of 1 × m (m ═ 2k +3, k ∈ N), which are obtained by decomposing m × m radial symmetric convolution kernels in advance, to obtain 1 first feature map of 1 × 1 and (m-1)/2 second feature maps of 1 m (m ═ 2k +3, k ∈ N); then, convolving (m-1)/2 second feature maps of 1 m (m is 2k +3, k is N) by using convolution kernels of (m-1)/2 m (m is 2k +3, k is N) which are obtained by decomposing m radial symmetric convolution kernels in advance and correspond to the convolution kernels of (m-1)/2 m (m is 2k +3, k is N) one by one to obtain a third feature map of (m-1)/2 m 1(m is 2k +3, k is N);
and the output module is used for summing the 1 first feature map with 1 x 1 and the third feature map with (m-1)/2 m x 1(m is 2k +3, k belongs to N) to obtain a target feature map and outputting the target feature map.
8. A convolution optimization apparatus based on decomposed radially symmetric convolution kernels according to claim 7, wherein each convolution kernel of m × 1(m — 2k +3, k ∈ N) and a corresponding convolution kernel of m × 1(m — 2k +3, k ∈ N) form an equal ratio symmetric vector set, i.e. ISV, specifically:
let anIs a parameter value, a1Constantly equal to 1, the length of the ISV is m (m ═ 2k +3, k ∈ N),
ISV＝(ISV_1,ISV_2)；
wherein, ISV _1 is a vector of 1 × m, and ISV _2 is a vector of m × 1;
9. a convolution optimization terminal device based on decomposed radial symmetric convolution kernels is characterized by comprising: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the computer program when executed by the processor implementing the method of convolution optimization based on decomposed radially symmetric convolution kernels of any of claims 1 to 6.
10. A computer-readable storage medium, comprising a stored computer program, wherein the computer program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform a method of convolution optimization based on a decomposed radially symmetric convolution kernel according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810852407.5A CN109034249B (en) | 2018-07-27 | 2018-07-27 | Convolution optimization method and device based on decomposed radial symmetric convolution kernel, terminal equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810852407.5A CN109034249B (en) | 2018-07-27 | 2018-07-27 | Convolution optimization method and device based on decomposed radial symmetric convolution kernel, terminal equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109034249A true CN109034249A (en) | 2018-12-18 |
| CN109034249B CN109034249B (en) | 2021-08-06 |
Family
ID=64646640
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810852407.5A Active CN109034249B (en) | 2018-07-27 | 2018-07-27 | Convolution optimization method and device based on decomposed radial symmetric convolution kernel, terminal equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109034249B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886273A (en) * | 2019-02-26 | 2019-06-14 | 四川大学华西医院 | A kind of CMR classification of image segmentation system |
| CN109978137A (en) * | 2019-03-20 | 2019-07-05 | 厦门美图之家科技有限公司 | A kind of processing method of convolutional neural networks |
| CN110334798A (en) * | 2019-03-13 | 2019-10-15 | 北京地平线机器人技术研发有限公司 | Characteristic extracting method and device, instruction generation method and device |
| CN111680678A (en) * | 2020-05-25 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Target area identification method, device, equipment and readable storage medium |
| CN112712461A (en) * | 2019-10-24 | 2021-04-27 | Tcl集团股份有限公司 | Image deconvolution processing method and device and terminal equipment |
| CN113591025A (en) * | 2021-08-03 | 2021-11-02 | 深圳思谋信息科技有限公司 | Feature map processing method and device, convolutional neural network accelerator and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346621A (en) * | 2013-07-30 | 2015-02-11 | 展讯通信(天津)有限公司 | Method and device for creating eye template as well as method and device for detecting eye state |
| CN105139017A (en) * | 2015-08-27 | 2015-12-09 | 重庆理工大学 | License plate positioning algorithm fusing affine invariant corner feature and visual color feature |
| CN106557743A (en) * | 2016-10-26 | 2017-04-05 | 桂林电子科技大学 | A kind of face characteristic extraction system and method based on FECNN |
| US9953236B1 (en) * | 2017-03-10 | 2018-04-24 | TuSimple | System and method for semantic segmentation using dense upsampling convolution (DUC) |
| CN107967459A (en) * | 2017-12-07 | 2018-04-27 | 北京小米移动软件有限公司 | convolution processing method, device and storage medium |
-
2018
- 2018-07-27 CN CN201810852407.5A patent/CN109034249B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104346621A (en) * | 2013-07-30 | 2015-02-11 | 展讯通信(天津)有限公司 | Method and device for creating eye template as well as method and device for detecting eye state |
| CN105139017A (en) * | 2015-08-27 | 2015-12-09 | 重庆理工大学 | License plate positioning algorithm fusing affine invariant corner feature and visual color feature |
| CN106557743A (en) * | 2016-10-26 | 2017-04-05 | 桂林电子科技大学 | A kind of face characteristic extraction system and method based on FECNN |
| US9953236B1 (en) * | 2017-03-10 | 2018-04-24 | TuSimple | System and method for semantic segmentation using dense upsampling convolution (DUC) |
| CN107967459A (en) * | 2017-12-07 | 2018-04-27 | 北京小米移动软件有限公司 | convolution processing method, device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 胡帮义: "基于数字图像检测和虚拟现实的桥梁施工控制技术研究", <<中国优秀硕士学位论文全文数据库 工程科技II辑>> * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109886273A (en) * | 2019-02-26 | 2019-06-14 | 四川大学华西医院 | A kind of CMR classification of image segmentation system |
| CN110334798A (en) * | 2019-03-13 | 2019-10-15 | 北京地平线机器人技术研发有限公司 | Characteristic extracting method and device, instruction generation method and device |
| CN110334798B (en) * | 2019-03-13 | 2021-06-08 | 北京地平线机器人技术研发有限公司 | Feature data extraction method and device and instruction generation method and device |
| CN109978137A (en) * | 2019-03-20 | 2019-07-05 | 厦门美图之家科技有限公司 | A kind of processing method of convolutional neural networks |
| CN109978137B (en) * | 2019-03-20 | 2021-03-16 | 厦门美图之家科技有限公司 | Processing method of convolutional neural network |
| CN112712461A (en) * | 2019-10-24 | 2021-04-27 | Tcl集团股份有限公司 | Image deconvolution processing method and device and terminal equipment |
| CN112712461B (en) * | 2019-10-24 | 2024-04-19 | Tcl科技集团股份有限公司 | Image deconvolution processing method and device and terminal equipment |
| CN111680678A (en) * | 2020-05-25 | 2020-09-18 | 腾讯科技（深圳）有限公司 | Target area identification method, device, equipment and readable storage medium |
| CN113591025A (en) * | 2021-08-03 | 2021-11-02 | 深圳思谋信息科技有限公司 | Feature map processing method and device, convolutional neural network accelerator and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109034249B (en) | 2021-08-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109034249B (en) | Convolution optimization method and device based on decomposed radial symmetric convolution kernel, terminal equipment and computer readable storage medium | |
| US12033038B2 (en) | Learning data augmentation policies | |
| Liu et al. | Learning converged propagations with deep prior ensemble for image enhancement | |
| Zhang et al. | Learning deep CNN denoiser prior for image restoration | |
| Liu et al. | Deep proximal unrolling: Algorithmic framework, convergence analysis and applications | |
| CN108229479B (en) | Training method and device of semantic segmentation model, electronic equipment and storage medium | |
| US11636670B2 (en) | Method for recognizing object in image | |
| CN110059733A (en) | The optimization and fast target detection method, device of convolutional neural networks | |
| CN107292458B (en) | Prediction method and prediction device applied to neural network chip | |
| CN107292352B (en) | Image classification method and device based on convolutional neural network | |
| CN109389166A (en) | The depth migration insertion cluster machine learning method saved based on partial structurtes | |
| US11663705B2 (en) | Image haze removal method and apparatus, and device | |
| CN113870283B (en) | Portrait matting method, device, computer equipment and readable storage medium | |
| CN116452810A (en) | Multi-level semantic segmentation method and device, electronic equipment and storage medium | |
| CN110148136A (en) | Insulator image segmentation method and device and computer readable storage medium | |
| CN113743426A (en) | Training method, device, equipment and computer readable storage medium | |
| CN110880018B (en) | Convolutional neural network target classification method | |
| CN114626042A (en) | Face verification attack method and device | |
| CN116109509A (en) | Real-time low-illumination image enhancement method and system based on pixel-by-pixel gamma correction | |
| Jung et al. | Extension of convolutional neural network with general image processing kernels | |
| Yifei et al. | Flower image classification based on improved convolutional neural network | |
| US20240070816A1 (en) | Diffusion model image generation | |
| CN110807752A (en) | Image attention mechanism processing method based on convolutional neural network | |
| CN116342504A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN113128521B (en) | Method, system, computer equipment and storage medium for extracting characteristics of miniaturized artificial intelligent model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |