WO2021068376A1

WO2021068376A1 - Convolution processing method and system applied to convolutional neural network, and related components

Info

Publication number: WO2021068376A1
Application number: PCT/CN2019/121105
Authority: WO
Inventors: 金良; 范宝余; 郭振华
Original assignee: 浪潮电子信息产业股份有限公司
Priority date: 2019-10-11
Filing date: 2019-11-27
Publication date: 2021-04-15
Also published as: CN110796250A

Abstract

A convolution processing method, system and device applied to a convolutional neural network, and a readable storage medium, comprising: obtaining a target operation object, the target operation object being specifically an input feature (S11); performing side window convolution calculation on the target operation object to obtain calculation results in multiple directions (S12); and performing cross entropy optimization processing on the calculation results in the multiple directions to obtain a convolution result of the target operation object (S13). According to the method, performance loss caused by a convolution operation with a convolution kernel as a central point in an original convolutional neural network is avoided, the ability of the convolution operation to obtain more generalized features of data is improved by comprehensively analyzing a side window convolution operation in the multiple directions, and thus the performance of the convolutional neural network is improved.

Description

Convolution processing method, system and related components applied to convolution neural network

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on October 11, 2019, the application number is 201910963711.1, and the application name is "Convolutional processing methods, systems and related components applied to convolutional neural networks." The entire content is incorporated into this application by reference.

Technical field

This application relates to the field of deep learning, and in particular to a convolution processing method, system and related components applied to a convolutional neural network.

Background technique

In deep learning, convolutional neural network is a relatively important type of neural network. Its biggest feature is convolution operation. It is often used in the training process to extract different features through convolution, and then combine all these features organically. Make corresponding decisions.

However, because the center point of the convolution kernel is selected for the corresponding multiplication and addition operation in the traditional convolution operation, when a pixel is on the boundary, placing the center of the window on the pixel for the convolution operation will blur the edge, which will reduce the characteristic In addition, the convolutional neural network usually has many layers, and each layer has multiple convolution kernel filters. The layers are connected to form a directed acyclic graph. Such a central convolution will increase the When the resolution is reduced, the performance of the convolutional neural network is reduced.

Application content

In view of this, the purpose of this application is to provide a data storage method, system, device, and readable storage medium during abnormal shutdown, so as to store important data during abnormal shutdown and ensure that data is not lost. The specific plan is as follows:

A convolution processing method, system and related components applied to a convolutional neural network to solve the technical problem of fuzzy edges. The specific plan is as follows:

A convolution processing method applied to a convolutional neural network, including:

Acquiring a target operation object; the target operation object is specifically an input feature;

Perform side window convolution calculation on the target operation object to obtain calculation results in multiple directions;

Perform cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object.

Preferably, the process of performing side window convolution calculation on the target operation object to obtain calculation results in multiple directions specifically includes:

Determine the directions of the four computing side windows as upper left, upper right, lower left, and lower right;

The four calculation side windows are used to perform side window convolution calculations on the target operation object to obtain calculation results in four directions.

Preferably, when there are multiple target operation objects, the process of performing side-window convolution calculations on the target operation objects by using the calculation side windows to obtain calculation results in four directions specifically includes:

Performing a unified side window convolution calculation on multiple target operation objects by using the calculation side window in the upper left direction to obtain multiple calculation results in the upper left direction;

Performing a unified side window convolution calculation on a plurality of the target operation objects by using the calculation side window in the upper right direction to obtain multiple calculation results in the upper right direction;

Performing a unified side window convolution calculation on multiple target operation objects by using the calculation side window in the lower left direction to obtain multiple calculation results in the lower left direction;

A unified side window convolution calculation is performed on a plurality of the target operation objects by using the calculation side window in the lower right direction to obtain a plurality of calculation results in the lower right direction.

Preferably, the process of performing cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object specifically includes:

Perform cross-entropy optimization processing on the calculation results in the multiple directions, and determine the calculation result with the smallest cross-entropy value as the convolution result of the target operation object.

Preferably, the target operation object is specifically a boundary input feature and/or a texture input feature.

Preferably, when there are multiple convolution kernels, the convolution processing method further includes: performing a weighted average on the convolution result corresponding to each convolution kernel to obtain the final convolution result of the target operation object.

Correspondingly, this application also discloses a convolution processing system applied to a convolutional neural network, including:

The acquisition module is used to acquire a target operation object; the target operation object is specifically an input feature;

A calculation module, configured to perform side window convolution calculation on the target operation object to obtain calculation results in multiple directions;

The result determination module is configured to perform cross-entropy optimization processing on the calculation results of the multiple directions to obtain the convolution result of the target operation object.

Preferably, the calculation module is specifically used for:

Correspondingly, this application also discloses a convolution processing device applied to a convolutional neural network, including:

Memory, used to store computer programs;

The processor is configured to implement the steps of the convolution processing method applied to the convolutional neural network as described above when the computer program is executed.

Correspondingly, the present application also discloses a readable storage medium having a computer program stored on the readable storage medium, and when the computer program is executed by a processor, the convolution applied to the convolutional neural network as described above is realized. Processing method steps.

The present application discloses a convolution processing method applied to a convolutional neural network, including: obtaining a target operation object; the target operation object is specifically an input feature; performing side window convolution calculation on the target operation object to obtain multiple Calculation results in three directions; performing cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object. This application solves the performance loss caused by the convolution operation with the convolution kernel as the center point in the original convolutional neural network. By comprehensively analyzing the side window convolution operation in multiple directions, the data obtained by the convolution operation is improved The ability to generalize features improves the performance of convolutional neural networks.

Description of the drawings

FIG. 1 is a flowchart of steps of a convolution processing method applied to a convolutional neural network in an embodiment of the application;

Fig. 2a is a schematic diagram of an image of a side window in an embodiment of the application;

2b, 2c, and 2d are respectively schematic diagrams of images of side windows in different directions in an embodiment of the application;

FIG. 3 is a flowchart of steps of a specific convolution processing method applied to a convolutional neural network in an embodiment of the application;

4 is a structural distribution diagram of a convolution processing system applied to a convolutional neural network in an embodiment of the application;

FIG. 5 is a structural distribution diagram of a convolution processing device applied to a convolutional neural network in an embodiment of the application.

Detailed ways

The technical solutions in the application will be clearly and completely described below in conjunction with the drawings in the specification of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Example one

Because the center point of the convolution kernel is selected for the corresponding multiplication and addition operation in the traditional convolution operation, when a pixel is on the boundary, placing the center of the window on the pixel for the convolution operation will blur the edge, which will reduce the feature’s reliability. Resolvability, coupled with the convolutional neural network usually has many layers, each layer has multiple filters, and the layers are connected to form a directed acyclic graph. Such a convolution at the center position will aggravate the decrease in resolvability Circumstance, thereby reducing the performance of the convolutional neural network. This application solves the performance loss caused by the convolution operation with the convolution kernel as the center point in the original convolutional neural network. By comprehensively analyzing the side window convolution operation in multiple directions, it improves the convolution operation acquisition The ability of data to generalize features, thereby improving the performance of convolutional neural networks.

The embodiment of the application discloses a convolution processing method applied to a convolutional neural network, as shown in FIG. 1, including:

S11: Obtain a target operation object; the target operation object is specifically an input feature;

It can be understood that the convolution processing method is applicable to all input features in the convolutional neural network, where the input features include both the first layer input features of the initial input layer, such as image pixels, and also the hiding in the neural network. The input features involved in the layer and output layer, such as the fine-grained features of the bottom layer, and the semantic features of the high layer.

S12: Perform side window convolution calculation on the target operation object to obtain calculation results in multiple directions;

It is understandable that the definition of a side window is shown in Figure 2a, where θ is the angle between the window and the horizontal line, r is the radius of the window, ρ∈{0,r}, {x,y} is the target pixel The position of i, r is a user-defined parameter used to control all side windows. By changing the values of θ and {x, y}, the direction of the window and the corresponding target pixel i can be controlled. In order to simplify the calculation in continuous space, usually only 8 directional side windows in discrete space are calculated, let θ=k×π/2, k∈[0,3], when ρ=r, four up, down, left, and right can be obtained The side windows in the direction are represented by capital letters U (up), D (down), L (left), and R (right), as shown in Figure 2b and Figure 2c; when ρ = 0, you can get the upper left, The side windows in the upper right, lower left, and lower right directions are represented by the letters NW (northwest), NE (northeast), SE (southeast), and SW (southwest) respectively, as shown in Figure 2d. The convolution operation is calculated in each side window, and the output of 8 directions can be obtained as the calculation result of that direction.

S13: Perform cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object.

This step specifically includes: performing cross-entropy optimization processing on the calculation results in the multiple directions, and determining the calculation result with the smallest cross-entropy value as the convolution result of the target operation object.

It is understandable that, in this step, the cross-entropy optimization processing is performed on the calculation results in multiple directions, which can be replaced with L2 norm or other clustering methods to measure the final output to determine the final convolution result. Since the final convolution effect is obtained by comprehensively analyzing the convolution results in multiple directions, it can better reflect the target characteristics. In the continuous processing of the subsequent convolutional neural network, more generalized features can be obtained, thereby improving The learning ability of convolutional neural networks.

However, in the training process, because the convolutional neural network needs to obtain the first-order partial derivative based on the objective function or the loss function, the side window convolution calculation algorithm based on the L2 norm may cause the algorithm to converge slowly, for example, based on sigmoid Inactive function, when the input data value is too large or too small, the first-order partial derivative tends to zero. Therefore, in this application, the optimization process of cross entropy with obvious advantages in learning speed is selected, and based on the optimization of cross entropy, the calculation result corresponding to the smallest cross entropy value among the current processing points is selected.

Further, the target operation object is specifically a boundary input feature and/or a texture input feature. In addition, the target operation object in this embodiment may also be other types of input features that can perform convolution processing operations.

It is understandable that the above description is based on the convolution processing method when the number of convolution kernel filters is 1. When there are multiple convolution kernel filters, the convolution results corresponding to each convolution kernel are weighted and averaged to obtain the target The final convolution result of the operation object. Specifically, according to the data format nchw of caffe (Convolutional Architecture for Fast Feature Embedding), where n is batch data, c is the number of channels of channel input data, h is hight height, and w is width width. That is, assuming that the shape of the input data is [1,384,13,13], the convolution kernel size kernel_size of the convolution kernel is 3, the padding is 1, the convolution step stride is 1, and the number of convolution kernels num_out is 256. Since pad is 1, stride is 1, the shape of the output data obtained by the convolution operation is [1, 256, 13, 13]. When applying the convolution processing method of this embodiment, the input object is taken as the target operation object, and when the side window convolution calculation in 8 directions is performed on the input data, the calculation results in 8 directions are obtained, and then the optimization processing can be performed. Obtain the convolution result of the side window on the current point of the current channel (shape is [1,1,1,1]); since the number of channels of the input data is 384, on other channels of the current point, follow the above steps to calculate other Channel side window convolution result (shape is [1,384,1,1]); the current side window convolution results on all channels are weighted and averaged to obtain the output result of the current point (shape is [1,1,1, 1]); Next, according to the above 3 steps, calculate the side window convolution results of other points of the input data in the current filer (shape is [1,1,13,13]); finally, because the number of filters is 256, Calculate the convolution result of the side window in other filters (shape is [1,256,13,13]) to obtain the final convolution result of the entire input object.

This embodiment discloses a convolution processing method applied to a convolutional neural network, including: obtaining a target operation object; the target operation object is specifically an input feature; performing side window convolution calculation on the target operation object to obtain Calculation results in multiple directions; performing cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object. This embodiment solves the performance loss caused by the convolution operation with the convolution kernel as the center point in the original convolutional neural network. By comprehensively analyzing the side window convolution operation in multiple directions, the acquisition of the convolution operation is improved. The ability of data to generalize features, thereby improving the performance of convolutional neural networks.

Example two

The embodiment of the present application discloses a specific convolution processing method applied to a convolutional neural network. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically, see Figure 3:

S21: Acquire a target operation object; the target operation object is specifically an input feature;

S22: Determine the directions of the four calculation side windows as upper left, upper right, lower left, and lower right;

It is understandable that the selection of the side window needs to weigh the target characteristics and the amount of calculation. In the previous embodiment, the side window convolution calculation has a large number of repetitive calculations, and the speed may be greatly reduced. Based on this point, improvements are made. This embodiment Only the four directions in Figure 2d are selected in Figure 2d, which reduces the amount of calculation while ensuring the calculation effect of the side window convolution.

S23: Perform a side window convolution calculation on the target operation object by using the four calculation side windows to obtain calculation results in four directions.

Specifically, this step can be performed as follows:

It is understandable that, in order to make full use of the performance of the graphics card or CPU, adjust the operation sequence of multiple target operation objects, first calculate all the side window convolutions in the same direction, and then calculate the side window convolution in the other direction, so that you can Make full use of graphics card performance to improve calculation speed. Therefore, this embodiment does not limit the calculation order to upper left, upper right, lower left, and lower right. The calculation order is only an example. Calculations are performed in other calculation orders, as long as all side window convolution calculations are performed in the same direction. This embodiment has the effect of improving the calculation speed.

S24: Perform cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object.

Compared with the convolution operation of the convolutional neural network in the prior art, this embodiment replaces the previous convolution calculation centered on the convolution kernel with the side window convolution calculation in multiple directions, and optimizes the cross-entropy. The final convolution result is processed, and the local receptive field characteristics of the convolutional neural network are fully utilized, so that the extracted features can better reflect the characteristics of the target, and the generalization performance of the features is stronger. Furthermore, the convolution in this embodiment More generalized features can be extracted, so a relatively shallow and narrow network can be designed, which can improve the performance of the convolutional neural network and reduce the amount of model parameters.

Example three

Correspondingly, this application also discloses a convolution processing system applied to a convolutional neural network, as shown in FIG. 4, including:

The obtaining module 01 is used to obtain a target operation object; the target operation object is specifically an input feature;

The calculation module 02 is configured to perform side window convolution calculation on the target operation object to obtain calculation results in multiple directions;

The result determination module 03 is configured to perform cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object.

In some specific embodiments, the calculation module 02 is specifically configured to:

In some specific embodiments, the calculation module 02 is specifically used for:

A unified side window convolution calculation is performed on a plurality of the target operation objects by using the calculation side window in the lower right direction to obtain multiple calculation results in the lower right direction.

In some specific embodiments, the result determining module 03 is specifically configured to:

In some specific embodiments, the target operation object is specifically a boundary input feature and/or a texture input feature.

In some specific embodiments, the convolution processing system further includes: a weighted average module, configured to perform a weighted average on the convolution result corresponding to each convolution kernel when there are multiple convolution kernels to obtain The final convolution result of the target operation object.

The embodiment of the application solves the performance loss caused by the convolution operation with the convolution kernel as the center point in the original convolutional neural network. By comprehensively analyzing the side window convolution operation in multiple directions, the convolution operation is improved. The ability to obtain more generalized features of the data improves the performance of the convolutional neural network.

Example four

Correspondingly, this application also discloses a convolution processing device applied to a convolutional neural network. As shown in FIG. 5, it includes a processor 11 and a memory 12; wherein, the processing 11 executes the data stored in the memory 12 The computer program implements the following steps:

In some specific embodiments, when the processor 11 executes the computer subprogram stored in the memory 12, the following steps may be specifically implemented:

When there are multiple convolution kernels, the convolution result corresponding to each convolution kernel is weighted and averaged to obtain the final convolution result of the target operation object.

Further, the convolution processing device in this embodiment may further include:

The input interface 13 is used to obtain a computer program imported from the outside world, and save the obtained computer program in the memory 12, and can also be used to obtain various instructions and parameters transmitted by external terminal devices, and transmit them to the processor 11 , So that the processor 11 uses the above-mentioned various instructions and parameters to carry out corresponding processing. In this embodiment, the input interface 13 may specifically include, but is not limited to, a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, and the like.

The output interface 14 is used to output various data generated by the processor 11 to a terminal device connected to it, so that other terminal devices connected to the output interface 14 can obtain various data generated by the processor 11. In this embodiment, the output interface 14 may specifically include, but is not limited to, a USB interface, a serial interface, and the like.

The communication unit 15 is used to establish a remote communication connection between the convolution processing device and the external server, so that the convolution processing device can mount the image file to the external server. In this embodiment, the communication unit 15 may specifically include, but is not limited to, a remote communication unit based on wireless communication technology or wired communication technology.

The keyboard 16 is used to obtain various parameter data or instructions input by the user by hitting the keycap in real time.

The display 17 is used for real-time display of relevant information of the convolution processing process, so that the user can understand the current processing situation of the convolution neural network in time.

The mouse 18 can be used to assist the user in inputting data and simplify the user's operation.

Example five

Further, the embodiment of the present application also discloses a computer-readable storage medium. The computer-readable storage medium mentioned here includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, and Erase programmable ROM, register, hard disk, removable hard disk, CD-ROM or any other form of storage medium known in the technical field. A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the following steps are implemented:

In some specific embodiments, when the computer subprogram stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented:

Finally, it should be noted that those skilled in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage. Unit. The storage unit described in all the embodiments described in this application includes: read-only memory, random access memory, magnetic disk, or the like.

In this article, terms such as "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes no Other elements clearly listed, or they also include elements inherent to the process, method, article, or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or equipment that includes the element.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other.

The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, this application will not be limited to the embodiments shown in this document, but should conform to the widest scope consistent with the principles and novel features disclosed in this document.

Claims

A convolution processing method applied to a convolutional neural network, which is characterized in that it includes:

Acquiring a target operation object; the target operation object is specifically an input feature;

Perform side window convolution calculation on the target operation object to obtain calculation results in multiple directions;

Perform cross-entropy optimization processing on the calculation results in the multiple directions to obtain the convolution result of the target operation object.
The convolution processing method according to claim 1, wherein the process of performing side window convolution calculation on the target operation object to obtain calculation results in multiple directions specifically includes:

Determine the directions of the four computing side windows as upper left, upper right, lower left, and lower right;

The four calculation side windows are used to perform side window convolution calculations on the target operation object to obtain calculation results in four directions.
The convolution processing method according to claim 2, wherein when there are multiple target operation objects, the side window convolution calculation is performed on the target operation object by using the calculation side windows respectively, to obtain four The process of calculating the result of the direction includes:

Performing a unified side window convolution calculation on multiple target operation objects by using the calculation side window in the upper left direction to obtain multiple calculation results in the upper left direction;

Performing a unified side window convolution calculation on a plurality of the target operation objects by using the calculation side window in the upper right direction to obtain multiple calculation results in the upper right direction;

Performing a unified side window convolution calculation on multiple target operation objects by using the calculation side window in the lower left direction to obtain multiple calculation results in the lower left direction;

A unified side window convolution calculation is performed on a plurality of the target operation objects by using the calculation side window in the lower right direction to obtain a plurality of calculation results in the lower right direction.
The convolution processing method according to any one of claims 1 to 3, wherein the cross-entropy optimization processing is performed on the calculation results of the multiple directions to obtain the convolution result of the target operation object The process includes:

Perform cross-entropy optimization processing on the calculation results in the multiple directions, and determine the calculation result with the smallest cross-entropy value as the convolution result of the target operation object.
The convolution processing method according to claim 4, wherein the target operation object is specifically a boundary input feature and/or a texture input feature.
The convolution processing method according to claim 5, wherein when there are multiple convolution kernels, the convolution processing method further comprises:

A weighted average is performed on the convolution result corresponding to each convolution kernel to obtain the final convolution result of the target operation object.
A convolution processing system applied to a convolutional neural network, which is characterized in that it comprises:

The acquisition module is used to acquire a target operation object; the target operation object is specifically an input feature;

A calculation module, configured to perform side window convolution calculation on the target operation object to obtain calculation results in multiple directions;

The result determination module is configured to perform cross-entropy optimization processing on the calculation results of the multiple directions to obtain the convolution result of the target operation object.
The convolution processing system according to claim 7, wherein the calculation module is specifically configured to:

Determine the directions of the four computing side windows as upper left, upper right, lower left, and lower right;

The four calculation side windows are used to perform side window convolution calculations on the target operation object to obtain calculation results in four directions.
A convolution processing device applied to a convolutional neural network, which is characterized in that it comprises:

Memory, used to store computer programs;

The processor is configured to implement the steps of the convolution processing method applied to a convolutional neural network according to any one of claims 1 to 6 when the computer program is executed.
A readable storage medium, characterized in that a computer program is stored on the readable storage medium, and when the computer program is executed by a processor, it is applied to a convolutional neural network as claimed in any one of claims 1 to 6 The steps of the convolution processing method.