CN111445420B

CN111445420B - Image operation method and device of convolutional neural network and electronic equipment

Info

Publication number: CN111445420B
Application number: CN202010276826.6A
Authority: CN
Inventors: 王艳龙
Original assignee: Beijing Aixin Technology Co ltd
Current assignee: Beijing Aixin Technology Co ltd
Priority date: 2020-04-09
Filing date: 2020-04-09
Publication date: 2023-06-06
Anticipated expiration: 2040-04-09
Also published as: CN111445420A

Abstract

The invention provides an image operation method, an image operation device and electronic equipment of a convolutional neural network, which comprise the steps of obtaining operation parameters of a preset convolutional neural network; the operational parameter is associated with a storage attribute of the electronic device; collecting image blocks from the image to be processed according to the operation parameters; for each image block, calculating a pooling feature matrix of the image block through the convolutional neural network; and forming a pooling feature matrix of the image to be processed according to the pooling feature matrix of each image block. According to the invention, the scale of the feature matrix output by the pooling layer is deduced according to the internal cache of the electronic equipment, and further, the window scale parameter matched with the network parameter of the convolutional neural network is set, so that the image block acquired according to the window scale parameter does not have redundant data in the convolutional feature matrix obtained by convolution operation, thus the pooling calculation can be directly carried out on the convolution result output by the convolutional layer, the caching to DDR is not needed, and the memory resource and the network broadband are saved.

Description

Image operation method and device of convolutional neural network and electronic equipment

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image computing method and apparatus for a convolutional neural network, and an electronic device.

Background

The convolutional neural network comprises an input layer, a hidden layer and an output layer; wherein the hidden layer comprises a convolution layer and a pooling layer, which is also called a sampling layer. In the related art, a convolutional neural network accelerator generally collects image blocks by adopting a window with a fixed scale, then carries out convolutional calculation on the collected image blocks through a convolutional layer, caches the convolutional result to a DDR (Double Data Rate Synchronous Dynamic Random Access Memory, double-rate synchronous dynamic random access memory), acquires the convolutional result from the DDR, carries out pooling calculation through a pooling layer, and outputs the pooling result. In the mode, because the acquisition window scale of the image block is fixed, if the acquisition window scale is too large relative to convolution calculation of a certain convolution layer, the acquired image block is larger, and redundant data exists; moreover, the convolution results of the convolution layer output may also have redundant data with respect to the subsequent pooling layer. Therefore, the problem of redundant data needs to be solved by adopting the method of caching the convolution result to the DDR and then reading the convolution result from the DDR. However, accessing the convolution results in DDR wastes a lot of memory resources and network bandwidth.

Overall, the existing convolutional neural network operation mode can cause a great deal of waste of memory resources and network bandwidth.

Disclosure of Invention

Therefore, the invention aims to provide an image operation method, an image operation device and electronic equipment of a convolutional neural network, which can reduce the occupation of memory resources and save network broadband.

In a first aspect, an embodiment of the present invention provides an image operation method of a convolutional neural network, where the method is applied to an electronic device that implements convolutional neural network operation; the method comprises the following steps: acquiring operation parameters of a preset convolutional neural network; the operational parameter is associated with a storage attribute of the electronic device; collecting image blocks from the image to be processed according to the operation parameters; for each image block, the following processing is performed: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block; and according to the position of each image block in the image to be processed, forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed.

In a preferred embodiment of the present invention, the step of obtaining the operation parameters of the preset convolutional neural network includes: acquiring hardware parameters of the electronic equipment; and determining the operation parameters of the preset convolutional neural network according to the hardware parameters.

In a preferred embodiment of the present invention, the step of determining the operation parameters of the preset convolutional neural network according to the hardware parameters includes: determining constraint conditions of a preset convolutional neural network according to the hardware parameters; wherein the hardware parameter includes an internal cache size of the electronic device; the constraint includes: the maximum scale of a convolution feature matrix output by a convolution layer in the convolution neural network or the maximum scale of an image block input by the convolution layer; and determining the operation parameters of the convolutional neural network according to the constraint conditions.

In a preferred embodiment of the present invention, the step of determining the operation parameter of the convolutional neural network according to the constraint condition includes: if the constraint condition comprises the maximum scale of the convolution feature matrix output by the convolution layer, determining the maximum scale of the image block input by the convolution layer according to the maximum scale of the convolution feature matrix; determining the scale of a pooling feature matrix output by a pooling layer in the convolutional neural network according to the maximum scale of an image block input by the convolutional layer; and determining the operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer.

In a preferred embodiment of the present invention, the step of determining the maximum scale of the image block input by the convolution layer according to the maximum scale of the convolution feature matrix includes: the maximum scale of the image block input by the convolution layer is determined by the following formula: i_ cov _width_valid_max= (o_ cov _width_max-1) cs_w+1; i_ cov _height_valid_max= (o_ cov _height_max-1) ×cs_h+1; wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; o_ cov _width_max is the maximum width of the convolution feature matrix output by the convolution layer; o_ cov _height_max is the maximum height of the convolution feature matrix output by the convolution layer; cs_w is the horizontal step size of the convolution layer; cw_h is the vertical step of the convolutional layer.

In a preferred embodiment of the present invention, the step of determining the scale of the pooling feature matrix output by the pooling layer in the convolutional neural network according to the maximum scale of the image block input by the convolutional layer includes: the scale of the pooling feature matrix output by the pooling layer in the convolutional neural network is determined by the following formula: (pw_w+ (pl_width-1) ps_w-1) cs_w+1=i_ cov _width_valid_max; (pw_h+ (pl_height-1) ps_h-1) cs_h+1=i_ cov _height_valid max; wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; cs_w is the horizontal step size of the convolution layer; cs_h is the vertical step length of the convolution layer; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; pl_width is the width of the pooling feature matrix output by the pooling layer; pl_height is the height of the pooling feature matrix output by the pooling layer; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer.

In a preferred embodiment of the present invention, the step of determining the operation parameter of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer includes: the dimension of the pooling feature matrix output by the pooling layer is rounded; and determining the operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer after rounding.

In a preferred embodiment of the present invention, the step of determining the operation parameters of the convolutional neural network according to the dimensions of the pooled feature matrix output by the pooled layer after rounding includes: the window scale for collecting image blocks from the image to be processed in the operation parameter is calculated by the following formula: i_ cov _width= (pw_w+ (o_pl_width-1) ×ps_w-1) ×cs_w+cw_w; i_ cov _height= (pw_h+ (o_pl_height-1) ps_h-1) cs_h+cw_h; where i_ cov _width is the width of the window; i_ cov _height is the height of the window; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer; o_pl_width is the width of the pooling feature matrix output by the pooling layer after rounding; o_pl_height is the height of the pooling feature matrix output by the pooling layer after rounding; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; cs_w is the horizontal step size of the convolution layer; cs_h is the vertical step size of the convolutional layer.

In a preferred embodiment of the present invention, the step of determining the operation parameters of the convolutional neural network according to the dimensions of the pooled feature matrix output by the pooled layer after rounding includes: the window displacement for collecting image blocks from the image to be processed in the operation parameters is calculated by the following formula: cb_ws= ((pw_w+ (o_pl_width-1) ×ps_w-1) ×cs_w+1- (pw_w-ps_w)). Cs_w; cb_hs= ((pw_h+ (o_pl_height-1) ps_h-1) cs_h+1- (pw_h-ps_h))cs_h; wherein cb_ws is the horizontal displacement step length of the window; the cb_hs is the vertical displacement step of the window.

In a preferred embodiment of the present invention, the above-mentioned operation parameters include: the step for collecting the image blocks from the image to be processed according to the operation parameters comprises the following steps: determining a first image block from the image to be processed according to the window scale parameter in the operation parameter and a preset default initial position; traversing the image to be processed according to the window scale parameter and the window displacement parameter and a preset sequence to obtain image blocks except the first image block.

In a preferred embodiment of the present invention, the step of forming the pooled feature matrix of each image block into the pooled feature matrix of the image to be processed according to the position of each image block in the image to be processed includes: and according to the position of each image block in the image to be processed, performing splicing processing on the pooling characteristic matrixes of the image blocks adjacent to each other in position to obtain the pooling characteristic matrix of the image to be processed.

In a second aspect, the embodiment of the invention further provides an image computing device of the convolutional neural network, which is applied to electronic equipment for realizing convolutional neural network operation; the device comprises: the operation parameter acquisition module is used for acquiring operation parameters of a preset convolutional neural network; the operational parameter is associated with a storage attribute of the electronic device; the image block acquisition module is used for acquiring image blocks from the image to be processed according to the operation parameters; the image block pooling feature matrix calculation module is used for carrying out the following processing on each image block: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block; and the image pooling feature matrix calculation module is used for forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed according to the position of each image block in the image to be processed.

In a fourth aspect, the embodiment of the invention further provides an electronic device for realizing convolutional neural network operation, where the electronic device includes a processing device and a storage device; the storage device has stored thereon a computer program which, when executed by the processing device, performs the steps of the image calculation method of the convolutional neural network described above.

In a fifth aspect, an embodiment of the present invention further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processing device, performs the steps of the image operation method of a convolutional neural network described above.

The embodiment of the invention has the following beneficial effects:

the embodiment of the invention provides an image operation method, an image operation device and electronic equipment of a convolutional neural network, wherein operation parameters of a preset convolutional neural network are firstly obtained; the operational parameter is associated with a storage attribute of the electronic device; then, according to the operation parameters, collecting image blocks from the image to be processed; for each image block, the following processing is performed: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block; and forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed according to the position of each image block in the image to be processed. In this mode, the following principle is adopted: under the condition that the convolution kernel parameters and the pooling window parameters of the convolution neural network are determined, if the size of an image block input into the convolution layer is certain, the scale of the convolution feature matrix output by the convolution layer and the scale of the pooling feature matrix output by the pooling layer are also determined; therefore, under the condition that the convolution kernel parameters and the pooling window parameters of the convolution neural network are determined, the scale of the feature matrix output by the pooling layer can be deduced according to the storage attribute of the electronic equipment, and then the proper window scale parameters (namely, the operation parameters of the convolution neural network are deduced in advance based on the association relation of the window scale parameters) are set, so that the size of an acquired image block is matched with the convolution kernel parameters and the pooling window parameters of the convolution neural network, redundant data does not appear in the convolution feature matrix obtained after the convolution operation of the image block, the convolution result output by the convolution layer can be directly subjected to pooling calculation without caching the convolution result to DDR, the pipeline design from the convolution operation to pooling operation is realized, the occupation of memory resources can be reduced, and the network broadband is saved.

In addition, the operation method can be suitable for the neural network with different convolution kernel parameters and pooling window parameters, so that the method has good universality.

Additional features and advantages of the disclosure will be set forth in the description which follows, or in part will be obvious from the description, or may be learned by practice of the techniques of the disclosure.

The foregoing objects, features and advantages of the disclosure will be more readily apparent from the following detailed description of the preferred embodiments taken in conjunction with the accompanying drawings.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of an image operation method of a convolutional neural network according to an embodiment of the present invention;

FIG. 3 is a schematic view of window displacement of an image block acquired in convolutional neural network image operation according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of an image acquisition block according to an embodiment of the present invention;

fig. 5 is a flowchart of another image operation method of a convolutional neural network according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a convolutional neural network operation process for pipeline design according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an image computing device of a convolutional neural network according to an embodiment of the present invention.

Icon: 100-an electronic device; 102-a processing device; 104-a storage device; 106-an input device; 108-an output device; 110-an image acquisition device; 112-bus system; 71-an operation parameter acquisition module; 72-an image block acquisition module; 73-an image block pooling feature matrix calculation module; 74-image pooling feature matrix calculation module.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Considering the problem that the existing convolutional neural network operation mode can cause waste of a large amount of memory resources and network bandwidth, the image operation device and electronic equipment of the convolutional neural network provided by the embodiment of the invention can be applied to hardware design for realizing the convolutional neural network and various application scenes related to the convolutional neural network operation. For the sake of understanding the present embodiment, first, an image operation method of a convolutional neural network disclosed in the present embodiment is described in detail.

Here, an example electronic apparatus, an image operation apparatus, and an electronic apparatus for realizing a convolutional neural network of an embodiment of the present invention are described with reference to fig. 1.

As shown in fig. 1, an electronic device 100 includes one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and one or more image capture devices 110, which are interconnected by a bus system 112 and/or other forms of connection mechanisms (not shown). It should be noted that the components and structures of the electronic device shown in fig. 1 are exemplary only and not limiting, as the electronic device may have other components and structures as desired.

The processing device 102 may be a smart terminal or a device that includes a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, may process data from other components in the electronic device, and may also control other components in the electronic device to perform targeted object statistics functions.

The storage 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, random Access Memory (RAM) and/or cache memory (cache) and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer readable storage medium and the processing device 102 may execute the program instructions to implement client functions and/or other desired functions in embodiments of the present invention described below (implemented by the processing device). Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer readable storage medium.

The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.

The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, and the like.

The image capture device 110 may capture training pictures and store the captured preview video frames or image data in the storage 104 for use by other components.

For example, the image calculation, the apparatus and the devices in the electronic apparatus for implementing the convolutional neural network according to the embodiment of the present invention may be integrally disposed, or may be disposed in a scattered manner, such as integrally disposing the processing apparatus 102, the storage apparatus 104, the input apparatus 106 and the output apparatus 108 into a single body, and disposing the image capturing apparatus 110 at a designated position where a picture can be captured. When the devices in the above electronic apparatus are integrally provided, the electronic apparatus may be implemented as an intelligent terminal such as a camera, a smart phone, a tablet computer, a vehicle-mounted terminal, or the like.

Fig. 2 is a schematic flow chart of an image operation method of a convolutional neural network, which is applied to an electronic device for implementing convolutional neural network operation and is shown in fig. 2, and the method includes the following steps:

Step S202: acquiring operation parameters of a preset convolutional neural network; the operational parameter is associated with a storage attribute of the electronic device.

Here, the electronic device for realizing the convolutional neural network operation may be an FPGA (Field Programmable Gate Array ) chip, an Asic (Application Specific Integrated Circuit, application specific integrated circuit) chip, or the like.

The convolutional neural network is constructed according to preset operation parameters, the operation parameters are associated with storage attributes of the electronic equipment, and the storage attributes comprise a cache in a hardware circuit. In one possible implementation, the operation parameters include a window scale parameter and a window displacement parameter for acquiring an image block from the image to be processed. In actual operation, image block acquisition is performed according to window scale parameters, which typically include the width and height of the window. The window displacement parameter is a movement parameter of the acquisition window when acquiring an image, and generally comprises a displacement direction and a displacement step. Here, referring to fig. 3, a schematic diagram of window displacement of an acquired image block in convolutional neural network image operation is shown, and in the embodiment shown in fig. 3, the situation of horizontal displacement and vertical displacement of the acquired window is shown.

In the embodiment of the invention, the window scale parameter is associated with the scale of the feature matrix output by the pooling layer in the convolutional neural network, wherein the convolutional neural network comprises an input layer, a hidden layer and an output layer, and the hidden layer comprises the convolutional layer and the pooling layer. In actual operation, firstly sampling an image to be processed according to the window scale parameter to obtain an image block, then inputting the image block into a convolution layer to carry out convolution operation to obtain a convolution feature matrix, inputting the convolution feature into a pooling layer, and outputting the pooling feature matrix of the image block. Here, since in the case where the convolution kernel parameter and the pooling window parameter are determined, if the size of the input image block is fixed, the scale of the convolution feature matrix output by the convolution layer is also determined, and the scale of the pooling feature matrix output by the pooling layer is also determined; therefore, the method can be used for carrying out reverse calculation, when the convolution kernel parameters and the pooling window parameters of the convolution neural network are determined, the scale of the feature matrix output by the pooling layer can be deduced according to the storage attribute (such as RAM cache) of the electronic equipment, and further, the proper window scale parameters are set, so that the size of the acquired image block is matched with the convolution kernel parameters and the pooling window parameters of the convolution neural network, and for example, redundant data of the image block with the proper size can not appear in the convolution feature matrix obtained after convolution operation.

In this way, the pooled computation can be directly carried out on the convolution feature matrix after the convolution operation to obtain the pooled feature matrix of the image block, and compared with the existing technology for obtaining the pooled feature matrix of the image block, the method does not need to buffer the convolution feature matrix to DDR, and can realize the pipeline design from the convolution operation to the pooled operation.

It should be noted that, in actual implementation, the window scale parameter in the above operation parameters is not only associated with the scale of the feature matrix output by the pooling layer in the convolutional neural network, but also associated with parameters of each layer of the convolutional neural network, such as the convolutional layer convolutional kernel size, step size, pooling layer window, step size, and the like, and meanwhile, the window scale parameter is also associated with the buffer capacity of the system.

Step S204: and acquiring image blocks from the image to be processed according to the operation parameters.

In practice, the step of capturing image blocks from the image to be processed may be achieved by the following steps 21-22:

(21) And determining a first image block from the image to be processed according to the window scale parameter in the operation parameter and a preset default initial position.

The initial position may be any position in the image to be processed, taking the position of the upper left corner of the acquisition window as an example, and the initial position may be set as the position of the pixel point where the upper left corner of the image to be processed is located.

(22) Traversing the image to be processed according to the window scale parameter and the window displacement parameter and a preset sequence to obtain image blocks except the first image block.

Here, image block acquisition is performed through the window scale parameters, so that an image block with a specific size is obtained; and the acquisition window carries out displacement according to the window displacement parameters, wherein a preset sequence is a window displacement direction route, the image to be processed is traversed, and the image blocks are sequentially acquired until the last image block in the image to be processed is acquired.

Referring to fig. 4, a schematic diagram of an image block is shown, in the embodiment shown in fig. 4, the window scale parameter includes a window width and a window height, and the window displacement parameter includes a horizontal displacement step and a vertical displacement step, where the order of acquiring the window displacement is sequentially horizontally moved from the upper left corner of the image to the right edge of the image, and then, according to the vertical displacement step, the image block is continuously horizontally moved and acquired.

Step S206: for each image block, the following processing is performed: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; and carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain the pooling feature matrix of the current image block.

In this embodiment, after each image block is subjected to convolution calculation to obtain a convolution feature matrix, the convolution feature matrix is directly subjected to pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain the pooled feature matrix corresponding to the image block.

Step S208: and according to the position of each image block in the image to be processed, forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed.

In one possible implementation manner, the pooling feature matrix of the image blocks adjacent to each other can be spliced according to the position of each image block in the image to be processed, so as to obtain the pooling feature matrix of the image to be processed.

The embodiment of the invention provides an image operation method of a convolutional neural network, which comprises the steps of firstly obtaining operation parameters of a preset convolutional neural network; the operational parameter is associated with a storage attribute of the electronic device; then, according to the operation parameters, collecting image blocks from the image to be processed; for each image block, the following processing is performed: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block; and forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed according to the position of each image block in the image to be processed. In this mode, the following principle is adopted: under the condition that the convolution kernel parameters and the pooling window parameters of the convolution neural network are determined, if the size of an image block input into the convolution layer is certain, the scale of the convolution feature matrix output by the convolution layer and the scale of the pooling feature matrix output by the pooling layer are also determined; therefore, under the condition that the convolution kernel parameters and the pooling window parameters of the convolution neural network are determined, the scale of the feature matrix output by the pooling layer can be deduced according to the storage attribute of the electronic equipment, and then the proper window scale parameters (namely the operation parameters of the convolution neural network are deduced in advance based on the association relation of the window scale parameters) are set, so that the size of an acquired image block is matched with the convolution kernel parameters and the pooling window parameters of the convolution neural network, redundant data does not appear in the convolution feature matrix obtained after the convolution operation of the image block, the convolution result output by the convolution layer can be directly subjected to pooling calculation without caching the convolution result to DDR, the pipeline design from the convolution operation to pooling operation is realized, the occupation of memory resources can be reduced, and the network broadband is saved.

On the basis of the image operation method of the convolutional neural network shown in fig. 2, the embodiment also provides another image operation method of the convolutional neural network, which is applied to an electronic device for implementing convolutional neural network operation, where the method mainly describes the specific implementation process of step S202 (i.e. obtaining the preset operation parameters of the convolutional neural network) in the foregoing embodiment. As shown in fig. 5, which is a flow chart of an operation method of the convolutional neural network, as can be seen from fig. 5, the method includes the following steps:

step S502: and acquiring hardware parameters of the electronic equipment.

Here, the electronic device is an electronic device for implementing the operation of the convolutional neural network, that is, an electronic device for running the operation method of the convolutional neural network, where the electronic device may be a CNN hardware accelerator. The hardware parameters typically include, among other things, the type of processor used by the device, the dominant frequency, and the memory capacity.

Step S504: determining operation parameters of a preset convolutional neural network according to the hardware parameters; the operational parameter is associated with a storage attribute of the electronic device.

In one possible implementation manner, the operation parameters of the preset convolutional neural network may be determined through the following steps 31-32:

(31) Determining constraint conditions of a preset convolutional neural network according to the hardware parameters; wherein the hardware parameter includes an internal cache size of the electronic device; the constraint includes: the convolution neural network outputs the maximum scale of the convolution feature matrix or the maximum scale of the image block input by the convolution layer.

Here, for a preset convolutional neural network, that is, if the convolutional kernel parameter and the pooling window parameter of the convolutional neural network are determined, if the internal buffer size of the electronic device running the convolutional neural network is known, the maximum scale of the convolutional feature matrix output by the convolutional layer in the convolutional neural network can be correspondingly determined, and for the convolutional layer of the convolutional neural network, the scale of the image block input by the convolutional layer can be correspondingly obtained based on the scale of the convolutional feature matrix output by the convolutional layer, so that the maximum scale of the image block input by the convolutional layer is obtained.

(32) And determining the operation parameters of the convolutional neural network according to the constraint conditions.

In at least one possible implementation, if the constraint includes a maximum dimension of the convolution feature matrix output by the convolution layer, the step of determining the operational parameters of the convolution neural network according to the constraint may be implemented by the following steps 41-44:

(41) And determining the maximum scale of the image block input by the convolution layer according to the maximum scale of the convolution feature matrix.

In this embodiment, the maximum scale of the image block input by the convolution layer is determined by the following formula:

i_ cov _width_valid_max= (o_ cov _width_max-1) ×cs_w+1 (formula 1.1)

i_ cov _height_valid_max= (o_ cov _height_max-1) ×cs_h+1 (formula 1.2)

Wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; o_ cov _width_max is the maximum width of the convolution feature matrix output by the convolution layer; o_ cov _height_max is the maximum height of the convolution feature matrix output by the convolution layer; cs_w is the horizontal step size of the convolution layer; cw_h is the vertical step of the convolutional layer.

(42) And determining the scale of the pooling feature matrix output by the pooling layer in the convolutional neural network according to the maximum scale of the image block input by the convolutional layer.

In this embodiment, the scale of the pooling feature matrix output by the pooling layer in the convolutional neural network is determined by the following formula:

(pw_w+ (pl_width-1) ps_w-1) cs_w+1=i_ cov _width_valid_max (formula 1.3)

(pw_h+ (pl_height-1) ps_h-1) cs_h+1=i_ cov _height_valid max (equation 1.4)

Wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; cs_w is the horizontal step size of the convolution layer; cs_h is the vertical step length of the convolution layer; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; pl_width is the width of the pooling feature matrix output by the pooling layer; pl_height is the height of the pooling feature matrix output by the pooling layer; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer.

(43) And determining the operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer.

In at least one possible implementation manner, the dimension of the pooling feature matrix output by the pooling layer may be rounded first; and then, determining the operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer after rounding.

When the scale of the pooling feature matrix is rounded, the rounding can be performed by the following formula:

o_pl_width=floor (pl_width) (formula 1.5)

o_pl_height=floor (pl_height) (formula 1.6)

In the above description, o_pl_width is the width of the pooling feature matrix output by the pooling layer after rounding; o_pl_height is the height of the pooling feature matrix output by the pooling layer after rounding; floor represents rounding. In practice, it is typically rounded downwards, for example, if the calculated scale is 5.3, then it is rounded to 5.

Here, substituting the above equations 1.5 and 1.6 into the above equations 1.3 and 1.4, the following equations can be obtained:

i_ cov _width_valid= (pw_w+ (o_pl_width-1) ×ps_w-1) ×cs_w+1 (formula 1.7)

i_ cov _height_valid= (pw_h+ (o_pl_height-1) ×ps_h-1) ×cs_h+1 (formula 1.8)

In actual operation, since the size of the image block input by the convolution layer is related to the size of the convolution kernel, in consideration of padding, after the scale of the pooled feature matrix output by the pooled layer after rounding is obtained, the window scale of the operation parameter for collecting the image block from the image to be processed can be calculated by the following formula:

i_ cov _width= (pw_w+ (o_pl_width-1) ×ps_w-1) ×cs_w+cw_w (formula 1.9)

i_ cov _height= (pw_h+ (o_pl_height-1) ps_h-1) cs_h+cw_h (formula 1.10), wherein i_ cov _width is the width of the window; i_ cov _height is the height of the window; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer; o_pl_width is the width of the pooling feature matrix output by the pooling layer after rounding; o_pl_height is the height of the pooling feature matrix output by the pooling layer after rounding; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; cs_w is the horizontal step size of the convolution layer; cs_h is the vertical step size of the convolutional layer.

Here, the window scale of the acquired image block is that of the image block input into the convolution layer, and accordingly, the convolution feature matrix scale output by the convolution layer can be obtained as follows:

o_ cov _width= (i_ cov _width_valid-1)/cs_w+1 (equation 1.11)

o_ cov _height= (i_ cov _height_valid-1)/cs_h+1 (equation 1.12)

Wherein o_ cov _width is the width of the convolution feature matrix; o_ cov _height is the height of the convolution feature matrix.

Also, in one possible implementation manner, the window displacement for acquiring the image block from the image to be processed in the operation parameter can be calculated by the following formula:

cb_ws= ((pw_w+ (o_pl_width-1) ×ps_w-1) ×cs_w+1- (pw_w-ps_w))

cb_hs= ((pw_h+ (o_pl_height-1) ×ps_h-1) ×cs_h+1- (pw_h-ps_h)). Cs_h (formula 1.14)

Wherein cb_ws is the horizontal displacement step length of the window; the cb_hs is the vertical displacement step of the window.

As can be seen from the above equations 1.13 and 1.14, the window displacement of the image block is related to the pooling parameter in addition to the convolution parameter and the size of the first image block, and when pw_w is equal to ps_w and pw_h is equal to ps_h, the window displacement is related to only the convolution parameter and the size of the first image block, and from the above two equations, adjacent image blocks of the pooled output can be obtained without any overlap. As shown in FIG. 6, a schematic diagram of a convolutional neural network operation process of a pipeline design is shown, and as can be seen from FIG. 6, the four image blocks in the figure are subjected to convolution and pooling operation, so that seamless butt joint of the image blocks is completed, and the pipeline design from convolution to pooling is truly realized.

Thus, the window scale parameter and the window displacement parameter of the convolutional neural network are determined according to the scale of the pooling feature matrix output by the pooling layer, and further image block acquisition can be performed according to the operation parameter. And the operation parameters of the convolutional neural network obtained by calculation according to the steps are matched with the hardware configuration of the electronic equipment running the convolutional neural network, so that the image block acquired according to the operation parameters does not have redundant data in the convolutional feature matrix obtained after the convolution operation, and the convolution result output by the convolution layer can be directly subjected to pooling calculation without caching the convolution result to the DDR so as to solve the problem of redundant data.

Step S506: and acquiring image blocks from the image to be processed according to the operation parameters.

Step S508: for each image block, the following processing is performed: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; and carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain the pooling feature matrix of the current image block.

Step S510: and according to the position of each image block in the image to be processed, forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed.

Here, in the steps S506 to S510 in the present embodiment, corresponding to the steps S204 to S208 in the above embodiment, the description of the corresponding content may refer to the corresponding parts of the above embodiment, and will not be repeated here.

In order to more clearly understand the process of acquiring the operation parameters of the convolutional neural network in the image operation method of the convolutional neural network provided in the present embodiment, an application example is described below.

Assuming that the constraint conditions of the convolutional neural network are determined according to the internal RAM buffer size of a certain electronic device as shown in the following tables 1 and 2:

TABLE 1 image block width hardware constraints

cs_w	I_cov_width_valid_max
		1	32
2	63
		3	64

Table 2 image block height hardware constraints

cs_h	i_cov_height_valid_max
		1	16
2	31
		3	46

And, certain layer network characteristics in the convolutional neural network are as follows: pw_w=5, ps_w=4, cs_w=2, cw_w=3; pw_h=5, ps_h=4, cs_h=2, cw_h=3.

From cs_w=2, table look-up: i_ cov _width_valid_max=63.

Bringing the value of i_ cov _width_valid_max into the above equation (1.3) yields:

(5+ (pl_w-1) ×4-1) ×2+1=63, and pl_width=7.75.

And then from the formula (formula 1.5): o_pl_width=7.

Bringing the value of o_pl_width (equation 1.7) into i_ cov _width_valid=57, it can be seen that this value is already closest to the ideal value i_ cov _width_valid_max.

Further, i_ cov _width=59 is calculated from (formula 1.9);

bringing the value of i_ cov _width into (equation 1.11) yields: o_ cov _width=29.

Therefore, the block width (i.e., window width) i_ cov _width of the input convolution layer is 59, the width o_ cov _width of the convolution feature matrix of the output convolution layer is 29, and the width o_pl_w of the pooling feature matrix of the output pooling layer is 7. Similarly, the image block height (i.e., window width) i_ cov _height of the output convolution layer can be found to be 27, the height o_ cov _height of the convolution feature matrix of the output convolution layer to be 13, and the height o_pl_height of the pooling feature matrix of the output pooling layer to be 3.

Therefore, the operation parameters of the preset convolutional neural network are determined according to the hardware parameters of the electronic equipment. In actual operation, according to the network parameters of the convolutional neural network on the electronic device and the requirement of the hardware design and storage resource size, according to the above formulas 1.1 to 1.14, each window information (convolutional input, convolutional output, pooled output) and window shift information can be calculated. Because the window parameters are irrelevant to the channel number of the convolutional neural network, the operation parameters can be configured to registers in a design module through CPU in the hardware design, and the pipeline design from convolution to pooling can be realized by combining certain control logic. Here, the operation method can be suitable for the neural network with different convolution kernel parameters and pooling window parameters, so that the method has good universality.

According to the image operation method of the convolutional neural network, constraint conditions of the convolutional neural network are determined according to hardware parameters of electronic equipment for realizing the convolutional neural network operation, operation parameters of the convolutional neural network are further determined according to the constraint conditions, and image blocks are acquired based on the operation parameters, so that redundant data do not appear in a convolutional feature matrix obtained after the convolutional operation of the image blocks, and therefore a convolutional result output by a convolutional layer can be directly subjected to pooling calculation without caching the convolutional result to DDR, pipeline design from the convolutional operation to pooling operation is realized, occupation of memory resources can be reduced, further the CNN hardware realization area is reduced, and network broadband is saved.

Corresponding to the image computing method of the convolutional neural network shown in fig. 2, the embodiment of the invention further provides an image computing device of the convolutional neural network, which is applied to an electronic device for implementing convolutional neural network operation, referring to fig. 7, a schematic structural diagram of the image computing device of the convolutional neural network is shown, and as can be seen from fig. 7, the device includes an operation parameter acquisition module 71, an image block acquisition module 72, an image block pooling feature matrix calculation module 73 and an image pooling feature matrix calculation module 74 which are sequentially connected, wherein the functions of each module are as follows:

An operation parameter obtaining module 71, configured to obtain an operation parameter of a preset convolutional neural network; the operational parameter is associated with a storage attribute of the electronic device;

an image block acquisition module 72, configured to acquire an image block from the image to be processed according to the operation parameter;

the image block pooling feature matrix calculating module 73 is configured to perform, for each image block, the following processing: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block;

an image pooling feature matrix calculation module 74, configured to form a pooling feature matrix of each image block into a pooling feature matrix of the image to be processed according to a position of each image block in the image to be processed.

The embodiment of the invention provides an image operation device of a convolutional neural network, which comprises the steps of firstly acquiring operation parameters of a preset convolutional neural network; the operational parameter is associated with a storage attribute of the electronic device; then, according to the operation parameters, collecting image blocks from the image to be processed; for each image block, the following processing is performed: performing convolution calculation on the current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; carrying out pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block; and forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed according to the position of each image block in the image to be processed. In the device, the following principle is adopted: under the condition that the convolution kernel parameters and the pooling window parameters of the convolution neural network are determined, if the size of an image block input into the convolution layer is certain, the scale of the convolution feature matrix output by the convolution layer and the scale of the pooling feature matrix output by the pooling layer are also determined; therefore, under the condition that the convolution kernel parameters and the pooling window parameters of the convolution neural network are determined, the scale of the feature matrix output by the pooling layer can be deduced according to the storage attribute of the electronic equipment, and then the proper window scale parameters (namely the operation parameters of the convolution neural network are deduced in advance based on the association relation of the window scale parameters) are set, so that the size of an acquired image block is matched with the convolution kernel parameters and the pooling window parameters of the convolution neural network, redundant data does not appear in the convolution feature matrix obtained after the convolution operation of the image block, the convolution result output by the convolution layer can be directly subjected to pooling calculation without caching the convolution result to DDR, the pipeline design from the convolution operation to pooling operation is realized, the occupation of memory resources can be reduced, and the network broadband is saved.

In one possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: acquiring hardware parameters of the electronic equipment; and determining the operation parameters of the preset convolutional neural network according to the hardware parameters.

In another possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: determining constraint conditions of a preset convolutional neural network according to the hardware parameters; wherein the hardware parameter includes an internal cache size of the electronic device; the constraint includes: the maximum scale of a convolution feature matrix output by a convolution layer in the convolution neural network or the maximum scale of an image block input by the convolution layer; and determining the operation parameters of the convolutional neural network according to the constraint conditions.

In another possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: if the constraint condition comprises the maximum scale of the convolution feature matrix output by the convolution layer, determining the maximum scale of the image block input by the convolution layer according to the maximum scale of the convolution feature matrix; determining the scale of a pooling feature matrix output by a pooling layer in the convolutional neural network according to the maximum scale of an image block input by the convolutional layer; and determining the operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer.

In another possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: the maximum scale of the image block input by the convolution layer is determined by the following formula: i_ cov _width_valid_max= (o_ cov _width_max-1) cs_w+1; i_ cov _height_valid_max= (o_ cov _height_max-1) ×cs_h+1; wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; o_ cov _width_max is the maximum width of the convolution feature matrix output by the convolution layer; o_ cov _height_max is the maximum height of the convolution feature matrix output by the convolution layer; cs_w is the horizontal step size of the convolution layer; cw_h is the vertical step of the convolutional layer.

In another possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: the scale of the pooling feature matrix output by the pooling layer in the convolutional neural network is determined by the following formula: (pw_w+ (pl_width-1) ps_w-1) cs_w+1=i_ cov _width_valid_max; (pw_h+ (pl_height-1) ps_h-1) cs_h+1=i_ cov _height_valid max; wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; cs_w is the horizontal step size of the convolution layer; cs_h is the vertical step length of the convolution layer; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; pl_width is the width of the pooling feature matrix output by the pooling layer; pl_height is the height of the pooling feature matrix output by the pooling layer; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer.

In another possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: the dimension of the pooling feature matrix output by the pooling layer is rounded; and determining the operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer after rounding.

In another possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: the window scale for collecting image blocks from the image to be processed in the operation parameter is calculated by the following formula: i_ cov _width= (pw_w+ (o_pl_width-1) ×ps_w-1) ×cs_w+cw_w; i_ cov _height= (pw_h+ (o_pl_height-1) ps_h-1) cs_h+cw_h; where i_ cov _width is the width of the window; i_ cov _height is the height of the window; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer; o_pl_width is the width of the pooling feature matrix output by the pooling layer after rounding; o_pl_height is the height of the pooling feature matrix output by the pooling layer after rounding; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; cs_w is the horizontal step size of the convolution layer; cs_h is the vertical step size of the convolutional layer.

In another possible implementation manner, the above-mentioned operation parameter obtaining module 71 is further configured to: the window displacement for collecting image blocks from the image to be processed in the operation parameters is calculated by the following formula: cb_ws= ((pw_w+ (o_pl_width-1) ×ps_w-1) ×cs_w+1- (pw_w-ps_w)). Cs_w; cb_hs= ((pw_h+ (o_pl_height-1) ps_h-1) cs_h+1- (pw_h-ps_h))cs_h; wherein cb_ws is the horizontal displacement step length of the window; the cb_hs is the vertical displacement step of the window.

In another possible implementation manner, the above operation parameters include: the above-mentioned image block acquisition module 72 is further configured to acquire a window scale parameter and a window displacement parameter of an image block from an image to be processed: determining a first image block from the image to be processed according to the window scale parameter in the operation parameter and a preset default initial position; traversing the image to be processed according to the window scale parameter and the window displacement parameter and a preset sequence to obtain image blocks except the first image block.

In another possible implementation, the image pooling feature matrix calculation module 74 is further configured to: and according to the position of each image block in the image to be processed, performing splicing processing on the pooling characteristic matrixes of the image blocks adjacent to each other in position to obtain the pooling characteristic matrix of the image to be processed.

The image computing device of the convolutional neural network provided by the embodiment of the invention has the same implementation principle and technical effects as those of the embodiment of the image computing method of the convolutional neural network, and for the sake of brief description, reference is made to corresponding contents in the embodiment of the image computing method of the convolutional neural network where the embodiment of the image computing device of the convolutional neural network is not mentioned.

The embodiment of the invention also provides a machine-readable storage medium, which stores machine-executable instructions that, when being called and executed by a processor, cause the processor to implement the image operation method of the convolutional neural network, and the implementation of the method can be found in the foregoing embodiment, and will not be described herein.

The image computing method of the convolutional neural network, the image computing device of the convolutional neural network and the computer program product of the electronic device provided by the embodiment of the invention comprise a computer readable storage medium storing program codes, and the instructions included in the program codes can be used for executing the image computing method of the convolutional neural network described in the method embodiment, and specific implementation can be seen in the method embodiment and will not be repeated here.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In addition, in the description of embodiments of the present invention, unless explicitly stated and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art.

In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. The image operation method of the convolutional neural network is characterized in that the method is applied to electronic equipment for realizing convolutional neural network operation; the method comprises the following steps:

acquiring hardware parameters of the electronic equipment;

determining constraint conditions of a preset convolutional neural network according to the hardware parameters; wherein the hardware parameter includes an internal cache size of the electronic device; the constraint conditions include: the maximum scale of a convolution feature matrix output by a convolution layer in the convolution neural network or the maximum scale of an image block input by the convolution layer;

determining the maximum scale of the image block input by the convolution layer according to the constraint condition if the constraint condition comprises the maximum scale of the convolution feature matrix output by the convolution layer;

determining the scale of a pooling feature matrix output by a pooling layer in the convolutional neural network according to the maximum scale of the image block input by the convolutional layer;

determining operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer; the operation parameters are associated with storage attributes of the electronic equipment;

Collecting image blocks from the image to be processed according to the operation parameters;

for each image block, the following processing is performed: performing convolution calculation on a current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; performing pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block;

and according to the position of each image block in the image to be processed, forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed.

2. The method of claim 1, wherein the step of determining the largest scale of the image block of the convolutional layer input based on the largest scale of the convolutional feature matrix comprises:

determining the maximum scale of the image block input by the convolution layer by the following formula:

i_cov_width_valid_max＝(o_cov_width_max-1)*cs_w+1；

i_cov_height_valid_max＝(o_cov_height_max-1)*cs_h+1；

wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; o_ cov _width_max is the maximum width of the convolution feature matrix output by the convolution layer; o_ cov _height_max is the maximum height of the convolution feature matrix output by the convolution layer; cs_w is the horizontal step length of the convolution layer; cw_h is the vertical step size of the convolutional layer.

3. The method of claim 1, wherein the step of determining the scale of the pooling feature matrix of the pooling layer output in the convolutional neural network based on the maximum scale of the image block of the convolutional layer input comprises:

determining the scale of a pooling feature matrix output by a pooling layer in the convolutional neural network through the following formula:

(pw_w+(pl_width-1)*ps_w-1)*cs_w+1＝i_cov_width_valid_max；

(pw_h+(pl_height-1)*ps_h-1)*cs_h+1＝i_cov_height_valid max；

wherein i_ cov _width_valid_max is the maximum width of the image block input by the convolution layer; i_ cov _height_valid_max is the maximum height of the image block input by the convolution layer; cs_w is the horizontal step length of the convolution layer; cs_h is the vertical step length of the convolution layer; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; pl_width is the width of the pooling feature matrix output by the pooling layer; pl_height is the height of the pooling feature matrix output by the pooling layer; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer.

4. The method of claim 1, wherein the step of determining the operational parameters of the convolutional neural network based on the scale of the pooling feature matrix output by the pooling layer comprises:

Rounding the scale of the pooling feature matrix output by the pooling layer;

and determining the operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer after rounding.

5. The method of claim 4, wherein the step of determining the operational parameters of the convolutional neural network based on the dimensions of the rounded pooled feature matrix output by the pooled layer comprises:

the window scale for collecting image blocks from the image to be processed in the operation parameters is calculated by the following formula:

i_cov_width＝(pw_w+(o_pl_width-1)*ps_w-1)*cs_w+cw_w；

i_cov_height＝(pw_h+(o_pl_height-1)*ps_h-1)*cs_h+cw_h；

wherein i_ cov _width is the width of the window; i_ cov _height is the height of the window; pw_w is the pooling window width of the pooling layer; pw_h is the pooling window height of the pooling layer; o_pl_width is the width of the pooling feature matrix output by the pooling layer after rounding; o_pl_height is the height of the pooling feature matrix output by the pooling layer after rounding; ps_w is the horizontal step size of the pooling layer; ps_h is the vertical step length of the pooling layer; cs_w is the horizontal step length of the convolution layer; cs_h is the vertical step size of the convolution layer.

6. The method of claim 5, wherein the step of determining the operational parameters of the convolutional neural network based on the dimensions of the rounded pooled feature matrix output by the pooled layer comprises:

The window displacement for collecting the image block from the image to be processed in the operation parameters is calculated by the following formula:

cb_ws＝((pw_w+(o_pl_width-1)*ps_w-1)*cs_w+1-(pw_w-ps_w))*cs_w；

cb_hs＝((pw_h+(o_pl_height-1)*ps_h-1)*cs_h+1-(pw_h-ps_h))*cs_h；

7. The method of claim 1, wherein the operational parameters include: the step for collecting the image blocks from the image to be processed according to the operation parameters comprises the following steps:

determining a first image block from the image to be processed according to the window scale parameter in the operation parameters and a preset default initial position;

traversing the image to be processed according to the window scale parameter and the window displacement parameter and a preset sequence to obtain image blocks except the first image block.

8. The method of claim 1, wherein the step of forming the pooled feature matrix of each image block into the pooled feature matrix of the image to be processed according to the position of each image block in the image to be processed comprises:

and according to the position of each image block in the image to be processed, performing splicing processing on the pooling characteristic matrixes of the image blocks adjacent to each other in position to obtain the pooling characteristic matrixes of the image to be processed.

9. An image operation device of a convolutional neural network is characterized in that the device is applied to electronic equipment for realizing convolutional neural network operation; the device comprises:

the operation parameter acquisition module is used for acquiring hardware parameters of the electronic equipment; determining constraint conditions of a preset convolutional neural network according to the hardware parameters; wherein the hardware parameter includes an internal cache size of the electronic device; the constraint conditions include: the maximum scale of a convolution feature matrix output by a convolution layer in the convolution neural network or the maximum scale of an image block input by the convolution layer; determining the maximum scale of the image block input by the convolution layer according to the constraint condition if the constraint condition comprises the maximum scale of the convolution feature matrix output by the convolution layer; determining the scale of a pooling feature matrix output by a pooling layer in the convolutional neural network according to the maximum scale of the image block input by the convolutional layer; determining operation parameters of the convolutional neural network according to the scale of the pooling feature matrix output by the pooling layer; the operation parameters are associated with storage attributes of the electronic equipment;

The image block acquisition module is used for acquiring image blocks from the image to be processed according to the operation parameters;

the image block pooling feature matrix calculating module is used for carrying out the following processing on each image block: performing convolution calculation on a current image block through a convolution layer of the convolution neural network to obtain a convolution feature matrix of the current image block; performing pooling calculation on the convolution feature matrix through a pooling layer of the convolution neural network to obtain a pooling feature matrix of the current image block;

and the image pooling feature matrix calculation module is used for forming the pooling feature matrix of each image block into the pooling feature matrix of the image to be processed according to the position of each image block in the image to be processed.

10. An electronic device, characterized in that the electronic device comprises a processing device and a storage means;

the storage device has stored thereon a computer program which, when executed by the processing device, performs the steps of the image operation method of a convolutional neural network according to any one of 1 to 8.

11. A computer-readable storage medium having stored thereon a computer program, characterized in that the computer program, when run by a processing device, performs the steps of the image operation method of a convolutional neural network according to any one of claims 1 to 8.