CN111967579A - Method and apparatus for performing convolution calculation on image using convolution neural network - Google Patents

Method and apparatus for performing convolution calculation on image using convolution neural network Download PDF

Info

Publication number
CN111967579A
CN111967579A CN202010775401.XA CN202010775401A CN111967579A CN 111967579 A CN111967579 A CN 111967579A CN 202010775401 A CN202010775401 A CN 202010775401A CN 111967579 A CN111967579 A CN 111967579A
Authority
CN
China
Prior art keywords
control channel
neural network
convolution
convolutional
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010775401.XA
Other languages
Chinese (zh)
Inventor
梁喆
曹宇辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Aixin Technology Co ltd
Original Assignee
Beijing Aixin Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Aixin Technology Co ltd filed Critical Beijing Aixin Technology Co ltd
Publication of CN111967579A publication Critical patent/CN111967579A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
    • G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Neurology (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method and a device for carrying out convolution calculation on an image by using a convolution neural network; wherein the image is input into a convolutional neural network; the first convolutional layer outputs a first characteristic diagram, the first characteristic diagram comprises a first control channel, and the first convolutional layer is a middle convolutional layer of the convolutional neural network; the method comprises the following steps: taking an area with a characteristic value smaller than a preset threshold value in the first control channel as a first control area, wherein the characteristic value of any position of the first control channel is used for representing the probability that the corresponding position of the first characteristic diagram belongs to the target characteristic; determining a redundant area of the first characteristic diagram according to the first control area; and the other areas except the redundant area in the first characteristic diagram enter the next convolution layer of the first convolution layer to carry out convolution operation, so that a second characteristic diagram is obtained. In the method, the redundant area does not carry out the next layer of convolution operation, so that invalid calculation can be prevented, the calculation amount of the neural network is reduced, the efficiency of convolution calculation is improved, and the power consumption of convolution calculation is reduced.

Description

Method and apparatus for performing convolution calculation on image using convolution neural network
Technical Field
The invention relates to the technical field of convolutional neural networks, in particular to a method and a device for carrying out convolutional calculation on an image by using a convolutional neural network.
Background
In the related art, the convolutional neural network algorithm includes a one-step method and a two-step method. The two-step method needs two models for calculation, firstly, the first model is adopted for calculation of a rough detection network, and an effective area, such as a probability thermodynamic diagram of the appearance of a target (such as a human face), is obtained; and then, based on the output image of the first model, the image after the cutout is used as the input image of the second model, and the second model is used for calculating a fine detection network or an attribute network, so that a required accurate detection or attribute result is obtained. The convolutional neural network of the two-step method requires two models, and also requires a CPU (Central Processing Unit) to perform cumbersome image Processing operations (such as matting, resizing, aligning, etc.), and often the input image of the latter model is small and many, and these operations affect the computational efficiency of the convolutional neural network, resulting in low overall efficiency.
The one-step method is to directly calculate the detection result and the attribute result of each region in the whole image through a model. The whole model needs to calculate all regions of the input image once from beginning to end, and when the input image is large, the requirement of the overall calculation power of the model is particularly large to obtain enough precision, so that the convolutional neural network is low in speed, low in efficiency and high in power consumption.
Disclosure of Invention
In view of the above, the present invention provides a method and an apparatus for performing convolution calculation on an image using a convolutional neural network, so as to reduce the calculation amount of the neural network, improve the efficiency of the convolution calculation, and reduce the power consumption of the convolution calculation.
In a first aspect, an embodiment of the present invention provides a method for performing convolution calculation on an image using a convolutional neural network, where the image is input into the convolutional neural network; outputting a first feature map by a first convolution layer of the convolutional neural network, wherein the first feature map comprises a first control channel, and the first convolution layer is a middle convolution layer of the convolutional neural network; the method comprises the following steps: taking an area, of which the characteristic value is smaller than a preset threshold value, in a first control channel as a first control area, wherein the first control channel is a channel in a first characteristic diagram, and the characteristic value of any position of the first control channel is used for representing the probability that the corresponding position of the first characteristic diagram belongs to the target characteristic; determining a redundant area of the first characteristic diagram according to the first control area; and the other areas except the redundant area in the first characteristic diagram enter the next convolution layer of the first convolution layer to carry out convolution operation, so that a second characteristic diagram is obtained.
In a preferred embodiment of the present invention, the step of determining the redundant area of the first characteristic map according to the first control area includes: and taking the area corresponding to the coordinate position of the first control area in the first characteristic diagram as a redundant area of the first characteristic diagram.
In a preferred embodiment of the present invention, after the step of obtaining the second feature map, the method further includes: the feature value of the region in the second feature map corresponding to the redundant region of the first feature map is set to a fixed value.
In a preferred embodiment of the present invention, the above-mentioned third convolutional layer of the convolutional neural network outputs a third feature map, the third feature map includes a third control channel, and the third convolutional layer is an intermediate convolutional layer different from the first convolutional layer in the convolutional neural network; the method further comprises the following steps: taking a region of which the characteristic value is smaller than a preset threshold value in a third control channel as a third control region, wherein the third control channel is one channel in a third characteristic diagram, and the characteristic value of any position of the third control channel is used for representing the probability that the corresponding position of the third characteristic diagram belongs to the target characteristic; determining a redundant area of the third feature map according to the third control area; and the other areas except the redundant area in the third characteristic diagram enter the next convolution layer of the third convolution layer to carry out convolution operation, so that a fourth characteristic diagram is obtained.
In a preferred embodiment of the present invention, a third convolutional layer of the convolutional neural network outputs a third feature map, where the third convolutional layer is an intermediate convolutional layer of the convolutional neural network different from the first convolutional layer; the method further comprises the following steps: determining a redundant area of the third feature map according to the first control area; the other areas except the redundant area in the third characteristic diagram enter the next convolution layer of the third convolution layer to carry out convolution operation, and a fourth characteristic diagram is obtained; and the channel size of the third characteristic diagram is the same as that of the first characteristic diagram.
In a preferred embodiment of the present invention, the control channel is established by the following steps: determining the position of a convolutional layer of a control channel in the convolutional neural network, wherein the output characteristic diagram comprises the position of the convolutional layer of the control channel; taking one channel of the output characteristic diagram of the convolution layer of which the output characteristic diagram comprises a control channel as the control channel; marking the characteristic value of the training control channel according to the position of the target in the training image, so that the characteristic value of the position, corresponding to the target, in the training control channel is a first characteristic value, and the characteristic value of the position, corresponding to the non-target, in the training control channel is a second characteristic value; inputting the training image into a convolutional neural network, and obtaining a characteristic value of a control channel according to initial parameters of the convolutional neural network; obtaining a loss value according to the characteristic value of the control channel and the characteristic value of the training control channel; if the loss value is converged, the training is finished; and if the loss value is not converged, adjusting the parameters of the convolutional neural network according to the loss value.
In a second aspect, an embodiment of the present invention further provides an apparatus for performing convolution calculation on an image using a convolutional neural network, where the image is input into the convolutional neural network; outputting a first feature map by a first convolution layer of the convolutional neural network, wherein the first feature map comprises a first control channel, and the first convolution layer is a middle convolution layer of the convolutional neural network; the above-mentioned device includes: the first control area determining module is used for taking an area, of which the characteristic value is smaller than a preset threshold value, in a first control channel as a first control area, the first control channel is a channel in a first characteristic diagram, and the characteristic value of any position of the first control channel is used for representing the probability that the corresponding position of the first characteristic diagram belongs to the target characteristic; the redundant area determining module is used for determining a redundant area of the first characteristic diagram according to the first control area; and the second characteristic diagram obtaining module is used for performing convolution operation on the next convolution layer entering the first convolution layer in the other areas except the redundant area in the first characteristic diagram to obtain a second characteristic diagram.
In a third aspect, an embodiment of the present invention further provides an electronic system, where the electronic system includes: the device comprises an image acquisition device, a processing device and a storage device; the image acquisition equipment is used for acquiring an image; the storage means has stored thereon a computer program which, when run by a processing apparatus, performs the method of convolution calculation of an image using a convolutional neural network as described above.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processing device to perform the steps of the method for performing convolution calculation on an image using a convolutional neural network as described above.
The embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a method and a device for performing convolution calculation on an image by using a convolution neural network, wherein an area with a characteristic value smaller than a preset threshold value in a first control channel of a first characteristic diagram is used as a first control area, and a redundant area of the first characteristic diagram is determined according to the first control area; and (4) enabling other areas except the redundant area in the first characteristic diagram to enter a next convolution layer of the first convolution layer for convolution operation to obtain a second characteristic diagram. In the method, the redundant area does not carry out the next layer of convolution operation, so that invalid calculation can be prevented, the calculation amount of the neural network is reduced, the efficiency of convolution calculation is improved, and the power consumption of convolution calculation is reduced.
Additional features and advantages of the disclosure will be set forth in the description which follows, or in part may be learned by the practice of the above techniques of the disclosure, or may be learned by the practice of the disclosure.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic structural diagram of an electronic system according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for performing convolution calculations on an image using a convolutional neural network according to an embodiment of the present invention;
FIG. 3 is a flow chart of another method for performing convolution calculations on an image using a convolutional neural network according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a method for performing convolution calculation on an image using a convolutional neural network according to an embodiment of the present invention;
FIG. 5 is a flow chart of another method for performing convolution calculations on an image using a convolutional neural network according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating another method for performing convolution calculation on an image using a convolutional neural network according to an embodiment of the present invention;
FIG. 7 is a flowchart of another method for performing convolution calculations on an image using a convolutional neural network according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an apparatus for performing convolution calculation on an image by using a convolutional neural network according to an embodiment of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In view of the problems of slow speed and low calculation efficiency of the conventional convolutional neural network algorithm, embodiments of the present invention provide a method and an apparatus for performing convolutional calculation on an image using a convolutional neural network, and the technique may be applied to various devices such as a server, a computer, a camera, a mobile phone, a tablet computer, a vehicle central control device, and the like, and may be implemented by using corresponding software and hardware, and the embodiments of the present invention are described in detail below.
The first embodiment is as follows:
first, an example electronic system 100 for implementing the method and apparatus for performing convolution calculations on images using a convolutional neural network of an embodiment of the present invention is described with reference to FIG. 1.
As shown in FIG. 1, an electronic system 100 includes one or more processing devices 102, one or more memory devices 104, an input device 106, an output device 108, and one or more image capture devices 110, which are interconnected via a bus system 112 and/or other type of connection mechanism (not shown). It should be noted that the components and structure of the electronic system 100 shown in fig. 1 are exemplary only, and not limiting, and that the electronic system may have other components and structures as desired.
Processing device 102 may be a gateway or may be an intelligent terminal or device that includes a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may process data from and control other components of electronic system 100 to perform desired functions.
Storage 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, Random Access Memory (RAM), cache memory (or the like). The non-volatile memory may include, for example, Read Only Memory (ROM), a hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer-readable storage medium and executed by processing device 102 to implement the client functionality (implemented by the processing device) of the embodiments of the invention described below and/or other desired functionality. Various applications and various data, such as various data used and/or generated by the applications, may also be stored in the computer-readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, and the like.
Image capture device 110 may capture images and store the captured image data in storage 104 for use by other components.
For example, the devices in the method and apparatus for implementing the convolution calculation on the image using the convolutional neural network according to the embodiment of the present invention may be integrally disposed, or may be dispersedly disposed, such as integrally disposing the processing device 102, the storage device 104, the input device 106 and the output device 108, and disposing the image capturing device 110 at a designated position where a picture can be captured. When the above-described devices in the electronic system are integrally provided, the electronic system may be implemented as an intelligent terminal such as a camera, a smart phone, a tablet computer, a vehicle-mounted terminal, and the like.
Example two:
the embodiment provides a method for performing convolution calculation on an image by using a convolution neural network, wherein the method comprises the steps of firstly inputting the image into the convolution neural network; the first convolutional layer of the convolutional neural network outputs a first characteristic diagram, the first characteristic diagram comprises a first control channel, and the first convolutional layer is an intermediate convolutional layer of the convolutional neural network.
The convolutional neural network in this embodiment is mainly used for classifying, target detecting, or target identifying an image, and the calculation of the convolutional neural network may extract features of a person or an object in the image, identify the person or the object in the image, or count the person or the object in the image.
Convolutional neural networks are typically composed of one or more convolutional layers, each of which is composed of several convolutional kernels, and a top fully-connected layer, and may also include associated weights and pooling layers. Convolution layers perform convolution operations with convolution kernels in order to extract different features of the input. The middle convolutional layer is a convolutional layer of the neural network. The first convolution layer belongs to the middle convolution layer, the first convolution layer outputs a first characteristic diagram, the first characteristic diagram is composed of a plurality of channels, and the first control channel is one of the channels; the first convolution layer includes a plurality of convolution kernels, wherein one convolution kernel corresponds to the first control channel.
Based on the above description, a flowchart of a method for performing convolution calculation on an image by using a convolution neural network as shown in fig. 2 includes the following steps:
step S202, a region of a first control channel, where a feature value is smaller than a preset threshold, is used as a first control region, where the first control channel is a channel in a first feature map, and a feature value of any position of the first control channel is used to represent a probability that a corresponding position of the first feature map belongs to a target feature.
For the convolutional neural network of this embodiment, an image that may include an object to be processed needs to be input, the image may be a picture, a photograph, and other various types and formats of graphic files, and the processing of the image includes operations such as recognizing or detecting the object in the image. The convolutional neural network performs convolution on the image through convolutional layers, and a feature graph output by each convolutional layer serves as the input of the next convolutional layer.
Then, the first profile of the first convolutional layer output includes a plurality of channels, wherein one channel is a first control channel. The feature value of any region in the first control channel refers to a value output by convolution operation of the region, and the feature value is trained to be used for explaining the probability that the position corresponding to the position in the first feature map belongs to the target feature. The preset threshold is determined by a user according to experience or actual requirements, and a region of the first control channel in which the characteristic value is smaller than the preset threshold is taken as a first control region. If the characteristic value of a certain region in the first control channel is smaller than the preset threshold value, the region is indicated to have a smaller probability of belonging to the target characteristic at the corresponding position in the first characteristic diagram. For example, the first channel of the current convolution operation includes a position A, B, C, which corresponds to positions a, b, and c in the first feature map; if the characteristic value of the position A output through convolution calculation is 0.5, the characteristic value of the position B output through convolution calculation is 0.7, the characteristic value of the position C output through convolution calculation is 0.9, and the preset threshold value is 0.8; the probabilities that the positions a, B and c in the first feature map contain the face features are 0.5, 0.7 and 0.9 respectively, and the position a and the position B belong to the first control area because the feature values of the positions a and B are smaller than the preset threshold value.
Step S204, a redundant area of the first characteristic diagram is determined according to the first control area.
The redundant area refers to an area where the next convolution calculation is not needed, that is, an area in the feature map that does not include the target feature. Because the regions in the first control region correspond to the regions of the first feature map one to one, the redundant region of the first feature map corresponding to the first control region can be determined according to the corresponding relationship, and the redundant region of the first feature map has a smaller probability of containing the features of the target.
For example, in the above example, the position A, B, C of the first channel corresponds to the positions a, B, and c in the first characteristic diagram one-to-one, and the position a and the position B constitute the first control area.
Step S206, the other areas except the redundant area in the first characteristic diagram enter the next convolution layer of the first convolution layer for convolution operation to obtain a second characteristic diagram.
The regions other than the redundant region are regions having a high probability of including the target feature, and these regions are required to be subjected to convolution operation of the next layer. Therefore, the other regions except the redundant region are input to the next convolution layer after the first convolution layer, and the second characteristic map is output from the next convolution layer. The redundant area has smaller probability of including the target characteristics, so that convolution operation in the next convolution layer is not performed, and invalid calculation can be prevented.
For example, a convolutional neural network has 4 convolutional layers, wherein the 2 nd convolutional layer is used as a first convolutional layer, the characteristic diagram output by the 2 nd convolutional layer is a first characteristic diagram, the first characteristic diagram has 10 channels, and the 10 th channel is a first control channel. Then, the area of the 10 th channel of the 2 nd convolutional layer, in which the feature value is smaller than the preset threshold, is the first control area, the corresponding position of the first control area in the first feature map is the redundant area, the other areas in the first feature map except the redundant area are input into the 3 rd convolutional layer, the next convolutional operation is performed, and the feature map output by the 3 rd convolutional layer is the second feature map.
The existing convolution calculation algorithm needs much calculation power to obtain a high-precision result, and an important reason is that a model carries out a large amount of invalid calculation on non-effective areas occupying most of the total image at the later stage (high level). In the embodiment, a large amount of invalid calculations can be saved in the calculation process of the convolutional neural network, and the calculation power effect and the power consumption performance of the convolutional calculation algorithm can be greatly improved. And the calculation result of the convolutional neural network does not cause the deviation of the accuracy due to the saving of invalid calculation.
In many cases, for example, in the detection, feature extraction, recognition and the like of human faces and human bodies in large scenes, the regions actually required to be detected and recognized often occupy a small proportion of the whole image, so that a hardware accelerator with a function of performing convolution calculation only on partial regions of a feature map can fully utilize the advantage of working only in an interested region calculation unit, thereby greatly reducing invalid calculation and improving the power consumption performance ratio.
The embodiment of the invention provides a method for performing convolution calculation on an image by using a convolution neural network, wherein an area, of which a characteristic value is smaller than a preset threshold value, in a first control channel of a first characteristic diagram is used as a first control area, and a redundant area of the first characteristic diagram is determined according to the first control area; and (4) enabling other areas except the redundant area in the first characteristic diagram to enter a next convolution layer of the first convolution layer for convolution operation to obtain a second characteristic diagram. In the method, the redundant area does not carry out the next layer of convolution operation, so that invalid calculation can be prevented, the calculation amount of the neural network is reduced, the efficiency of convolution calculation is improved, and the power consumption of convolution calculation is reduced.
Example three:
the embodiment provides another method for performing convolution calculation on an image by using a convolution neural network, and the method is implemented on the basis of the embodiment; the present embodiment focuses on a specific process of determining the redundant area of the first feature map according to the first control area. Another method for performing convolution calculations on an image using a convolutional neural network, as shown in fig. 3, includes the following steps:
step S302, a region of a first control channel, where a feature value is smaller than a preset threshold, is used as a first control region, where the first control channel is a channel in a first feature map, and a feature value at any position of the first control channel is used to represent a probability that a corresponding position of the first feature map belongs to a target feature.
The first control channel is determined by means of pre-training and can be executed through the steps A1-A6:
step A1, determining the position of the convolution layer of the control channel in the convolution neural network.
To train the first control channel, it is first necessary to determine where the output signature includes the convolutional layer of the control channel in the convolutional neural network. The selected convolutional layer cannot be too far forward (e.g., the 2 nd convolutional layer is selected), because the first few intermediate convolutional layers extract more low-order features (e.g., edge features), whether a certain position in the feature map contains a target feature or not cannot be accurately judged according to the feature maps output by the first few middle convolution layers, a control channel cannot be trained possibly, high-order features (such as the nose, the face and the mouth of a person) can be extracted by the later middle convolution layers (more than 10 layers, for example), whether a certain position in the feature map contains a target feature or not can be accurately judged according to the output feature maps of the middle convolution layers, however, the selected middle convolution layers cannot be too far back, generally, the control channel may be a channel in the convolutional layer after layer 10, since if the selected intermediate convolutional layer is too late, the resulting savings are less computationally intensive and less effective.
The position of the convolutional layer of the output characteristic diagram including the control channel in the convolutional neural network can be determined by a user, and can also be determined through steps B1-B4;
and step B1, determining the characteristic range extracted by each convolution layer of the convolution neural network according to the parameters of the convolution neural network.
The parameters of the convolutional neural network are optimized through a back propagation algorithm, and the extraction range of the convolutional layer is determined by the parameters of the convolutional neural network. In general, the first layer of convolutional layers may only extract some low-level features such as edges, lines, and corners, and more layers of convolutional layers may iteratively extract more complex features from the low-level features. That is, the feature range of the first layer of convolutional layer is a low-level feature, the extraction range of the convolutional layer of more layers is more complicated, and the number of extracted edge features of the convolutional layer of more layers is smaller.
And step B2, sorting the convolution layers of the convolutional neural network according to the number of the edge features included in the feature range from small to large to obtain a sorting result.
And according to the sequence of the number of the extracted edge features from small to large, the fewer the number of the extracted edge features of the convolution layer which is ranked more forward, the more suitable the convolution layer which comprises the control channel and is used as the output feature diagram.
And step B3, extracting a preset number of convolutional layers of the convolutional neural network from the first convolutional layer in the sequencing result.
The predetermined number is the number of convolutional layers that are intended to include control channels as an output profile, although the predetermined number may be slightly greater than the number of convolutional layers that are intended to include control channels as an output profile. Because, the extracted convolutional layer may be culled.
And step B4, determining the extracted convolutional layer of the convolutional neural network as the convolutional layer of which the output characteristic diagram comprises the control channel.
The more the ranking result is, the less the extracted edge features are, the preset number of convolutional layers is extracted from the first convolutional layer in the ranking result, which is equivalent to the preset number of convolutional layers with the minimum number of included edge features, and the extracted convolutional layers can be ensured to be used as convolutional layers of which the output feature diagram includes control channels.
In the above manner, the convolutional layers are sorted in the order from small to large of the number of edge features included in the feature range, and a preset number of convolutional layers are extracted according to the sorting result. The number of edge features included in the extracted convolutional layer can be ensured to be small, and the convolutional layer which can be used as an output feature map and comprises a control channel can be used.
In addition, since the output feature map does not have to be too late in the position of the convolutional layer including the control channel, the method further includes, after the step of determining the extracted convolutional layer of the convolutional neural network as the convolutional layer whose output feature map includes the control channel:
and if the distance between the position of the extracted convolutional layer of the convolutional neural network and the last convolutional layer of the convolutional neural network is smaller than a preset distance threshold, removing the convolutional layer smaller than the preset distance threshold from the convolutional layer of which the output characteristic diagram comprises the control channel.
And removing the convolution layers with the distance smaller than the preset distance threshold value from the convolution layers of which the output characteristic graphs comprise the control channels and which are determined in the step B4. And if the distance between the middle convolutional layer N and the last convolutional layer determined in the step B4 is smaller than the preset distance threshold, the middle convolutional layer N is too close to the last convolutional layer, and in this case, the significance of determining the redundant region is not large, the calculation amount of the neural network is not reduced basically, and the efficiency and the power consumption cannot be improved. Since the step of determining the redundant region requires an additional amount of computation, if the amount of computation saved by saving the redundant region is less than the amount of computation required to determine the redundant region, the amount of computation increases for the overall neural network.
In the above manner, the convolutional layers whose distances are smaller than the preset distance threshold are removed from the convolutional layers whose output feature maps include the control channel. The method can prevent the increase of the calculated amount, ensure that each output characteristic diagram comprises the control channel in the convolution layer of the control channel, reduce the calculated amount of the neural network, improve the efficiency of convolution calculation and reduce the power consumption of the convolution calculation.
Step a2, one channel of the output characteristic map including the output characteristic map of the convolutional layer of the control channel is taken as the control channel.
After determining the convolutional layers of which the output characteristic diagram comprises the control channel, selecting one channel from the determined convolutional layers as the control channel. There is no particular requirement for the step of selecting the control channel, and the control channel only needs to be one channel in which the output characteristic map determined in step a1 includes the output characteristic map of the convolutional layer of the control channel. For example, if step a1 determines that convolutional layer X may be a convolutional layer that outputs a feature map including control channels, and the feature map output by convolutional layer X has Y channels, then one of the Y channels may be selected as a control channel.
Step A3, labeling the eigenvalue of the training control channel according to the position of the target in the training image, so that the eigenvalue of the position corresponding to the target in the training control channel is a first eigenvalue, and the eigenvalue of the position corresponding to the non-target in the training control channel is a second eigenvalue.
The training image may be an image containing a target. According to the position of the target in the training image and the convolutional layer where the control channel is located, a user can deduce which positions in the feature diagram where the control channel is located contain the target features, so that the training control channel can be labeled. If the position of the feature map contains the target feature, setting the feature value of the position corresponding to the training control channel as a first feature value; and if the position of the feature map does not contain the target feature, setting the feature value of the position corresponding to the training control channel as a second feature value. For example, the first characteristic value may be set to 1 and the second characteristic value may be set to 0.
And step A4, inputting the training image into the convolutional neural network, and obtaining the characteristic value of the control channel according to the initial parameters of the convolutional neural network.
The initial parameters of the convolutional neural network refer to the initial parameters of the convolutional neural network set before the parameter training of the convolutional neural network. And inputting the training image into a convolutional neural network, and outputting the characteristic value of the control channel.
And step A5, obtaining a loss value according to the characteristic value of the control channel and the characteristic value of the training control channel.
By means of a predetermined loss function, a loss value between the characteristic value of the control channel and the characteristic value of the training control channel can be determined. The loss value is used for explaining the difference between the characteristic value of the training control channel and the characteristic value of the control channel, and the larger the loss value is, the less the characteristic value of the control channel is expected to be.
Step A6, if the loss value is converged, the training is completed; and if the loss value is not converged, adjusting the parameters of the convolutional neural network according to the loss value.
The convergence of the loss value is a condition for stopping the training, and the convergence of the loss value means that the loss value is close to a certain value. If the loss value is converged, the difference between the characteristic value of the control channel and the characteristic value of the training control channel is relatively fixed, and then the training can be completed; if the loss value is not converged, the convolutional neural network still needs to be trained, namely parameters of the convolutional neural network need to be adjusted according to the loss value, and training is carried out again until the loss value is converged.
In addition to the convergence of the loss value, there are other conditions for completing the training, such as: and when the training times reach a preset training time threshold value, finishing training of all images in the training set, and the like. The specific training end condition is determined by the staff according to the actual situation, and the training can be ended only when a plurality of training end conditions are met, or the training can be ended when one of the plurality of training end conditions is met.
In the above mode, the position of the convolutional layer of the feature map including the control channel in the convolutional neural network is determined, and then one channel of the output feature map of the convolutional layer is used as the control channel to train the neural network.
In step S304, an area corresponding to the first control area coordinate position in the first feature map is set as a redundant area of the first feature map.
It has been mentioned above that the areas in the first control areas correspond to the areas of the first feature maps one to one, and the specific correspondence relationship may be that the areas in the first feature maps correspond to the coordinate positions of the first control areas one to one, that is, the coordinates of the position of each first control area have an area in the first control area corresponding to it.
In the above manner, the purpose that the areas in the first control area correspond to the areas of the first characteristic diagram one to one is achieved by a method of corresponding coordinate positions.
Step S306, the other areas except the redundant area in the first characteristic diagram enter the next convolution layer of the first convolution layer for convolution operation to obtain a second characteristic diagram.
In step S308, the feature value of the region corresponding to the redundant region of the first feature map in the second feature map is set to a fixed value.
The region corresponding to the redundant region of the first feature map is not subjected to convolution calculation on the next convolution layer, but a preset fixed value is output, and the fixed value indicates that the corresponding region has a lower probability of including the target feature. In the above manner, the convolution calculation is not performed after the region corresponding to the redundant region of the first feature map, but a fixed value is directly output, so that invalid calculation can be prevented, the calculation amount of the neural network is reduced, the efficiency of the convolution calculation is improved, and the power consumption of the convolution calculation is reduced.
Example four:
the embodiment provides another method for performing convolution calculation on an image by using a convolution neural network, and the method is implemented on the basis of the embodiment; the present embodiment focuses on a specific process in which the output characteristic diagrams of the plurality of convolutional layers include control channels.
First, for the case that only one convolutional layer in the convolutional neural network is a convolutional layer whose output feature map includes a control channel, see a schematic diagram of a method for performing convolutional calculation on an image using the convolutional neural network of fig. 4, as shown in fig. 4, the original image refers to an image input to the convolutional neural network, and the convolutional operation is implemented through the convolutional layer, where convolutional layer n is a convolutional layer whose output feature map includes a control channel. The characteristic diagram output by the convolutional layer n is provided with m channels which are respectively in one-to-one correspondence with the m convolutional kernels, wherein the Channel m is a control Channel. If some feature values in the Channel m are smaller than the threshold (for example, the leftmost part of the Channel m is marked by a cross box), the leftmost part in the whole feature map is considered not to correspond to the target feature, and the convolution operation is not performed on the leftmost part when the lower layer convolution operation (the (n + 1) th layer convolution layer) is performed.
Next, in the convolutional neural network, when a plurality of convolutional layers are convolutional layers whose output characteristic maps include control channels, there are two cases, depending on whether the control channels for determining whether the output characteristic map of each convolutional layer of the plurality of convolutional layers enters the next convolutional layer to perform the convolutional calculation are the same.
In the first case, if the control channel for determining whether the output characteristic map of each convolutional layer of the plurality of convolutional layers enters the next convolutional layer for convolution operation is different, the convolutional layer having two convolutional layers in the convolutional neural network as the output characteristic map including the control channel is taken as an example.
Assume that a third convolutional layer of the convolutional neural network outputs a third feature map, the third feature map contains a third control channel, and the third convolutional layer is an intermediate convolutional layer of the convolutional neural network different from the first convolutional layer.
In a convolutional neural network, the position of the third convolutional layer is behind the first convolutional layer, and the third convolutional layer is not necessarily adjacent to the first convolutional layer. The characteristic diagrams output by the third convolution layer and the first convolution layer both comprise control channels, namely the third characteristic diagram output by the third convolution layer comprises a third control channel.
Based on the above description, for the convolution operation process of the second convolution layer, the flowchart of another method for performing convolution calculation on an image by using a convolutional neural network as shown in fig. 5 includes the following steps:
step S502, a region of a third control channel, where a feature value is smaller than a preset threshold, is taken as a third control region, where the third control channel is a channel in a third feature map, and a feature value of any position of the third control channel is used to represent a probability that a corresponding position of the third feature map belongs to the target feature.
Since the third control channel is different from the first control channel, the third control channel needs to be trained separately, and a region in the trained third control channel, in which the feature value is smaller than the preset threshold value, is taken as a third control region. The feature values in the trained third feature channel represent the probability of belonging to the target feature at the corresponding position in the third feature map. It should be noted that the various parameters of the third control channel and the first control channel may be the same or different.
For example: assuming that the first signature graph output by the first convolution layer has 5 channels, where the 5 th channel is a first control channel, the corresponding preset threshold is 0.7, the first control channel has five regions A, B, C, D, E, and the feature values are 0.7, 0.6, 0.8, 0.9, and 0.8, respectively, then the first control region is B, and the position corresponding to the region B in the first signature graph has a smaller probability of including the target feature. Assuming that the third feature map output by the third convolutional layer has 3 channels, wherein the 3 rd channel is a third control channel, the corresponding preset threshold is 0.8, the third control channel has 3 regions, which are X, Y, Z respectively, and the feature value ratio is 0.8, 0.7 and 0.9, then the third control region is Y, and the region of the third feature map corresponding to the region Y has a smaller probability of containing the target feature.
Step S504, a redundant area of the third feature map is determined according to the third control area.
After the third control area is determined, the corresponding position of the third control area in the third feature map, that is, the redundant area of the third feature map, may be determined according to the corresponding relationship between the area of the third control channel and the corresponding position of the third feature map.
Step S506, the other areas except the redundant area in the third feature map enter the next convolution layer of the third convolution layer for convolution operation, and a fourth feature map is obtained.
If a certain region in the third feature map is a redundant region, the region is indicated to have a lower probability of including the target feature, so that the subsequent convolution operation is not required; if a certain region in the third feature map does not belong to the redundant region, the following convolution operation is required. And inputting other areas except the redundant area in the third feature map into the next volume of lamination of the third volume of lamination, and performing convolution operation on the areas to obtain a fourth feature map output by the next volume of lamination of the third volume of lamination.
In the above manner, if the feature map output by the third convolutional layer includes the third control channel, and the third control channel is not the same as the first control channel, it is necessary to train the third control channel separately, and the corresponding parameters of the third control channel are not necessarily the same as those of the first control channel.
For the principle that the feature map output by the plurality of convolutional layers includes a control channel, see fig. 6, another method for performing convolution calculation on an image by using a convolutional neural network. After the original image is input into the convolutional neural network, the feature maps output by the convolutional layer n and the convolutional layer l respectively comprise convolutional layers of a control Channel, wherein a first feature map output by the convolutional layer n comprises a control Channel m, which areas in the output feature map of the convolutional layer n do not enter the next convolutional layer of the convolutional layer n to carry out convolutional operation is determined according to the feature value of the Channel m, a second feature map output by the convolutional layer l comprises a control Channel k, and which areas in the output feature map of the convolutional layer l do not enter the next convolutional layer of the convolutional layer l to carry out convolutional operation is determined according to the feature value of the Channel k.
If some feature values in the Channel m are smaller than the threshold (for example, the leftmost part of the Channel m is marked by a cross box), the leftmost part in the feature map of the first feature map is considered not to correspond to the target feature, and the convolution operation is not performed on the leftmost part when the lower layer convolution operation (the (n + 1) th layer convolution layer) is performed. Similarly, for Channel k, the part of the feature value smaller than the threshold is also marked by a cross box, and it is considered that none of the leftmost part in the third feature map corresponds to the target feature, and when the convolution operation (i +1 th convolutional layer) is performed, the convolution operation is not performed on the part.
If the control Channel k is different from the control Channel m, the Channel m and the Channel k need to be trained separately, and the parameters corresponding to the trained Channel k and the parameters corresponding to the Channel m may be the same or different.
In the second case, if the control channel for determining whether the output feature map of each convolutional layer of the plurality of convolutional layers enters the next convolutional layer for convolution operation is the same, the convolutional layer having two convolutional layers in the convolutional neural network as the output feature map including the control channel is taken as an example.
Assume that a third convolutional layer of the convolutional neural network outputs a third feature map, the third convolutional layer being an intermediate convolutional layer of the convolutional neural network different from the first convolutional layer.
Based on the above description, for the convolution operation process of the second convolution layer, the flowchart of another method for performing convolution calculation on an image by using a convolutional neural network as shown in fig. 7 includes the following steps:
step S702, determining a redundant area of the third feature map according to the first control area; the channel size of the third characteristic diagram is the same as that of the first characteristic diagram. To avoid an additional training step, the redundant area of the third profile can be determined from the control area of the first profile. At this time, the size of the channels of the third feature map needs to be the same as the size of the channels of the first feature map, for example, each channel of the first feature map is an m × n matrix, and there are 5 channels in total, and each channel of the third feature map is also an m × n matrix, and then the control channel of the first feature map can be used to determine which region in the third feature map enters the next stage of convolution operation. The third profile may contain the same number of channels as the first profile or may contain a different number of channels. This is done on the premise that the convolution layer between the third convolution layer and the first convolution layer performs such a convolution operation on the first feature map output by the first convolution layer that the positions including the target feature in the third feature map and the first feature map are substantially the same.
Step S704, the other regions in the third feature map except the redundant region enter the next convolution layer of the third convolution layer for convolution operation, so as to obtain a fourth feature map.
Since the regions of the third feature map other than the redundant regions have a high probability of including the target feature, it is necessary to input the regions of the third feature map other than the redundant regions into the next convolutional layer of the third convolutional layer for convolution operation, and output the fourth feature map from the next convolutional layer of the third convolutional layer.
In the above-described aspect, the feature map output by the third convolutional layer may include the third control channel, and the numerical value of the first control channel is assigned to the third control channel as the feature value of the third control channel, and the feature map output by the third convolutional layer may also include no third control channel. Therefore, under the condition that the positions of the target features contained in the first feature map and the third feature map are basically the same, the training time can be saved, and the computing power resource of the neural network can be reduced.
In the mode, the regions of the characteristic diagram are determined not to be redundant regions through the control channel, and convolution operation is carried out on convolution layers of the next layer of input values of the regions, so that the region-by-region granularity can be refined, and unnecessary operation and waste of power consumption are saved to the maximum extent. Meanwhile, a certain control channel in the neural network characteristic diagram is adopted for determination, so that an upper-layer CPU (central processing unit) is not required for configuration operation, and the real-time performance of the system is greatly improved.
It should be noted that the above embodiments of the method are all described in a progressive manner, each embodiment is different from the other embodiments in terms of description, and the same and similar parts among the embodiments can be referred to each other.
Example five:
corresponding to the above method embodiment, the image is input into a convolutional neural network; the first convolutional layer of the convolutional neural network outputs a first characteristic diagram, the first characteristic diagram comprises a first control channel, and the first convolutional layer is an intermediate convolutional layer of the convolutional neural network.
Based on the above description, referring to fig. 8, a schematic structural diagram of an apparatus for performing convolution calculation on an image by using a convolution neural network, the apparatus comprising:
a first control area determining module 81, configured to use an area, in a first control channel, where a feature value is smaller than a preset threshold as a first control area, where the first control channel is a channel in a first feature map, and a feature value at any position of the first control channel is used to represent a probability that a corresponding position of the first feature map belongs to a target feature;
a redundant area determination module 82 for determining a redundant area of the first profile based on the first control area;
the second feature map obtaining module 83 is configured to perform convolution operation on the next convolution layer of the first convolution layer in the other areas except the redundant area in the first feature map to obtain a second feature map.
Further, the redundant area determining module is configured to: and taking the area corresponding to the coordinate position of the first control area in the first characteristic diagram as a redundant area of the first characteristic diagram.
Further, the apparatus further includes a fixed value setting module, configured to: after the step of obtaining a second signature, the method further comprises: the feature value of the region in the second feature map corresponding to the redundant region of the first feature map is set to a fixed value.
Further, a third convolutional layer of the convolutional neural network outputs a third feature map, the third feature map includes a third control channel, and the third convolutional layer is an intermediate convolutional layer different from the first convolutional layer in the convolutional neural network; the above-mentioned device still includes:
a third control region determination module to: taking a region of which the characteristic value is smaller than a preset threshold value in a third control channel as a third control region, wherein the third control channel is one channel in a third characteristic diagram, and the characteristic value of any position of the third control channel is used for representing the probability that the corresponding position of the third characteristic diagram belongs to the target characteristic;
a third redundant area determination module to: determining a redundant area of the third feature map according to the third control area;
a fourth feature map obtaining module configured to: and the other areas except the redundant area in the third characteristic diagram enter the next convolution layer of the third convolution layer to carry out convolution operation, so that a fourth characteristic diagram is obtained.
Further, a third convolutional layer of the convolutional neural network outputs a third feature map, wherein the third convolutional layer is an intermediate convolutional layer different from the first convolutional layer in the convolutional neural network; the above-mentioned device still includes:
a third redundant area acquisition module to: determining a redundant area of the third feature map according to the first control area;
a fourth feature map determination module to: the other areas except the redundant area in the third characteristic diagram enter the next convolution layer of the third convolution layer to carry out convolution operation, and a fourth characteristic diagram is obtained; and the channel size of the third characteristic diagram is the same as that of the first characteristic diagram.
Further, the control channel is established by the following steps: determining the position of a convolutional layer of a control channel in the convolutional neural network, wherein the output characteristic diagram comprises the position of the convolutional layer of the control channel; taking one channel of the output characteristic diagram of the convolution layer of which the output characteristic diagram comprises a control channel as the control channel; marking the characteristic value of the training control channel according to the position of the target in the training image, so that the characteristic value of the position, corresponding to the target, in the training control channel is a first characteristic value, and the characteristic value of the position, corresponding to the non-target, in the training control channel is a second characteristic value; inputting the training image into a convolutional neural network, and obtaining a characteristic value of a control channel according to initial parameters of the convolutional neural network; obtaining a loss value according to the characteristic value of the control channel and the characteristic value of the training control channel; if the loss value is converged, finishing the training; and if the loss value is not converged, adjusting the parameters of the convolutional neural network according to the loss value.
According to the device for performing convolution calculation on the image by using the convolution neural network, provided by the embodiment of the invention, an area, of which the characteristic value is smaller than a preset threshold value, in a first control channel of a first characteristic diagram is used as a first control area, and a redundant area of the first characteristic diagram is determined according to the first control area; and (4) enabling other areas except the redundant area in the first characteristic diagram to enter a next convolution layer of the first convolution layer for convolution operation to obtain a second characteristic diagram. In the method, the redundant area does not carry out the next layer of convolution operation, so that invalid calculation can be prevented, the calculation amount of the neural network is reduced, the efficiency of convolution calculation is improved, and the power consumption of convolution calculation is reduced.
Example six:
an embodiment of the present invention provides an electronic system, including: the device comprises an image acquisition device, a processing device and a storage device; the image acquisition equipment is used for acquiring an image; the storage means has stored thereon a computer program which, when run by a processing apparatus, performs the steps of the method of convolution calculation of an image using a convolutional neural network as described above.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working process of the electronic system described above may refer to the corresponding process in the foregoing method embodiments, and is not described herein again.
Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processing device, performs the steps of the method for performing convolution calculation on an image using a convolutional neural network as described above.
The computer program product of the method and the apparatus for performing convolution calculation on an image using a convolutional neural network according to the embodiment of the present invention includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method in the foregoing method embodiment, and specific implementation may refer to the method embodiment, and is not described herein again.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and/or the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (9)

1. A method of performing a convolution calculation on an image using a convolutional neural network, characterized in that the image is input to the convolutional neural network; outputting a first feature map by a first convolutional layer of the convolutional neural network, wherein the first feature map comprises a first control channel, and the first convolutional layer is a middle convolutional layer of the convolutional neural network; the method comprises the following steps:
taking an area, in the first control channel, of which the characteristic value is smaller than a preset threshold as a first control area, wherein the first control channel is one channel in the first characteristic diagram, and the characteristic value of any position of the first control channel is used for representing the probability that the corresponding position of the first characteristic diagram belongs to the target characteristic;
determining a redundant area of the first feature map according to the first control area;
and other areas except the redundant area in the first characteristic diagram enter the next convolution layer of the first convolution layer to carry out convolution operation, so that a second characteristic diagram is obtained.
2. The method of claim 1, wherein determining the redundant area of the first profile based on the first control area comprises:
and taking the area corresponding to the coordinate position of the first control area in the first characteristic diagram as a redundant area of the first characteristic diagram.
3. The method of claim 1, wherein after the step of obtaining a second profile, the method further comprises: and setting the characteristic value of the area corresponding to the redundant area of the first characteristic diagram in the second characteristic diagram as a fixed value.
4. The method of claim 1, wherein a third convolutional layer of the convolutional neural network outputs a third feature map, the third feature map containing a third control channel, the third convolutional layer being an intermediate convolutional layer of the convolutional neural network that is different from the first convolutional layer; the method further comprises the following steps:
taking a region of which the characteristic value is smaller than a preset threshold value in the third control channel as a third control region, wherein the third control channel is one channel in the third characteristic diagram, and the characteristic value of any position of the third control channel is used for representing the probability that the corresponding position of the third characteristic diagram belongs to the target characteristic;
determining a redundant area of the third feature map according to the third control area;
and other areas except the redundant area in the third characteristic diagram enter the next convolution layer of the third convolution layer to carry out convolution operation, so that a fourth characteristic diagram is obtained.
5. The method of claim 1, wherein a third convolutional layer of the convolutional neural network outputs a third feature map, the third convolutional layer being an intermediate convolutional layer of the convolutional neural network that is different from the first convolutional layer; the method further comprises the following steps:
determining a redundant area of the third feature map according to the first control area;
entering other areas except the redundant area in the third feature map into a next convolution layer of the third convolution layer for convolution operation to obtain a fourth feature map; wherein the channel size of the third feature map is the same as the channel size of the first feature map.
6. The method according to any one of claims 1-5, wherein the control channel is established by:
determining the position of a convolutional layer of a control channel in the convolutional neural network, wherein the output characteristic diagram comprises the position of the convolutional layer of the control channel;
taking one channel of the output characteristic diagram of the convolution layer of which the output characteristic diagram comprises a control channel as a control channel;
marking a characteristic value of a training control channel according to the position of a target in a training image, wherein the characteristic value of the training control channel corresponding to the position of the target is a first characteristic value, and the characteristic value of the training control channel corresponding to the position of a non-target is a second characteristic value;
inputting the training image into a convolutional neural network, and obtaining a characteristic value of the control channel according to an initial parameter of the convolutional neural network;
obtaining a loss value according to the characteristic value of the control channel and the characteristic value of the training control channel;
if the loss value is converged, the training is finished; and if the loss value is not converged, adjusting the parameters of the convolutional neural network according to the loss value.
7. An apparatus for performing convolution calculations on an image using a convolutional neural network, wherein the image is input to the convolutional neural network; outputting a first feature map by a first convolutional layer of the convolutional neural network, wherein the first feature map comprises a first control channel, and the first convolutional layer is a middle convolutional layer of the convolutional neural network; the device comprises:
a first control area determining module, configured to use an area, in the first control channel, where a feature value is smaller than a preset threshold as a first control area, where the first control channel is a channel in the first feature map, and a feature value at any position of the first control channel is used to represent a probability that a corresponding position of the first feature map belongs to a target feature;
a redundant area determination module, configured to determine a redundant area of the first feature map according to the first control area;
and the second characteristic diagram obtaining module is used for performing convolution operation on the next convolution layer of the first convolution layer after the other areas except the redundant area in the first characteristic diagram enter the next convolution layer of the first convolution layer to obtain a second characteristic diagram.
8. An electronic system, characterized in that the electronic system comprises: the device comprises an image acquisition device, a processing device and a storage device;
the image acquisition equipment is used for acquiring an image;
the storage means has stored thereon a computer program which, when executed by the processing apparatus, performs the method of convolution calculation of an image using a convolutional neural network as claimed in any one of claims 1 to 6.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processing device, carries out the steps of the method of carrying out convolution calculations on an image using a convolutional neural network as claimed in any one of claims 1 to 6.
CN202010775401.XA 2020-02-24 2020-08-04 Method and apparatus for performing convolution calculation on image using convolution neural network Pending CN111967579A (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010114390.0A CN111414992A (en) 2020-02-24 2020-02-24 Method and apparatus for performing convolution calculation on image using convolution neural network
CN2020101143900 2020-02-24

Publications (1)

Publication Number Publication Date
CN111967579A true CN111967579A (en) 2020-11-20

Family

ID=71492789

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202010114390.0A Pending CN111414992A (en) 2020-02-24 2020-02-24 Method and apparatus for performing convolution calculation on image using convolution neural network
CN202010775401.XA Pending CN111967579A (en) 2020-02-24 2020-08-04 Method and apparatus for performing convolution calculation on image using convolution neural network

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202010114390.0A Pending CN111414992A (en) 2020-02-24 2020-02-24 Method and apparatus for performing convolution calculation on image using convolution neural network

Country Status (1)

Country Link
CN (2) CN111414992A (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112116066B (en) * 2020-08-27 2022-12-20 苏州浪潮智能科技有限公司 Neural network computing method, system, device and medium

Also Published As

Publication number Publication date
CN111414992A (en) 2020-07-14

Similar Documents

Publication Publication Date Title
CN109255352B (en) Target detection method, device and system
US11798132B2 (en) Image inpainting method and apparatus, computer device, and storage medium
CN109829448B (en) Face recognition method, face recognition device and storage medium
CN110929569B (en) Face recognition method, device, equipment and storage medium
CN109829506B (en) Image processing method, image processing device, electronic equipment and computer storage medium
CN110991311A (en) Target detection method based on dense connection deep network
CN110084299B (en) Target detection method and device based on multi-head fusion attention
CN112001932B (en) Face recognition method, device, computer equipment and storage medium
CN111597884A (en) Facial action unit identification method and device, electronic equipment and storage medium
CN110807362A (en) Image detection method and device and computer readable storage medium
WO2023151237A1 (en) Face pose estimation method and apparatus, electronic device, and storage medium
TW202217662A (en) Visual positioning method, training method of related models, electronic device and computer-readable storage medium
CN112836625A (en) Face living body detection method and device and electronic equipment
CN113850136A (en) Yolov5 and BCNN-based vehicle orientation identification method and system
CN111291760A (en) Semantic segmentation method and device for image and electronic equipment
CN112528845A (en) Physical circuit diagram identification method based on deep learning and application thereof
CN114694158A (en) Extraction method of structured information of bill and electronic equipment
CN113487610B (en) Herpes image recognition method and device, computer equipment and storage medium
CN111160107B (en) Dynamic region detection method based on feature matching
CN109961103B (en) Training method of feature extraction model, and image feature extraction method and device
WO2022222036A1 (en) Method and apparatus for determining parking space
CN112784494B (en) Training method of false positive recognition model, target recognition method and device
CN111382791B (en) Deep learning task processing method, image recognition task processing method and device
CN111340139B (en) Method and device for judging complexity of image content
CN111967579A (en) Method and apparatus for performing convolution calculation on image using convolution neural network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination