CN111079643B - Face detection method and device based on neural network and electronic equipment - Google Patents

Abstract

The embodiment of the invention provides a face detection method, a face detection device and electronic equipment based on a neural network, which relate to the field of face recognition, and the method comprises the following steps: determining a face image to be detected, and determining at least one first input feature map based on the face image, wherein m sub-regions are preset in each first input feature map; for each subarea of each input characteristic diagram, performing characteristic extraction according to a preset small convolution kernel to obtain a first output characteristic diagram; for each input characteristic diagram, accumulating the first output characteristic diagrams corresponding to the input characteristic diagrams to obtain a second output characteristic diagram; and performing face recognition based on a second output feature map of the at least one input feature map. The method can have better GPU acceleration experience under the condition of not increasing the precision loss, improves the detection efficiency, and solves the problems that in the prior art, the calculation amount is increased and the detection efficiency is reduced due to the increase of the data amount.

Description

Face detection method, device and electronic equipment based on neural network

technical field

The present invention relates to the field of face detection, in particular to a neural network-based face detection method, device and electronic equipment.

Background technique

The face detection algorithm is a key part of the face recognition comparison verification system. With the development of deep learning technology, the current face detection algorithm is implemented on a large scale using convolutional neural networks.

As the diversity and complexity of scenes increase, the number of training data sets increases, and the scale of convolutional neural networks becomes larger and larger, requiring more calculations and memory access; this inevitably leads to a decrease in detection speed. Even with graphics processing unit (GPU) acceleration, it is difficult to meet the requirements of real-time detection.

Contents of the invention

In view of this, the object of the present invention is to provide a neural network-based face detection method, device and electronic equipment.

In order to achieve the above object, the technical solution adopted in the embodiment of the present invention is as follows:

In the first aspect, the embodiment of the present invention provides a neural network-based face detection method, including:

Determining the face image to be detected, and determining at least one first input feature map based on the face image, each of the first input feature maps is preset with m sub-regions;

For each of the sub-regions of each of the input feature maps, feature extraction is performed according to a preset small convolution kernel to obtain a first output feature map;

For each of the input feature maps, accumulating the first output feature maps corresponding to the input feature maps to obtain a second output feature map;

Face recognition is performed based on a second output feature map of at least one of the input feature maps.

In an optional implementation manner, the preset small convolution kernel is a 3*3 small convolution kernel, and the m is 9; the first output feature map corresponding to the input feature map is accumulated to obtain the first Two steps of outputting the feature map, including:

The nine first output feature maps corresponding to the input feature map are accumulated pixel by pixel to obtain a second output feature map that is the same as the output result of the 7*7 convolution kernel.

In an optional implementation manner, there are nine small convolution kernels, and each of the small convolution kernels corresponds to a start coordinate and an end coordinate of one of the nine sub-regions.

In an optional implementation manner, the step of accumulating the first output feature map corresponding to the input feature map to obtain the second output feature map includes:

In the Eltwise layer, the nine first output feature maps corresponding to the input feature map are accumulated pixel by pixel to obtain a second output feature map that is the same as the output result of the 7*7 convolution kernel.

In an optional implementation manner, the step of performing face recognition based on at least one second output feature map of the input feature map includes:

Perform fixed-size nearest neighbor expansion upsampling on the second output feature map of at least one of the input feature maps on the GPU acceleration engine to obtain at least one third input feature map;

Face recognition is performed based on at least one of the third input feature maps.

In a second aspect, an embodiment of the present invention provides a neural network-based face detection device, including:

A determining module, configured to determine a face image to be detected, and determine at least one first input feature map based on the face image, each of the first input feature maps is preset with m sub-regions;

An extraction module, configured to perform feature extraction according to a preset small convolution kernel for each of the sub-regions of each of the input feature maps to obtain a first output feature map;

An accumulation module, for each of the input feature maps, accumulating the first output feature maps corresponding to the input feature maps to obtain a second output feature map;

A detection module, configured to perform face recognition based on a second output feature map of at least one of the input feature maps.

In an optional implementation manner, the preset small convolution kernel is a 3*3 small convolution kernel, and the m is 9; An output feature map is accumulated pixel by pixel to obtain a second output feature map that is the same as the output result of the 7*7 convolution kernel.

In an optional implementation manner, there are nine small convolution kernels, and each of the small convolution kernels corresponds to a start coordinate and an end coordinate of one of the nine sub-regions.

In a third aspect, an embodiment of the present invention provides an electronic device, including a processor and a memory, the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions In order to realize the method described in any one of the foregoing embodiments.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the foregoing implementation manners is implemented.

The neural network-based face detection method, device, electronic device, and computer-readable storage medium provided by the embodiments of the present invention, wherein the method includes: determining a face image to be detected, and determining at least one face image based on the face image The first input feature map, each of the first input feature maps is preset with m sub-regions; for each of the sub-regions of each of the input feature maps, feature extraction is performed according to a preset small convolution kernel, Obtain a first output feature map; for each of the input feature maps, accumulate the first output feature maps corresponding to the input feature maps to obtain a second output feature map; based on at least one of the input feature maps the second Output feature maps for face recognition. Therefore, in the technical solution provided by the embodiment of the present invention, by improving the original detection network, the preset small convolution kernel is used to extract the features of the sub-regions of the input feature map, and then accumulate them, and finally perform face detection according to the accumulated output results. Recognition, so that it can have a better GPU acceleration experience without increasing the loss of precision, improve the detection efficiency, and alleviate the problems in the prior art that the calculation amount increases and the detection efficiency decreases due to the increase in the amount of data.

In order to make the above-mentioned objects, features and advantages of the present invention more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and thus It should be regarded as a limitation on the scope, and those skilled in the art can also obtain other related drawings based on these drawings without creative work.

Fig. 1 shows a flow chart of a neural network-based face detection method provided by an embodiment of the present invention;

Fig. 2 shows the execution scenario diagram of step S106 in Fig. 1;

Fig. 3 shows the specific flowchart of step S108 in Fig. 1;

Fig. 4 shows a schematic diagram of a neural network-based face detection device provided by an embodiment of the present invention;

Fig. 5 shows a schematic diagram of an electronic device provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. The components of the embodiments of the invention generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations.

Accordingly, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts belong to the protection scope of the present invention.

It should be noted that relative terms such as the terms "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. There is no such actual relationship or order between them. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Example 1

Please refer to Fig. 1, the embodiment of the present invention provides a kind of face detection method based on neural network, this method comprises:

Step S102, determine the face image to be detected, and determine at least one first input feature map based on the face image, each first input feature map is preset with m sub-regions;

Step S104, for each sub-region of each input feature map, perform feature extraction according to a preset small convolution kernel to obtain a first output feature map;

Step S106, for each input feature map, accumulating the first output feature map corresponding to the input feature map to obtain a second output feature map;

Step S108, performing face recognition based on the second output feature map of the at least one input feature map.

In the neural network-based face detection method provided by the embodiment of the present invention, by determining the face image to be detected, and determining at least one first input feature map based on the face image, each of the first input feature maps is pre-determined There are m sub-regions; then for each of the sub-regions of each of the input feature maps, feature extraction is performed according to a preset small convolution kernel to obtain a first output feature map; then for each of the input features , accumulating the first output feature maps corresponding to the input feature maps to obtain a second output feature map; finally performing face recognition based on the second output feature map of at least one of the input feature maps. This method can have a better GPU acceleration experience without increasing the loss of precision, improves the detection efficiency, and alleviates the problems in the prior art that the calculation amount increases and the detection efficiency decreases due to the increase in the amount of data.

In a possible implementation manner, the preset small convolution kernel is a 3*3 small convolution kernel, and the m is 9;

For step S106, the first output feature map corresponding to the input feature map is accumulated to obtain the second output feature map, including:

1. Accumulate the nine first output feature maps corresponding to the input feature map pixel by pixel to obtain a second output feature map that is the same as the output result of the 7*7 convolution kernel.

In a possible implementation manner, there are nine small convolution kernels, and each of the small convolution kernels corresponds to a start coordinate and an end coordinate of one of the nine sub-regions.

In a possible implementation manner, in step S106, the first output feature map corresponding to the input feature map is accumulated to obtain the second output feature map, which can be performed by the following steps:

(1) In the Eltwise layer, the nine first output feature maps corresponding to the input feature map are accumulated pixel by pixel to obtain the second output feature map that is the same as the output result of the 7*7 convolution kernel.

For ease of understanding, the execution scenario of step S106 is briefly described below in conjunction with FIG. 2:

In Figure 2, NewConv (new convolution kernel) is a 3x3 small convolution kernel, which is a new area-defined convolution operator: the small convolution kernel acts on the sub-area of the input feature map (feature map), which is equivalent to Increase the start coordinates and end coordinates of Conv (Convolution abbreviation, convolution), such as first expanding (split) the 7x7 convolution kernel (Kernel Convolution) into nine 3x3 small convolution kernels, each small convolution kernel The sub-areas of the function are different, and then the output feature map (first output feature map) of the 9 small convolution kernels is accumulated pixel by pixel (pixel) through the Eltwise layer to achieve the same output result as the 7x7 convolution kernel. That is, the second output feature map is the same.

In a possible implementation manner, referring to FIG. 3, for step S108, performing face recognition based on the second output feature map of at least one input feature map includes the following steps:

Step S302, performing fixed-size nearest neighbor extended upsampling on the second output feature map of at least one input feature map on the GPU acceleration engine to obtain at least one third input feature map;

Step S304, performing face recognition based on at least one of the third input feature maps.

The traditional bilinear interpolation upsampling of different sizes of feature maps needs to be re-implemented in the GPU acceleration engine, and the calculation load is high.

In order to solve the above problems, in the embodiment of the present invention, the method will adopt the fixed-size 2 times nearest neighbor expansion upsampling method: since the deep feature map that needs to be expanded in size is approximately half the size of the target shallow feature map, it only needs Double the scale of the deep feature map, and each pixel (pixel) in the new feature map can be mapped back to the nearest neighbor pixel (pixel) value of the original feature map by the following formula:

NewPixel(x,y)=SrcPixel(int(x/2),int(y/2));

Among them, x and y represent the coordinates. Since the nearest neighbor extended upsampling adopted by this method only involves GPU memory read and write operations and very simple coordinate calculations, compared with traditional bilinear interpolation upsampling to generate new feature map pixels, the nearest neighbor extended upsampling There is no need for multiplication and addition operations on the original pixel values, and a better acceleration effect has been achieved on the GPU acceleration engine, and the detection accuracy is higher.

The face detection based on the neural network provided by the embodiment of the present invention, combined with the operating characteristics of the GPU acceleration engine, explores the operations in the neural network structure that are relatively slow and the acceleration is not obvious. By proposing a region selection convolution operator, the large convolution By splitting the kernel into multiple small convolution kernels and superimposing them with Elitwise, the running accuracy of the large convolution kernel can be achieved under better GPU acceleration; at the same time, according to the scale relationship of the face detection feature map, the method uses Simpler fixed-size nearest-neighbor extended upsampling is better suited for GPU acceleration. In addition, this method has a better face detection effect in actual scenes while obtaining better acceleration effects, but the GPU utilization rate in this method is higher, and it can support more channels of video data deployment.

Example 2

Based on the same inventive concept, the embodiment of the present application also provides a neural network-based face detection device corresponding to the neural network-based face detection method. Since the problem-solving principle of the device in the embodiment of the present application is the same as that of the embodiment of the present application The above neural network-based face detection method is similar, so the implementation of the device can refer to the implementation of the method, and the repetition will not be repeated.

FIG. 4 is a schematic diagram of a neural network-based face detection device provided by an embodiment of the present application.

Referring to FIG. 4, the device includes: a determination module 401, an extraction module 402, an accumulation module 403, and a detection module 404;

Wherein, the determining module 401 is configured to determine the face image to be detected, and determine at least one first input feature map based on the face image, and each of the first input feature maps is preset with m sub-regions;

An extraction module 402, configured to perform feature extraction according to a preset small convolution kernel for each of the sub-regions of each of the input feature maps to obtain a first output feature map;

An accumulation module 403, configured to, for each of the input feature maps, accumulate the first output feature maps corresponding to the input feature maps to obtain a second output feature map;

A detection module 404, configured to perform face recognition based on a second output feature map of at least one of the input feature maps.

In an optional implementation manner, the preset small convolution kernel is a 3*3 small convolution kernel, and the m is 9; the accumulation module 403 is used to combine the 9 corresponding input feature maps The first output feature map is accumulated pixel by pixel to obtain the second output feature map that is the same as the output result of the 7*7 convolution kernel.

In an optional implementation manner, there are nine small convolution kernels, and each of the small convolution kernels corresponds to a start coordinate and an end coordinate of one of the nine sub-regions.

In an optional implementation manner, the accumulation module 403 is used to accumulate the 9 first output feature maps corresponding to the input feature map pixel by pixel at the Eltwise layer to obtain the output result of the convolution kernel of 7*7 The same second output feature map.

In an optional implementation manner, the detection module 404 is configured to perform fixed-size nearest neighbor extended upsampling on the second output feature map of at least one input feature map on the GPU acceleration engine to obtain at least one third input feature map ; Perform face recognition based on at least one of the third input feature maps.

The neural network-based face detection device provided in the embodiment of the present application has the same technical features as the neural network-based face detection method provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effect.

Referring to FIG. 5, an embodiment of the present invention also provides an electronic device 100, including:

Processor 41, memory 42, and bus 43; Memory 42 is used for storing execution order, comprises memory 421 and external memory 422; Memory 421 here is also called internal memory, is used for temporarily storing the operation data in processor 41, and with The data exchanged by the external memory 422 such as hard disk, the processor 41 exchanges data with the external memory 422 through the memory 421, when the electronic device 100 is running, the processor 41 communicates with the memory 42 through the bus 43, so that The processor 41 executes the following instructions in the user mode:

Determine the face image to be detected, and determine at least one first input feature map based on the face image, each of the first input feature maps is preset with m sub-regions; for each of each of the input feature maps For each of the sub-regions, feature extraction is performed according to a preset small convolution kernel to obtain a first output feature map; for each of the input feature maps, the first output feature map corresponding to the input feature map is accumulated, Obtaining a second output feature map; performing face recognition based on the second output feature map of at least one input feature map.

Optionally, the preset small convolution kernel is a 3*3 small convolution kernel, and the m is 9; the first output feature map corresponding to the input feature map is accumulated to obtain a second output feature map The step includes: accumulating the 9 first output feature maps corresponding to the input feature map pixel by pixel to obtain a second output feature map that is the same as the output result of the 7*7 convolution kernel.

Optionally, in the instructions executed by the processor 41, there are nine small convolution kernels, and each of the small convolution kernels corresponds to the start coordinates and end coordinates of one of the nine sub-regions.

Optionally, in the instructions executed by the processor 41, the step of accumulating the first output feature map corresponding to the input feature map to obtain the second output feature map includes: at the Eltwise layer, corresponding to the input feature map The nine first output feature maps of are accumulated pixel by pixel to obtain the second output feature map that is the same as the output result of the 7*7 convolution kernel.

Optionally, in the instructions executed by the processor 41, the step of performing face recognition based on the second output feature map of at least one of the input feature maps includes: performing the second output feature map of at least one of the input feature maps on the GPU acceleration engine. The second output feature map is subjected to fixed-size nearest neighbor expansion up-sampling to obtain at least one third input feature map; face recognition is performed based on at least one third input feature map.

An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the method provided by the above-mentioned embodiments are executed.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are only illustrative. For example, the flowcharts and structural diagrams in the accompanying drawings show the possible implementation architecture and functions of devices, methods and computer program products according to multiple embodiments of the present invention. and operation. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or part of code that includes one or more Executable instructions. It should also be noted that, in alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also to be noted that each block of the block diagrams and/or flow diagrams, and combinations of blocks in the block diagrams and/or flow diagrams, can be implemented by a dedicated hardware-based system that performs the specified function or action may be implemented, or may be implemented by a combination of special purpose hardware and computer instructions.

In addition, each functional module or unit in each embodiment of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a smart phone, a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. Should be covered within the protection scope of the present invention.

Claims (8)

1. A face detection method based on a neural network is characterized by comprising the following steps:

determining a face image to be detected, and determining at least one input feature map based on the face image, wherein each input feature map is preset with m sub-regions;

for each sub-region of each input feature map, performing feature extraction according to a preset small convolution kernel to obtain a first output feature map; wherein each of the small convolution kernels acts on a different one of the sub-regions of the input feature map, each of the small convolution kernels corresponding to a start coordinate and an end coordinate of the corresponding sub-region;

for each input characteristic diagram, performing pixel-by-pixel accumulation on first output characteristic diagrams corresponding to m sub-regions of the input characteristic diagram to obtain a second output characteristic diagram;

and performing face recognition based on a second output feature map of at least one input feature map.

2. The method according to claim 1, wherein the preset small convolution kernel is 3x3 small convolution kernel, and m is 9; accumulating the first output characteristic diagram corresponding to the input characteristic diagram to obtain a second output characteristic diagram, wherein the step comprises the following steps of:

and accumulating the 9 first output feature maps corresponding to the input feature map pixel by pixel to obtain a second output feature map which is the same as the output result of the 7-by-7 convolution kernel.

3. The method of claim 2, wherein the step of accumulating the first output feature map corresponding to the input feature map to obtain a second output feature map comprises:

and accumulating the 9 first output characteristic graphs corresponding to the input characteristic graphs pixel by pixel in an Eltwise layer to obtain a second output characteristic graph which is the same as the output result of the convolution kernel of 7 by 7.

4. The method of claim 1, wherein the step of performing face recognition based on a second output feature map of the at least one input feature map comprises:

performing fixed-size nearest neighbor extension upsampling on a second output feature map of at least one input feature map on a GPU acceleration engine to obtain at least one third input feature map;

and performing face recognition based on at least one third input feature map.

5. A face detection device based on a neural network is characterized by comprising:

the determining module is used for determining a face image to be detected and determining at least one input feature map based on the face image, wherein m sub-regions are preset in each input feature map;

the extraction module is used for extracting the characteristics of each subarea of each input characteristic diagram according to a preset small convolution kernel to obtain a first output characteristic diagram; wherein each of the small convolution kernels acts on a different one of the sub-regions of the input feature map, each of the small convolution kernels corresponding to a start coordinate and an end coordinate of the corresponding sub-region;

the accumulation module is used for accumulating the first output characteristic diagrams corresponding to the m sub-regions of the input characteristic diagram pixel by pixel to obtain a second output characteristic diagram;

and the detection module is used for carrying out face recognition based on a second output feature map of the at least one input feature map.

6. The apparatus of claim 5, wherein the preset small convolution kernel is 3x3 small convolution kernel, and m is 9; and the accumulation module is used for accumulating the 9 first output characteristic graphs corresponding to the input characteristic graphs pixel by pixel to obtain a second output characteristic graph which is the same as the output result of the 7-by-7 convolution kernel.

7. An electronic device comprising a processor and a memory, the memory storing machine executable instructions executable by the processor to implement the method of any one of claims 1-4.

8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-4.

Images