CN111860276B - Human body key point detection method, device, network equipment and storage medium - Google Patents

Abstract

The embodiment of the invention relates to the technical field of computer vision, and discloses a human body key point detection method, which is applied to a pre-trained key point detection model and comprises the following steps: extracting a first feature map of a target human body image; performing deconvolution of a plurality of layers according to the first feature map to obtain a second feature map, wherein when performing deconvolution of the next layer, the first feature map and the third feature map which have the same resolution as that of the third feature map obtained by deconvolution of the previous layer are combined and then subjected to deconvolution; outputting thermodynamic diagrams of all human body key points of the target human body image according to the second characteristic diagram, and determining all human body key points of the target human body image according to the thermodynamic diagrams. The embodiment of the invention also provides a human body key point detection device, network equipment and a storage medium. The human body key point detection method, the human body key point detection device, the network equipment and the storage medium can improve the detection precision of the human body key point and the accuracy of a human body posture estimation result.

Description

Human key point detection method, device, network equipment and storage medium

technical field

The invention relates to the technical field of computer vision, in particular to a human body key point detection method, device, network equipment and storage medium.

Background technique

Human pose estimation is an important research direction in the field of computer vision, and is widely used in human activity analysis, human-computer interaction, and video surveillance. Human pose estimation refers to the estimation of human pose by locating key points of the human body (such as shoulders, elbows, wrists, knees, and ankles, etc.) in images or videos by computer algorithms.

However, the inventors have found at least the following problems in the prior art: the accuracy of the current detection method for key points of the human body is not high, which may easily cause output errors of the key points of the human body and affect the result of human body pose estimation.

Contents of the invention

The purpose of the embodiment of the present invention is to provide a human body key point detection method, device, network equipment and storage medium, so that the detection accuracy of the human body key point and the accuracy of the human body posture estimation result are improved.

In order to solve the above technical problems, the embodiment of the present invention provides a human body key point detection method, which is applied to the pre-trained key point detection model, including: using the first sub-module of the key point detection model to extract the first part of the target human body image The feature map, the first feature map includes multiple groups of feature maps with different resolutions; the second sub-module of the key point detection model is used to perform deconvolution of several layers according to the first feature map to obtain the second feature map, wherein, after performing In the deconvolution of the next layer, the second sub-module is used to deconvolute the first feature map and the third feature map with the same resolution as the third feature map obtained by deconvolution of the previous layer; use the key The third sub-module of the point detection model outputs a heat map of each human key point of the target human body image according to the second feature map, and determines each human body key point of the target human body image according to the heat map.

The embodiment of the present invention also provides a human body key point detection device, including: an extraction module, used to extract the first feature map of the target human body image, the first feature map includes multiple groups of feature maps with different resolutions; deconvolution The module is used to perform deconvolution of several layers according to the first feature map to obtain the second feature map, wherein, when performing the deconvolution of the next layer, the third feature obtained by the deconvolution of the previous layer of resolution Deconvolution is performed after the first feature map and the third feature map with the same figure are combined; the detection module is used to output the heat map of each key point of the target human body image according to the second feature map, and determine the target human body image according to the heat map key points of each human body.

Embodiments of the present invention also provide a network device, including: at least one processor; and a memory connected to the at least one processor in communication; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by at least one processor. Executed by one processor, so that at least one processor can execute the above-mentioned human body key point detection method.

Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned human body key point detection method is realized.

Compared with the prior art, the embodiment of the present invention uses the key point detection model to extract multiple sets of first feature maps of different resolutions of the target human body image; deconvolution of several layers is performed according to the first feature map to obtain the second feature map , wherein, when deconvolution of the next layer is performed, deconvolution is performed after combining the first feature map and the third feature map with the same resolution as the third feature map obtained by deconvolution of the previous layer; according to the first The second feature map outputs a heat map of each key point of the human body image of the target human body, and determines each key point of the human body of the target human body image according to the heat map. Since the information of the third feature map and the information of the combined first feature map can complement each other, deconvolution after combining the two feature maps can enable the key point detection model to learn more human key points in the target human body image. Point information, thereby effectively improving the detection accuracy of human key points and the accuracy of human pose estimation results.

In addition, using the second sub-module of the key point detection model to perform deconvolution of several layers according to the first feature map, including: when performing deconvolution of each layer, using the second sub-module to integrate the deconvolution according to the number of channels for several groups. When deconvolution of each layer is performed, the deconvolution of each layer is divided into several groups according to the number of channels, so that the calculation amount of deconvolution can be greatly reduced on the original basis.

In addition, using the second sub-module of the key point detection model to perform deconvolution of several layers according to the first feature map, including: when performing deconvolution of the first layer, using the second sub-module according to the first minimum resolution The feature maps are deconvolved.

In addition, the third sub-module of the key point detection model is used to output the heat map of each human body key point of the target human body image according to the second feature map, including: using the third sub-module to group the second feature map according to the spatial relationship of human limbs According to the result of group convolution, the heat map of each key point of the target human body image is output. Since the limbs of the same part of the human body have a relatively fixed spatial relationship in the target human body image, and the features of the limbs of the same part in the target human body image are relatively similar, the second feature map is grouped and convolved according to the spatial relationship of the human limbs , which can make the key point detection models learn from each other and refer to the spatial position and image features of the same limb in the same group of convolutions, thereby effectively improving the detection accuracy of human key points.

In addition, use the third sub-module to perform group convolution on the second feature map according to the spatial relationship of human limbs, specifically: use the third sub-module to perform group convolution on the second feature map according to the preset key point grouping, the key point grouping The second feature map of each group includes the second feature map of the first human body key points that are in adjacent positions and belong to the same limb position of the human body. The first human body key points are key points predefined according to the spatial relationship of human body limbs.

In addition, using the first sub-module of the key point detection model to extract the first feature map of the target human body image is specifically: using the first sub-module to obtain the first feature map of the target human body image according to a lightweight convolutional neural network. Since the lightweight convolutional neural network can reduce the computing power requirements of the key point detection model, the acquisition of the first feature map according to the lightweight convolutional neural network can reduce the overall impact of the key point detection model on the device running the model. Computing power required.

In addition, using the first submodule to obtain the first feature map of the target human body image according to the lightweight convolutional neural network includes: using the first submodule to reduce the number of channels of the last convolutional layer of the lightweight convolutional neural network To the preset number, the first feature map of the target human body image is obtained according to the lightweight convolutional neural network after reducing the number of channels. By setting the preset number reasonably and reducing the channel number of the last convolutional layer of the lightweight convolutional neural network to the preset number, the key point detection model can be reduced in weight while the detection effect is not much different. The computational load of the level convolutional neural network, thereby further reducing the computing power requirements of the key point detection model on the device running the model.

Description of drawings

One or more embodiments are exemplified by pictures in the accompanying drawings, and these exemplifications are not intended to limit the embodiments.

FIG. 1 is a schematic flow diagram of a human body key point detection method provided in the first embodiment of the present invention;

Fig. 2 is a schematic diagram of the principle of the human body key point detection method provided by the first embodiment of the present invention;

Fig. 3 is a schematic flow chart of a human body key point detection method provided by the second embodiment of the present invention;

Fig. 4 is a schematic flow chart of a human body key point detection method provided by the third embodiment of the present invention;

Fig. 5 is a detection example diagram of the human body key point detection method provided by the third embodiment of the present invention;

Fig. 6 is a schematic diagram of the module structure of the human body key point detection device provided by the fourth embodiment of the present invention;

Fig. 7 is a schematic structural diagram of a network device provided in a fifth embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, various embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. However, those of ordinary skill in the art can understand that, in each implementation manner of the present invention, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following implementation modes, the technical solution claimed in this application can also be realized.

The first embodiment of the present invention relates to a human body key point detection method, using the first sub-module of the key point detection model to extract the first feature map of the target human body image, wherein the first feature map includes multiple sets of features with different resolutions Figure; use the second sub-module of the key point detection model to perform several layers of deconvolution according to the first feature map to obtain the second feature map, wherein, when performing the deconvolution of the next layer, use the second sub-module to The first feature map and the third feature map with the same resolution as the third feature map obtained by the previous layer of deconvolution are combined for convolution; the third sub-module of the key point detection model is used to output the target human body according to the second feature map The heat map of each key point of the human body in the image, and determine each key point of the target human body image according to the heat map. By linking the first feature map with the same resolution as the third feature map obtained by the previous layer of deconvolution to the third feature map, the two feature maps are combined for the next layer of deconvolution, due to the two features The information in the graph can complement each other, so the key point detection model can learn more information about the key points of the human body in the target human body image during the deconvolution process, thereby improving the detection accuracy of human key points and the accuracy of human body pose estimation. sex.

It should be noted that the human body key point detection method provided in the embodiment of the present invention is applied to a pre-trained key point detection model, that is, the key point detection model is executed as the main body. It can be understood that the first sub-module, the second sub-module and the third sub-module of the key point detection model are components of the key point detection model. Optionally, the key point detection model may also include other components (submodules), which are not specifically limited here. Optionally, the key point detection model can use data in RGB format during training, and the learning rate, training cycle, batch size, etc. adopted by the key point detection model during training can be set according to actual needs. This is not specifically limited.

The specific flow of the human body key point detection method provided by the embodiment of the present invention is shown in Figure 1, and specifically includes the following steps:

S101: Using the first sub-module of the key point detection model to extract a first feature map of a target human body image, where the first feature map includes multiple groups of feature maps with different resolutions.

Among them, the target human body image refers to the human body image of the key points of the human body to be detected, which can be obtained by using the target detection method, for example, using the target detection method to obtain the target human body image from the frame image of the video, and the specific target detection method can be carried out according to actual needs The setting is not specifically limited here.

Please refer to FIG. 2 , which is a schematic diagram of the principles of the human body key point detection method provided by the embodiment of the present invention. Specifically, the first sub-module of the key point detection model is used to extract the first feature map of the target human body image, corresponding to the feature extraction part in the figure; the second sub-module is used to perform deconvolution according to the first feature map, corresponding to the The deconvolution part in ; the third sub-module is used to output the heat map of the key points of the human body, corresponding to the output heat map part in the figure.

Optionally, the first sub-module may specifically use a sub-neural network to extract features of the target human body image, wherein the sub-neural network is, for example, a convolutional neural network, and the first sub-module of the target human body image is extracted through a convolutional layer of the convolutional neural network. A feature map, the specific sub-neural network used can be set according to actual needs, and is not specifically limited here. By setting the size of the convolution kernel of each layer in the sub-neural network to different sizes, multiple sets of first feature maps with different resolutions can be obtained, such as 64*48, 32*24, 16*12 and other different resolutions , the size of the convolution kernel of each layer can be set according to actual needs, and is not specifically limited here.

S102: Use the second sub-module of the key point detection model to perform several layers of deconvolution according to the first feature map to obtain the second feature map, wherein, when performing the deconvolution of the next layer, use the second sub-module to deconvolute The first feature map and the third feature map with the same resolution as the third feature map obtained by deconvolution of the previous layer are combined for deconvolution.

As shown in Figure 2, G00/G10 represents the deconvolution of the first layer, G01/G11 represents the deconvolution of the second layer, and G02/G12 represents the deconvolution of the third layer. In practical applications, the number of deconvolution layers of the keypoint detection model can be set as needed. It is understandable that when the number of deconvolution layers is more, the detection accuracy of key points of the human body is higher, but the corresponding calculation amount will be larger. Therefore, the actual number of deconvolution layers can be determined according to the application scenario and /or the computing power of the device running the model should be reasonably set.

In Figure 2, taking G01 as an example, the left (dark) block is the first feature map copied directly from the extracted first feature map, and the right (light) block is the deconvolution of the previous layer The obtained third feature map, in which the resolution of the left block is the same as that of the right block, for example, the resolution of the right block in G01 is 32*24, then the resolution copied from the first feature map is 32* The first feature map of 24 is used as the content of the left box. When performing the deconvolution of G02 (the next layer), deconvolution is performed after combining the left block and the right block in G01. Specifically, the first feature map corresponding to the left block and the third feature map corresponding to the right block can be used. The feature maps are placed side by side, and then the side-by-side feature maps are deconvolved using the deconvolution kernel.

In a specific example, S102 includes: when performing the deconvolution of the first layer, the key point detection model uses the second sub-module to perform deconvolution according to the first feature map with the minimum resolution.

Continuing to refer to Figure 2, when performing deconvolution of G00 (first layer), directly use the longest block on the left in Figure 2 (ie the first feature map with the smallest resolution) for deconvolution. Optionally, the first feature map with the smallest resolution is the feature map extracted by the last convolutional layer of the sub-neural network of the first sub-module.

S103: Using the third sub-module of the key point detection model to output a heat map of each human body key point of the target human body image according to the second feature map, and determine each human body key point of the target human body image according to the heat map.

Wherein, the number of key points of the human body can be set according to the actual situation, for example, 14, 16 or 63, etc., which is not specifically limited here.

Optionally, the key point detection model uses the third sub-module to convolve the second feature map obtained by the second sub-module according to the number of key points of the human body, and outputs a heat map with the same number of key points of the human body, wherein, Each heat map corresponds to a key point of the human body; then the key point detection model determines all key points of the human body in the target human body image based on all the heat maps. Optionally, when the third sub-module performs convolution on the second feature map, the size of the convolution kernel is 1*1.

Compared with the prior art, the human body key point detection method provided by the embodiment of the present invention uses the key point detection model to extract multiple sets of first feature maps of different resolutions of the target human body image; performs several layers of deconvolution according to the first feature map product to obtain the second feature map, wherein, when performing the deconvolution of the next layer, the first feature map and the third feature map with the same resolution as the third feature map obtained by the previous layer of deconvolution are combined performing deconvolution; outputting a heat map of each human key point of the target human body image according to the second feature map, and determining each human body key point of the target human body image according to the heat map. Since the information of the third feature map and the information of the combined first feature map can complement each other, deconvolution after combining the two feature maps can enable the key point detection model to learn more human key points in the target human body image. Point information, thereby effectively improving the detection accuracy of human key points and the accuracy of human pose estimation results.

The second embodiment of the present invention relates to a method for detecting key points of a human body. The second embodiment is roughly the same as the first embodiment, and the main difference is that: using the second sub-module of the key point detection model to perform deconvolution of several layers according to the first feature map, including: performing deconvolution of each layer During convolution, use the second sub-module to integrate the deconvolution into several groups according to the number of channels. By dividing the deconvolution into several groups, the calculation amount of the key point detection model can be greatly reduced, so that the key point detection model can be run on mobile terminals with low computing power, and the application range of the key point detection method of the human body can be broadened.

The specific flow of the human body key point detection method provided by the embodiment of the present invention is shown in Figure 3, which specifically includes the following steps:

S201: Using the first sub-module of the key point detection model to extract a first feature map of the target human body image, where the first feature map includes multiple groups of feature maps with different resolutions.

S202: Use the second submodule of the key point detection model to perform several layers of deconvolution according to the first feature map to obtain a second feature map, wherein, when performing deconvolution of the next layer, use the second submodule to deconvolute The third feature map obtained by the deconvolution of the previous layer of resolution is the same as the first feature map and the third feature map are combined for deconvolution. When performing deconvolution of each layer, the second sub-module is used according to the channel The number will be deconvoluted into groups.

S203: Using the third sub-module of the key point detection model to output a heat map of each human body key point of the target human body image according to the second feature map, and determine each human body key point of the target human body image according to the heat map.

Wherein, S201 and S203 are respectively the same as S101 and S103 in the first embodiment. For details, please refer to the description in the first embodiment. In order to avoid repetition, details are not repeated here.

For S202, it can be understood that in order to output the heat map of multiple key points of the human body, it is necessary to use multiple convolution kernels to deconvolute the first feature map, and the number of convolution kernels corresponds to the number of channels. Therefore, in There are multiple channels in the deconvolution of each layer, and the key point detection model can divide the deconvolution into several groups according to the number of channels. Wherein, the number of groups of each layer of deconvolution can be set according to actual needs, for example, divided into 2 groups, 4 groups or 8 groups, etc., which are not specifically limited here. Optionally, the number of groups of each layer of deconvolution may be the same or different, for example, the number of groups of G00 and G01 in FIG. 2 may be the same or different. In order to facilitate the deconvolution corresponding to the front and rear layers, the number of groups of each layer can be set to the same number of groups, for example, divided into 2 groups.

In order to evaluate the computational complexity of deconvolution, the computational complexity of the i-th layer deconvolution can be calculated using the following formula:

FLOPs _i = C _i-1 *H _i *W _i *C _i *C;

Among them, C represents a constant, which is related to the size of the convolution kernel of deconvolution and the calculation amount of each deconvolution; C _i-1 represents the number of channels input by deconvolution, and C _i represents the number of channels output by deconvolution, H _i and W _i denote the length and width of the feature map output by deconvolution, respectively.

Since H _i and W _i do not change during grouping, and C is a constant, if the deconvolution is divided into two groups, both C _i-1 and C _i are reduced to 1/2 of the original, and the operation of each group The amount is reduced to 1/4 of the original, and the total calculation amount is 1/2 of the original (without grouping). If the deconvolution is grouped into 4 groups, both C _i-1 and C _i are reduced to 1/4 of the original, then the calculation amount of each group is reduced to 1/16 of the original, and the total calculation amount is the original (without grouping) 1/4.

Compared with the prior art, the human body key point detection method provided by the embodiment of the present invention can make the deconvolution operation The amount is greatly reduced on the original basis, so that the key point detection model can improve the detection accuracy of human key points by increasing the model calculation amount (such as increasing the number of deconvolution layers) under the same computing power, or make the key points The detection model improves the speed of detection without increasing the amount of calculation of the model, reduces the computing power requirements of the equipment running the model, enables the key point detection model to run on devices with low computing power (such as mobile terminals), and broadens the scope of key points. The point detection model and the application range of the human body key point detection method.

In order to further reduce the computing power requirement of the key point detection model, in a specific example, in S201, the first feature map of the target human body image is extracted by using the first sub-module of the key point detection model, which may specifically be: using the first The sub-module acquires the first feature map of the target human body image according to the lightweight convolutional neural network.

Among them, the lightweight convolutional neural network can be selected according to actual needs, such as MobileNet V2 or ShuffleNet v2, etc., and there are no specific restrictions here. Since the lightweight convolutional neural network can reduce the computing power requirements of the key point detection model, the acquisition of the first feature map according to the lightweight convolutional neural network can reduce the overall impact of the key point detection model on the device running the model. Computing power required.

In a specific example, using the first sub-module to obtain the first feature map of the target human body image according to the lightweight convolutional neural network can be further refined as: using the first sub-module to convert the lightweight convolutional neural network The number of channels of the last convolutional layer is reduced to a preset number, and the first feature map of the target human body image is obtained according to the lightweight convolutional neural network after the reduced number of channels.

Among them, the preset number can be determined according to the effect of training. If the number of channels in the last convolutional layer of the lightweight convolutional neural network is reduced, the detection effect will be reduced to an acceptable extent compared with that before the reduction. , the reduced number of channels will be used as the preset number. It can be understood that reducing the number of channels to a preset number can further reduce the computation load of the lightweight convolutional neural network. For example, if the lightweight convolutional neural network is MobileNet V2, if the number of channels in the last layer of MobileNet V2 is reduced from 1280 to 160, the detection accuracy of the key point detection model will only decrease by 0.5%, and 160 can be used as the last layer. The preset number of layer channels.

By setting the preset number reasonably and reducing the channel number of the last convolutional layer of the lightweight convolutional neural network to the preset number, the key point detection model can be reduced in weight while the detection effect is not much different. The computational load of the level convolutional neural network, thereby further reducing the computing power requirements of the key point detection model on the device running the model.

The third embodiment of the present invention relates to a method for detecting key points of a human body. The third embodiment is roughly the same as the first embodiment, the main difference is that the third sub-module of the key point detection model is used to output the heat map of each human key point of the target human body image according to the second feature map, including: using the first The third sub-module performs group convolution on the second feature map according to the spatial relationship of human limbs, and outputs the heat map of each key point of the target human body image according to the result of the group convolution. Since the limbs of the same part of the human body have a relatively fixed spatial relationship in the image, the second feature map is grouped and convolved according to the spatial relationship of the human limbs, which can make the detection of the key points of the human body more accurate.

The specific flow of the human body key point detection method provided by the embodiment of the present invention is shown in Figure 4, and specifically includes the following steps:

S301: Using the first sub-module of the key point detection model to extract a first feature map of the target human body image, where the first feature map includes multiple groups of feature maps with different resolutions.

S302: Use the second sub-module of the key point detection model to perform several layers of deconvolution according to the first feature map to obtain the second feature map, wherein, when performing the deconvolution of the next layer, use the second sub-module to deconvolute The third feature map obtained by the deconvolution of the previous layer of resolution is the same as the first feature map and the third feature map are combined for deconvolution.

S303: Use the third sub-module of the key point detection model to perform group convolution on the second feature map according to the spatial relationship of human limbs, output the heat map of each key point of the target human body image according to the result of the group convolution, and according to the heat map Determining each human body keypoint of the target human body image.

Wherein, S301 and S302 are respectively the same as S101 and S102 in the first embodiment. For details, please refer to the description in the first embodiment. In order to avoid repetition, details are not repeated here.

For S303, specifically, because the limbs of the same part of the human body will have a relatively fixed spatial relationship in the target human body image, for example, the relative positions of the key points of the human body in the hand will be adjacent to each other, and the limbs of the same part will be concentrated due to the In a certain area of the target human body image, the features in the target human body image are generally similar. Therefore, grouping and convolving the second feature map according to the spatial relationship of human body limbs can make the key point detection models interact with each other in the same group of convolutions. Learning and referring to the spatial position and image features of the limbs of the same part can effectively improve the detection accuracy of human key points.

Optionally, according to the specific grouping of the second feature map according to the spatial relationship of human limbs, the number of groups can be set according to actual needs, and no specific limitation is set here. For example, you can group by regions such as head, torso, hands, and feet. Optionally, a part can also be subdivided to obtain a new group, for example, the left hand and the right hand in the hand are divided into one group.

It can be understood that the second feature map is grouped and convolved according to the spatial relationship of human limbs. During the training process, the grouped convolution of the second feature map can affect the previous group deconvolution of the key detection model through the backpropagation algorithm. and other parts, so that the overall detection effect of the key point detection model is obtained according to the spatial relationship of human body limbs, thereby effectively improving the detection accuracy of human key points.

In a specific example, the second feature map is grouped and convolved according to the spatial relationship of human limbs by using the third sub-module, which may specifically be: using the third sub-module to group the second feature map according to the preset key point grouping Convolution, wherein, the second feature map of each group in the key point grouping includes the second feature map of the first human body key point that is in the adjacent position and belongs to the same limb position of the human body, the first human body key point is based on the body limb Spatial relationships to predefined keypoints.

Please refer to FIG. 5 , which is a detection example diagram of the human body key point detection method provided by the embodiment of the present invention. In Figure 5, the key points of the human body are divided into 63 key points, and each key point is a key point predefined according to the spatial relationship of the body limbs. For example, 0 and 58 key points of the human body refer to the points on the left and right sides of the neck, respectively. In Figure 5, each box represents a grouping, where the key points of the head are divided into one group, the hands are divided into four groups, the torso is divided into two groups, and the feet are divided into two groups.

It can be understood that the key points of the human body in Figure 5 are the edge points of the target human body image. When it is necessary to know the key points where the skeleton is located in the target human body image, it can be based on the relative relationship between the key points where the skeleton is located and the key points of the human body defined in the figure. The position or relative distance is calculated.

Compared with the prior art, the human body key point detection method provided by the embodiment of the present invention performs group convolution on the second feature map according to the spatial relationship of human body limbs, and outputs the key points of each human body in the target human body image according to the result of the group convolution. heat map. Since the limbs of the same part of the human body have a relatively fixed spatial relationship in the target human body image, and the features of the limbs of the same part in the target human body image are relatively similar, the second feature map is grouped and convolved according to the spatial relationship of the human limbs , which can make the key point detection models learn from each other and refer to the spatial position and image features of the same limb in the same group of convolutions, thereby effectively improving the detection accuracy of human key points.

The division of steps in the above methods is only for the sake of clarity of description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this patent. ; Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.

The fourth embodiment of the present invention relates to a human body key point detection device 400, as shown in Figure 6, comprising: an extraction module 401, a deconvolution module 402, and a detection module 403, and the functions of each module are described in detail as follows:

The extraction module 401 is used to extract the first feature map of the target human body image, the first feature map includes multiple groups of feature maps with different resolutions;

The deconvolution module 402 is used to perform deconvolution of several layers according to the first feature map to obtain a second feature map, wherein, when deconvolution of the next layer is performed, the previous layer of resolution is deconvolved to obtain Deconvolution is performed after combining the same first feature map and third feature map of the third feature map;

The detection module 403 is configured to output a heat map of each human key point of the target human body image according to the second feature map, and determine each human body key point of the target human body image according to the heat map.

Further, the deconvolution module 402 is also used for:

When deconvolution of each layer is performed, the deconvolution is integrated into several groups according to the number of channels.

Further, the deconvolution module 402 is also used for:

When deconvolution of the first layer is performed, deconvolution is performed according to the first feature map of the minimum resolution.

Further, the detection module 403 is also used for:

Carry out group convolution on the second feature map according to the spatial relationship of human body limbs, and output the heat map of each key point of the human body in the target human body image according to the result of the group convolution.

Further, the detection module 403 is also used for:

Perform group convolution on the second feature map according to the preset key point grouping, wherein the second feature map of each group in the key point grouping includes the key points of the first human body that are in adjacent positions and belong to the same limb position of the human body In the second feature map, the first key points of the human body are key points predefined according to the spatial relationship of human body limbs.

Further, the extraction module 401 is also used for:

Obtain the first feature map of the target human body image based on a lightweight convolutional neural network.

Further, the extraction module 401 is also used for:

The number of channels of the last convolutional layer of the lightweight convolutional neural network is reduced to a preset number, and the first feature map of the target human body image is obtained according to the reduced number of channels of the lightweight convolutional neural network.

It is not difficult to find that this embodiment is a device embodiment corresponding to the first embodiment, the second embodiment and the third embodiment, and this embodiment can be mutually compatible with the first embodiment, the second embodiment and the third embodiment. Cooperate with implementation. The relevant technical details mentioned in the first embodiment, the second embodiment, and the third embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Correspondingly, the related technical details mentioned in this implementation manner can also be applied in the first implementation manner, the second implementation manner and the third implementation manner.

It is worth mentioning that all the modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, or a part of a physical unit, or multiple physical units. Combination of units. In addition, in order to highlight the innovative part of the present invention, units that are not closely related to solving the technical problems proposed by the present invention are not introduced in this embodiment, but this does not mean that there are no other units in this embodiment.

The fifth embodiment of the present invention relates to a network device. As shown in FIG. 7 , it includes at least one processor 501; and a memory 502 communicatively connected to at least one processor 501; The instructions executed by the processor 501 are executed by at least one processor 501, so that the at least one processor 501 can execute the above-mentioned human body key point detection method.

Wherein, the memory 502 and the processor 501 are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors 501 and various circuits of the memory 502 together. The bus may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, all of which are well known in the art and therefore will not be further described herein. The bus interface provides an interface between the bus and the transceivers. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing means for communicating with various other devices over a transmission medium. The data processed by the processor 501 is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor 501 .

Processor 501 is responsible for managing the bus and general processing, and may also provide various functions including timing, peripheral interface, voltage regulation, power management and other control functions. And the memory 502 may be used to store data used by the processor 501 when performing operations.

The sixth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The above method embodiments are implemented when the computer program is executed by the processor.

That is, those skilled in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing related hardware through a program, the program is stored in a storage medium, and includes several instructions to make a device (can It is a single-chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-OnlyMemory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc and other media that can store program codes.

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific examples for realizing the present invention, and in practical applications, various changes can be made to it in form and details without departing from the spirit and spirit of the present invention. scope.

Claims (9)

1. a human body key point detection method, is characterized in that, is applied to the key point detection model of pre-training, and described human body key point detection method comprises: Using the first sub-module of the key point detection model to extract the first feature map of the target human body image, the first feature map includes multiple groups of feature maps with different resolutions; Use the second sub-module of the key point detection model to perform several layers of deconvolution according to the first feature map to obtain a second feature map, wherein, when performing deconvolution of the next layer, use the first feature map The second sub-module performs deconvolution after combining the first feature map with the same resolution as the third feature map obtained by the previous layer of deconvolution with the third feature map; Utilize the third sub-module of the key point detection model to output the heat map of each human body key point of the target human body image according to the second feature map, and determine each human body key point of the target human body image according to the heat map point; Wherein, the third sub-module using the key point detection model outputs a heat map of each human body key point of the target human body image according to the second feature map, including: Using the third sub-module to perform group convolution on the second feature map according to the spatial relationship of human limbs, and output a heat map of each key point of the target human body image according to the result of the group convolution. 2. human body key point detection method according to claim 1, is characterized in that, the described second submodule utilizing described key point detection model carries out the deconvolution of several layers according to described first feature map, comprising: When deconvolution of each layer is performed, the second sub-module is used to integrate the deconvolution into several groups according to the number of channels. 3. human body key point detection method according to claim 1, is characterized in that, the described second submodule utilizing described key point detection model carries out the deconvolution of several layers according to described first feature map, comprising: When deconvolution of the first layer is performed, the second sub-module is used to perform deconvolution according to the first feature map with the minimum resolution. 4. The human body key point detection method according to claim 1, wherein the second feature map is grouped and convolved according to the spatial relationship of human limbs by using the third submodule, specifically: Use the third submodule to perform group convolution on the second feature map according to the preset key point grouping, the second feature map of each group in the key point grouping includes adjacent positions and belonging to The second feature map of the first key points of the human body at the same limb position of the human body, where the first key points of the human body are key points predefined according to the spatial relationship of the human body limbs. 5. human body key point detection method according to claim 1, is characterized in that, described utilizing the first submodule of described key point detection model to extract the first feature map of described target human body image, specifically: The first feature map of the target human body image is obtained by using the first submodule according to a lightweight convolutional neural network. 6. human body key point detection method according to claim 5, is characterized in that, described utilizing described first submodule to obtain the first feature map of described target human body image according to lightweight convolutional neural network, comprising: Use the first sub-module to reduce the number of channels of the last convolutional layer of the lightweight convolutional neural network to a preset number, and obtain the obtained information according to the reduced number of channels of the lightweight convolutional neural network. The first feature map of the target human body image. 7. A human key point detection device, characterized in that it comprises: An extraction module, used to extract the first feature map of the target human body image, the first feature map includes multiple groups of feature maps with different resolutions; The deconvolution module is used to perform deconvolution of several layers according to the first feature map to obtain a second feature map, wherein, when performing deconvolution of the next layer, the previous layer of resolution is deconvoluted performing deconvolution after combining the first feature map and the third feature map obtained with the same third feature map; A detection module, configured to output a heat map of each human body key point of the target human body image according to the second feature map, and determine each human body key point of the target human body image according to the heat map; Wherein, the outputting the heat map of each key point of the target human body image according to the second feature map includes: Perform group convolution on the second feature map according to the spatial relationship of human body limbs, and output a heat map of each human body key point of the target human body image according to the result of the group convolution. 8. A network device, characterized in that, comprising: at least one processor; and, a memory communicatively coupled to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor, so that the at least one processor can perform the operation described in any one of claims 1 to 6 The human key point detection method described above. 9. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the human body key point detection method according to any one of claims 1 to 6 is realized.

Images