CN115578753B - Human body key point detection method, device, electronic equipment and storage medium - Google Patents

Abstract

The invention provides a human body key point detection method, a device, electronic equipment and a storage medium, wherein the human body key point detection method comprises the following steps: acquiring a human body image to be detected; generating a first characteristic image and a second characteristic image corresponding to the human body image to be detected based on the human body image to be detected; determining a key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map and the key point concentration priori knowledge; the key point density priori knowledge is used for distinguishing the dense parts of the key points of the human body from the sparse parts of the key points of the human body in the second characteristic diagram. By the method, feature graphs with different resolutions can be adopted for detection of different key point positions, accurate detection of dense key point positions of a human body is achieved on the basis of guaranteeing detection accuracy of sparse key point positions of the human body, and detection accuracy of key points of the whole body of the human body is further improved.

Description

Human body key point detection method, device, electronic equipment and storage medium

technical field

The present invention relates to the technical field of artificial intelligence, in particular to a human body key point detection method, device, electronic equipment and storage medium.

Background technique

With the progress of society and the development of science and technology, online education, smart medical care, intelligent robots and other industries continue to rise, and there are more and more functional requirements for interaction based on key points of the human body. Among them, human key point detection is the basis of many high-level visual tasks, such as behavior recognition, clothing analysis, pedestrian re-identification, etc., using human key point detection technology.

In related technologies, the convolutional neural network is usually used to learn the key points of the human body in pictures or videos through supervised learning, but this method is mainly for detecting the key points of sparse parts of the human body.

However, for the dense key points of the human body, since the distribution density of the dense key points of the human body varies greatly in various parts of the human body, in related technologies, all key points of the human body are detected using the same network structure, which will result in low detection accuracy for dense key points. ; Therefore, how to improve the detection accuracy of the key points of the whole body of the human body is an urgent problem to be solved at present.

Contents of the invention

Aiming at the problems existing in the prior art, embodiments of the present invention provide a human body key point detection method, device, electronic equipment, and storage medium.

The present invention provides a human body key point detection method, comprising:

Obtain an image of a human body to be detected;

Based on the human body image to be detected, generate a first feature map and a second feature map corresponding to the human body image to be detected; the first feature map and the second feature map have different resolutions;

Based on the first feature map, the second feature map and prior knowledge of key point density, determine the key point detection result corresponding to the human body image to be detected; the key point density prior knowledge is used to distinguish all Describe the dense parts of the human body key points and the sparse parts of the human body key points in the second feature map.

Optionally, the prior knowledge of key point density is obtained in the following manner:

Obtain the number of key points of each human body part;

Based on the number of key points and the area of a two-dimensional rectangle corresponding to each of the human body parts, the prior knowledge of key point density is determined.

Optionally, the determining the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map, and key point density prior knowledge includes:

Based on the first feature map, determine a first detection result corresponding to a sparse human body key point in the human body image to be detected;

Based on the first feature map, determine a plurality of body part detection frames corresponding to the second feature map, and the body part detection frames are used to crop the second feature map;

Based on the second feature map, each of the body part detection frames and the prior knowledge of the density of key points, determine a second detection result corresponding to a human body key point dense part in the image of the human body to be detected.

Optionally, the determining a first detection result corresponding to a sparse human body key point in the human body image to be detected based on the first feature map includes:

performing a downsampling operation on the first feature map to obtain a first target feature map;

Inputting the first target feature map into a convolutional layer to obtain the first detection result output by the convolutional layer.

Optionally, based on the second feature map, each of the body part detection frames and the prior knowledge of key point density, determining the second detection result corresponding to the human body key point dense part in the human body image to be detected, include:

clipping the second feature map by using the body part detection frame to obtain a third feature map corresponding to each of the human body parts;

Based on the prior knowledge of key point density, at least one second target feature map is determined, and the second target feature map is a feature map corresponding to a dense part of the human body key points in each of the third feature maps;

Based on each of the second target feature maps, the second detection result is determined.

Optionally, the generating a first feature map and a second feature map corresponding to the human body image to be detected based on the human body image to be detected includes:

Inputting the human body image to be detected into a residual network model to obtain an initial feature map output by the residual network model;

performing multiple upsampling operations on the initial feature map to obtain the first feature map and the second feature map.

The present invention also provides a human body key point detection device, comprising:

The first acquisition module is used to acquire human body images to be detected;

A generating module, configured to generate a first feature map and a second feature map corresponding to the human body image to be detected based on the human body image to be detected; the first feature map and the second feature map have different resolutions;

The first determination module is used to determine the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map and key point density prior knowledge; the key point density The prior knowledge is used to distinguish dense parts of human body key points from sparse parts of human body key points in the second feature map.

The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the program, the key points of any one of the above-mentioned human bodies are realized. Detection method.

The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the human body key point detection method described in any one of the above-mentioned methods is realized.

The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the human body key point detection method described in any one of the above-mentioned methods is realized.

The human body key point detection method, device, electronic equipment, and storage medium provided by the present invention generate a first feature map and a second feature map with different resolutions based on the acquired human body image to be detected; due to the prior knowledge of the key point density It can distinguish dense parts of human body key points from sparse parts of human body key points in the second feature map. Therefore, based on the prior knowledge of the first feature map, the second feature map and the density of key points, different discrimination methods can be used for different key points. On the basis of ensuring the detection accuracy of the sparse parts of the key points of the human body, the accurate detection of the dense parts of the key points of the human body is realized, and the detection accuracy of the key points of the whole body of the human body is improved.

Description of drawings

In order to more clearly illustrate the present invention or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are the present invention. For some embodiments of the invention, those skilled in the art can also obtain other drawings based on these drawings without creative effort.

Fig. 1 is one of the schematic flow charts of the human body key point detection method provided by the present invention;

Fig. 2 is the second schematic flow diagram of the human body key point detection method provided by the present invention;

Fig. 3 is a schematic structural diagram of a human body key point detection device provided by the present invention;

Fig. 4 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention , but not all examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

In order to facilitate a clearer understanding of the embodiments of the present application, some relevant background knowledge is introduced as follows.

Since the distribution of key points of human joints is sparse and uniform, the convolutional neural network is directly used, and the mapping from images to key points of human body can be directly learned by using supervised learning methods.

However, for the dense key points of the human body, the distribution density of each part of the body is quite different, and the same network structure is used for all key points, resulting in low detection accuracy for dense key points.

Moreover, the method of detecting key points of the whole body of the human body in related technologies is to divide the human body into different parts and train a model for each part separately. This non-end-to-end method brings additional costs to model management, training, data production, etc. Challenges and burdens greatly restrict practical use.

Therefore, aiming at the above-mentioned existing technical problems, in order to improve the detection accuracy of the key points of the whole body of the human body, the present invention provides a method, device, electronic equipment and storage medium for detecting the key points of the human body.

The method for detecting human body key points provided by the present invention will be specifically described below in conjunction with FIG. 1 . Fig. 1 is one of the schematic flow charts of the human body key point detection method provided by the present invention, as shown in Fig. 1, the method includes step 101-step 103, wherein:

Step 101, acquiring an image of a human body to be detected.

First of all, it needs to be explained that the execution subject of the present invention can be any electronic device with the function of detecting key points of the human body, for example, it can be any kind of smart phone, smart watch, desktop computer, laptop computer, etc.

Due to the large difference in the distribution density of the dense key points of the human body in various parts of the human body, if the same network structure is used for detection of all key points of the human body, the detection accuracy of dense key points will be low.

Therefore, in order to improve the detection accuracy of the key points of the whole body of the human body, in this embodiment, it is first necessary to obtain an image of the human body to be detected.

Specifically, the human body image to be detected refers to the whole body image of the human body to be detected by key points, and the whole body image of the human body to be detected includes the head, upper arm, lower arm, torso, thigh, calf, hand, foot and other parts of the person.

Step 102. Based on the human body image to be detected, generate a first feature map and a second feature map corresponding to the human body image to be detected; the first feature map and the second feature map have different resolutions.

In this embodiment, after the human body image to be detected is obtained, it is necessary to generate a first feature map and a second feature map with different resolutions based on the human body image to be detected;

Wherein, the first feature map is a low-resolution feature map, for example, the resolution is 128*128; the second feature map is a high-resolution feature map, for example, the resolution is 512*512.

Step 103: Based on the first feature map, the second feature map, and the prior knowledge of key point density, determine the key point detection result corresponding to the human body image to be detected; the key point density prior knowledge is used In order to distinguish the human body key point dense part and the human body key point sparse part in the second feature map.

In this embodiment, it is necessary to determine the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map, and the prior knowledge of key point density.

Specifically, since the prior knowledge of the key point density is used to distinguish the human body key point dense part and the human body key point sparse part in the second feature map, the key point detection results corresponding to the human body image to be detected can be specifically divided into: The first detection result corresponding to the sparse key point parts of the human body and the second detection result corresponding to the dense key point parts of the human body.

Optionally, in a possible implementation of the embodiment of the present invention, the prior knowledge of key point density can be obtained through the following steps [1]-step [2]:

Step [1], obtaining the number of key points of each human body part;

Step [2], based on the number of key points and the two-dimensional rectangular area corresponding to each of the human body parts, determine the prior knowledge of key point density.

In this embodiment, the prior knowledge of key point density is used to distinguish between dense key point parts of the human body and sparse key point parts of the human body; where the various parts of the human body are: head, upper arm, lower arm, torso, thigh, calf , hands, feet, each part has a preset number of key points.

After obtaining the number of key points of each body part, for each body part, it is necessary to divide the number of key points of the part by the area of the two-dimensional rectangle corresponding to the part to obtain the target result corresponding to the part;

Then set a threshold. If the target result is greater than this threshold, the part is a dense part of human key points; otherwise, if the target result is smaller than this threshold, this part is a sparse part of human key points.

In the above embodiment, the prior knowledge of key point density is introduced, and based on the number of key points and the two-dimensional rectangular area corresponding to each body part, the human body key point dense part and the human body key point sparse part in the second feature map can be determined ; and then it is possible to use feature maps with different resolutions for detection of different key points.

The human body key point detection method provided by the present invention generates the first feature map and the second feature map with different resolutions based on the obtained human body image to be detected; Human body key point dense parts and human body key point sparse parts, therefore, based on the first feature map, the second feature map and the prior knowledge of key point density, it is possible to detect different key point parts using feature maps with different resolutions. On the basis of ensuring the detection accuracy of the sparse parts of the key points of the human body, the accurate detection of the dense parts of the key points of the human body is realized, and the detection accuracy of the key points of the whole body of the human body is improved.

Optionally, based on the human body image to be detected, generating the first feature map and the second feature map corresponding to the human body image to be detected can be specifically implemented through the following steps a)-step b):

Step a), input the human body image to be detected into the residual network model, and obtain the initial feature map output by the residual network model;

Step b), performing multiple upsampling operations on the initial feature map to obtain the first feature map and the second feature map.

In this embodiment, after obtaining the image of the human body to be detected (for example, the resolution is 512*512), it is necessary to input the image of the human body to be detected into the residual network model (for example, ResNet-50) for basic feature extraction to obtain the residual network The feature map of the initial feature map output by the model (for example, with a resolution of 16*16).

Then the initial feature map is subjected to multiple upsampling operations to obtain the first feature map and the second feature map.

Specifically, after obtaining an initial feature map with a resolution of 16*16, the initial feature map is input into multiple upsampling modules, and finally the first feature maps output by different upsampling modules (for example, a resolution of 128*128 ) and the second feature map (for example, the resolution is 512*512);

Among them, each upsampling module includes 2 times upsampling layer, 3*3 scale convolution layer, batch normalization layer and ReLU layer.

In the above embodiment, by inputting the human body image to be detected into the residual network model, and then performing multiple upsampling operations on the initial feature map output by the residual network model, the first feature map and the second feature map with different resolutions are obtained , and then different key points can be detected by using feature maps with different resolutions. On the basis of ensuring the detection accuracy of sparse key points of the human body, the accurate detection of dense key points of the human body is realized, thereby improving the accuracy of key points of the human body. point detection accuracy.

Optionally, the determination of the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map, and the prior knowledge of key point density can be performed through the following steps 1)- Step 3) Realize:

Step 1), based on the first feature map, determine the first detection result corresponding to the sparse part of the human body key points in the human body image to be detected;

Step 2), based on the first feature map, determine a plurality of body part detection frames corresponding to the second feature map, and the body part detection frames are used to crop the second feature map;

Step 3), based on the second feature map, the detection frame of each body part and the prior knowledge of key point density, determine the second detection result corresponding to the human body key point dense part in the human body image to be detected.

In this embodiment, since the sparse parts of the human body key points do not require high-resolution feature maps for detection, the first detection result corresponding to the sparse parts of the human body key points in the human body image to be detected can be determined directly based on the first feature map .

In order to be able to cut out the features of the dense parts of the key points of the human body on the high-resolution feature map (ie, the second feature map), and eliminate the interference of other parts of the feature;

Therefore, in this embodiment, a human body part detection network needs to be connected to the first feature map at the same time, and the head, upper arm, lower arm, torso, thigh, calf, hand, and foot parts of the body on the first feature map are respectively output Part detection frame, and the body part detection frame is proportionally enlarged to the second feature map; each body part detection frame is used to crop the second feature map to obtain the feature map corresponding to each human body part.

Finally, based on the second feature map, the detection frame of each body part, and the prior knowledge of key point density, the second detection result corresponding to the dense key point parts of the human body in the human body image to be detected can be determined.

In the above embodiment, based on the first feature map, the second feature map and the prior knowledge of key point density, it is possible to use feature maps with different resolutions for detection of different key points, and to ensure the detection accuracy of sparse key points of the human body. On the basis of this method, the accurate detection of the dense parts of the key points of the human body is realized, and the detection accuracy of the key points of the whole body of the human body is improved.

Optionally, based on the first feature map, determining the first detection result corresponding to the sparse part of the human body key points in the human body image to be detected can be specifically implemented through the following steps (1)-step (2):

Step (1), carrying out the down-sampling operation on the first feature map to obtain the first target feature map;

Step (2), inputting the first target feature map into a convolutional layer, and obtaining the first detection result output by the convolutional layer.

In this embodiment, since the sparse parts of the key points of the human body do not require a high-resolution feature map for detection, the first feature map is directly down-sampled to obtain the first target feature map; then the first target feature map Enter the convolutional layer to obtain the first detection result.

Specifically, first input the first feature map (for example, with a resolution of 128*128) into the downsampling module, wherein the downsampling module includes at least a 3*3 convolutional layer, a maximum pooling layer, a batch normalization layer, and a ReLU layer .

After inputting the first feature map with a resolution of 128*128 into the downsampling module, the first target feature map can be obtained (that is, the resolution of the first target feature map is reduced to 64*64).

Finally, input the first target feature map with a resolution of 64*64 into the 1*1 convolutional layer to obtain a multi-channel heat map, and output the first object corresponding to the sparse key points of the human body in the human body image to be detected in the form of a heat map. Test results.

In the above implementation manner, for the sparse key point parts of the human body, the first feature map is directly used for key point detection, which ensures the detection accuracy of the sparse key point parts of the human body.

Optionally, based on the second feature map, each of the body part detection frames and the prior knowledge of key point density, determining the second detection result corresponding to the human body key point dense part in the human body image to be detected, Specifically, it can be achieved through the following steps [a]-step [c]:

Step [a], using the body part detection frame to crop the second feature map to obtain a third feature map corresponding to each of the human body parts;

Step [b], based on the prior knowledge of the key point density, determine at least one second target feature map, the second target feature map is the feature corresponding to the key point dense part of the human body in each of the third feature maps picture;

Step [c], based on each of the second target feature maps, determine the second detection result.

In this embodiment, since the position and size information of each part of the human body on the second feature map has been obtained based on the body part detection frame, the body part detection frame can be used to crop the second feature map to obtain the corresponding body parts. The third feature map.

For example, a human body part detection network is connected after the first feature map, and the body part detection frames of the head, upper arm, lower arm, torso, thigh, calf, hand, and foot on the first feature map are respectively output, and the body The part detection frame is enlarged proportionally to the second feature map; the body part detection frame is used to crop the second feature map, and then the third part corresponding to the head, upper arm, lower arm, torso, thigh, calf, hand, and foot is obtained. feature map;

Then, based on the prior knowledge of the key point density, the feature map corresponding to the dense part of the human body key point in each third feature map is determined as the second target feature map.

In order to adopt the same key point detection network as the first feature map for all feature maps, the second target feature map needs to be scaled until it has the same resolution as the first feature map;

Finally, the scaled second target feature map is input into the downsampling module and the 1*1 convolutional layer, and the second detection result corresponding to the dense key points of the human body in the human body image to be detected can be obtained; wherein, the downsampling module includes at least 3 *3 convolutional layer, maximum pooling layer, batch normalization layer and ReLU layer.

In the above embodiment, based on the obtained human body image to be detected, the first feature map and the second feature map with different resolutions are generated; due to the prior knowledge of key point density, it is possible to distinguish the density of key points of the human body in the second feature map Therefore, based on the first feature map, the second feature map and the prior knowledge of key point density, it is possible to use feature maps with different resolutions for different key point parts to detect, ensuring that the key points of the human body On the basis of the detection accuracy of sparse parts, the accurate detection of dense parts of key points of the human body is realized, and the detection accuracy of key points of the whole body of the human body is improved. The point sparse parts all use the same key point detection network, and there is no need to train different models for different parts of the human body, thereby reducing the complexity of human key point detection, avoiding non-end-to-end, complex key point detection model training, and difficult model management. Problems such as high cost of data production save manpower and material resources.

Fig. 2 is the second schematic flow diagram of the human body key point detection method provided by the present invention, as shown in Fig. 2, the method includes step 201-step 208, wherein:

Step 201, acquiring an image of a human body to be detected.

Step 202: Input the human body image to be detected into the residual network model to obtain an initial feature map output by the residual network model.

Step 203 , performing multiple upsampling operations on the initial feature map to obtain a first feature map and a second feature map; the resolutions of the first feature map and the second feature map are different.

Step 204: Perform a down-sampling operation on the first feature map to obtain a first target feature map.

Step 205: Input the first target feature map into the convolutional layer, and obtain the first detection result output by the convolutional layer.

Step 206 , using the body part detection frame to crop the second feature map to obtain a third feature map corresponding to each body part.

Step 207 : Determine at least one second target feature map based on the prior knowledge of key point density, wherein the second target feature map is a feature map corresponding to the dense key point parts of the human body in each third feature map.

It should be noted that the prior knowledge of key point density can be obtained through the following steps [1]-step [2]:

Step [1], obtaining the number of key points of each human body part;

Step [2], based on the number of key points and the two-dimensional rectangular area corresponding to each of the human body parts, determine the prior knowledge of key point density.

Step 208: Determine a second detection result based on each second target feature map.

It should be noted that the present invention does not limit the execution order of steps 204-205 and steps 206-208, that is, the order of execution is not in particular order.

The human body key point detection method provided by the present invention generates the first feature map and the second feature map with different resolutions based on the obtained human body image to be detected; Human body key point dense parts and human body key point sparse parts, therefore, based on the first feature map, the second feature map and the prior knowledge of key point density, it is possible to detect different key point parts using feature maps with different resolutions. On the basis of ensuring the detection accuracy of the sparse parts of the key points of the human body, the accurate detection of the dense parts of the key points of the human body is realized, thereby improving the detection accuracy of the key points of the whole body of the human body; at the same time, this method is an end-to-end detection method for the key points of the human body Both the dense part and the key point sparse part use the same key point detection network, and there is no need to train different models for different parts of the human body, thereby reducing the complexity of human key point detection and avoiding the complexity of non-end-to-end key point detection model training. Problems such as difficult model management and high data production costs save manpower and material resources.

The human body key point detection device provided by the present invention is described below, and the human body key point detection device described below and the human body key point detection method described above can be referred to in correspondence. Fig. 3 is a schematic structural diagram of a human body key point detection device provided by the present invention. As shown in Fig. 3, the human body key point detection device 300 includes: an acquisition module 301, a generation module 302 and a determination module 303, wherein:

The first obtaining module 301 is used to obtain the image of the human body to be detected;

A generating module 302, configured to generate a first feature map and a second feature map corresponding to the human body image to be detected based on the human body image to be detected; the first feature map and the second feature map have different resolutions ;

The first determining module 303 is configured to determine the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map and key point density prior knowledge; the key point is dense The degree prior knowledge is used to distinguish dense parts of human body key points from sparse parts of human body key points in the second feature map.

The human body key point detection device provided by the present invention generates a first feature map and a second feature map with different resolutions based on the obtained human body image to be detected; Human body key point dense parts and human body key point sparse parts, therefore, based on the first feature map, the second feature map and the prior knowledge of key point density, it is possible to detect different key point parts using feature maps with different resolutions. On the basis of ensuring the detection accuracy of the sparse parts of the key points of the human body, the accurate detection of the dense parts of the key points of the human body is realized, and the detection accuracy of the key points of the whole body of the human body is improved.

Optionally, the device also includes:

The second obtaining module is used to obtain the number of key points of each human body part;

The second determination module is configured to determine the prior knowledge of key point density based on the number of key points and the two-dimensional rectangular area corresponding to each of the human body parts.

Optionally, the first determining module 303 is further configured to:

Based on the first feature map, determine a first detection result corresponding to a sparse human body key point in the human body image to be detected;

Based on the first feature map, determine a plurality of body part detection frames corresponding to the second feature map, and the body part detection frames are used to crop the second feature map;

Based on the second feature map, each of the body part detection frames and the prior knowledge of the density of key points, determine a second detection result corresponding to a human body key point dense part in the image of the human body to be detected.

Optionally, the first determining module 303 is further configured to:

performing a downsampling operation on the first feature map to obtain a first target feature map;

Inputting the first target feature map into a convolutional layer to obtain the first detection result output by the convolutional layer.

Optionally, the first determining module 303 is further configured to:

clipping the second feature map by using the body part detection frame to obtain a third feature map corresponding to each of the human body parts;

Based on the prior knowledge of key point density, at least one second target feature map is determined, and the second target feature map is a feature map corresponding to a dense part of the human body key points in each of the third feature maps;

Based on each of the second target feature maps, the second detection result is determined.

Optionally, the generation module 302 is further used to:

Inputting the human body image to be detected into a residual network model to obtain an initial feature map output by the residual network model;

performing multiple upsampling operations on the initial feature map to obtain the first feature map and the second feature map.

FIG. 4 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG. 4, the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, Wherein, the processor 410 , the communication interface 420 , and the memory 430 communicate with each other through the communication bus 440 . The processor 410 may call the logic instructions in the memory 430 to execute the method for detecting key points of the human body, the method comprising: acquiring an image of the human body to be detected; based on the image of the human body to be detected, generating a first feature corresponding to the image of the human body to be detected map and the second feature map; the resolutions of the first feature map and the second feature map are different; based on the first feature map, the second feature map and the prior knowledge of key point density, determine the The key point detection result corresponding to the human body image to be detected is described; the key point density prior knowledge is used to distinguish the human body key point dense part and the human body key point sparse part in the second feature map.

In addition, the above logic instructions in the memory 430 may be implemented in the form of software function units and be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .

On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can Executing the human body key point detection method provided by the above methods, the method includes: acquiring a human body image to be detected; based on the human body image to be detected, generating a first feature map and a second feature map corresponding to the human body image to be detected; The first feature map and the second feature map have different resolutions; based on the first feature map, the second feature map, and prior knowledge of key point density, determine the corresponding Key point detection results; the prior knowledge of the key point density is used to distinguish the human body key point dense part and the human body key point sparse part in the second feature map.

In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to perform the human body key point detection method provided by the above methods, the method The method includes: acquiring a human body image to be detected; generating a first feature map and a second feature map corresponding to the human body image to be detected based on the human body image to be detected; distinguishing the first feature map and the second feature map rate is different; based on the first feature map, the second feature map and the prior knowledge of key point density, determine the key point detection result corresponding to the human body image to be detected; the key point density prior knowledge is used In order to distinguish the human body key point dense part and the human body key point sparse part in the second feature map.

The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative effort.

Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (8)

1. A human key point detection method, characterized in that, comprising: Obtain an image of a human body to be detected; Based on the human body image to be detected, generate a first feature map and a second feature map corresponding to the human body image to be detected; the first feature map and the second feature map have different resolutions; Based on the first feature map, the second feature map and prior knowledge of key point density, determine the key point detection result corresponding to the human body image to be detected; the key point density prior knowledge is used to distinguish all Describe the dense parts of the human body key points and the sparse parts of the human body key points in the second feature map; Wherein, the determining the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map and key point density prior knowledge includes: Based on the first feature map, determine a first detection result corresponding to a sparse human body key point in the human body image to be detected; Based on the first feature map, determine a plurality of body part detection frames corresponding to the second feature map, and the body part detection frames are used to crop the second feature map; Based on the second feature map, each of the body part detection frames and the prior knowledge of the density of key points, determine a second detection result corresponding to a human body key point dense part in the image of the human body to be detected. 2. the human body key point detection method according to claim 1, is characterized in that, described key point intensity prior knowledge obtains by following way: Obtain the number of key points of each human body part; Based on the number of key points and the area of a two-dimensional rectangle corresponding to each of the human body parts, the prior knowledge of key point density is determined. 3. The human body key point detection method according to claim 1, characterized in that, based on the first feature map, determining the first detection result corresponding to the sparse part of the human body key point in the human body image to be detected includes : performing a downsampling operation on the first feature map to obtain a first target feature map; Inputting the first target feature map into a convolutional layer to obtain the first detection result output by the convolutional layer. 4. The human body key point detection method according to claim 1, characterized in that, the determination of the subject to be detected is determined based on the second feature map, each of the body part detection frames and the prior knowledge of the key point density. The second detection result corresponding to the dense part of the key points of the human body in the human body image, including: clipping the second feature map by using the body part detection frame to obtain a third feature map corresponding to each of the human body parts; Based on the prior knowledge of key point density, at least one second target feature map is determined, and the second target feature map is a feature map corresponding to a dense part of the human body key points in each of the third feature maps; Based on each of the second target feature maps, the second detection result is determined. 5. The method for detecting key points of a human body according to claim 1, wherein the first feature map and the second feature map corresponding to the human body image to be detected are generated based on the human body image to be detected, comprising: Inputting the human body image to be detected into a residual network model to obtain an initial feature map output by the residual network model; performing multiple upsampling operations on the initial feature map to obtain the first feature map and the second feature map. 6. A human body key point detection device, characterized in that it comprises: The first acquisition module is used to acquire human body images to be detected; A generating module, configured to generate a first feature map and a second feature map corresponding to the human body image to be detected based on the human body image to be detected; the first feature map and the second feature map have different resolutions; The first determination module is used to determine the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map and key point density prior knowledge; the key point density The prior knowledge is used to distinguish dense parts of human body key points from sparse parts of human body key points in the second feature map; Wherein, the determining the key point detection result corresponding to the human body image to be detected based on the first feature map, the second feature map and key point density prior knowledge includes: Based on the first feature map, determine a first detection result corresponding to a sparse human body key point in the human body image to be detected; Based on the first feature map, determine a plurality of body part detection frames corresponding to the second feature map, and the body part detection frames are used to crop the second feature map; Based on the second feature map, each of the body part detection frames and the prior knowledge of the density of key points, determine a second detection result corresponding to a human body key point dense part in the image of the human body to be detected. 7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor according to claim 1 is implemented when executing the program. The human body key point detection method described in any one of 5 to 5. 8. A non-transitory computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the human body key point detection method according to any one of claims 1 to 5 is realized .

Images