CN111523468B - Human body key point identification method and device - Google Patents
Human body key point identification method and device Download PDFInfo
- Publication number
- CN111523468B CN111523468B CN202010327619.9A CN202010327619A CN111523468B CN 111523468 B CN111523468 B CN 111523468B CN 202010327619 A CN202010327619 A CN 202010327619A CN 111523468 B CN111523468 B CN 111523468B
- Authority
- CN
- China
- Prior art keywords
- human body
- image
- key point
- body key
- heat map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 238000001514 detection method Methods 0.000 claims abstract description 87
- 238000012549 training Methods 0.000 claims abstract description 24
- 230000009471 action Effects 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims description 17
- 210000000746 body region Anatomy 0.000 description 11
- 238000010586 diagram Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 235000012434 pretzels Nutrition 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a human body key point identification method and a device, which relate to the technical field of image identification and specifically comprise the following steps: acquiring a target image of a key point of a human body to be identified; inputting a target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image, wherein the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true heat map of each human body key point, the radial action range of the true heat map of each human body key point is determined based on scale parameters, and the scale parameters are determined according to the distance values of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points; according to the heat map of each human body key point in the target image, the coordinates of each human body key point are determined, and the accuracy of detecting the human body key points is effectively improved.
Description
Technical Field
The application relates to the field of image processing, in particular to the technical field of image recognition, and especially relates to a human body key point recognition method and device.
Background
In the current general human body key point detection model training algorithm, because of the complexity of actual service data, enhancement operation is generally required to be carried out on the data, the generalization capability of the model is improved, and the accuracy of the model on human body key point identification is further improved. In general, the usual data enhancement modes are: firstly, turning over a picture with random probability level, then taking a human body detection frame as a center, intercepting a human body area according to a fixed size, finally carrying out affine transformation on the intercepted area, rotating at random angles, and finally reducing average value and standardizing to send the image into a network for training.
Disclosure of Invention
The embodiment of the application provides a human body key point identification method, device and equipment and a storage medium.
According to a first aspect, there is provided a human body key point recognition method, the method comprising: acquiring a target image of a key point of a human body to be identified; inputting a target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image, wherein the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true heat map of each human body key point, the radial action range of the true heat map of each human body key point is determined based on scale parameters, and the scale parameters are determined according to the distance values of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points; and determining the coordinates of each human body key point according to the heat map of each human body key point in the target image.
According to a second aspect, there is provided a human body key point recognition apparatus, the apparatus comprising: the acquisition module is configured to acquire a target image of a key point of a human body to be identified; the training module is configured to input a target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image, wherein the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true heat map of each human body key point, the radial action range of the true heat map of each human body key point is determined based on scale parameters, and the scale parameters are determined according to the distance values of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points; and the determining module is configured to determine coordinates of each human body key point according to the heat map of each human body key point in the target image.
According to a third aspect, there is provided an electronic device comprising one or more processors; and a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement the human body keypoint identification method as any of the embodiments of the first aspect.
According to a fourth aspect, there is provided a computer readable medium having stored thereon a computer program which when executed by a processor implements the human keypoint identification method as in any of the embodiments of the first aspect.
The method comprises the steps of obtaining target images of key points of a human body to be identified; inputting a target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image, wherein the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true value heat map of each human body key point, the radial action range of the true value heat map of each human body key point is determined based on scale parameters, the scale parameters are determined according to the distance value of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points, namely the scale parameters are distinguished according to the distance value, so that the situation that the scale parameters are set to a larger fixed value according to actual experience to be unfavorable for model regression training or the scale parameters are set to be a smaller fixed value according to actual experience to cause an error to become larger is avoided, and the accuracy of the coordinates of each human body key point determined by the heat map of each human body key point in the target image is further improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
FIG. 1 is an exemplary system architecture diagram in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of a human keypoint identification method in accordance with the present application;
FIG. 3 is a schematic diagram of an application scenario of a human keypoint identification method according to the present application;
FIG. 4 is a flow chart of one embodiment of a method of determining a truncated human image labeled with a true heat map for each human keypoint in accordance with the human keypoint identification method embodiment shown in FIG. 2;
FIG. 5 is a schematic diagram of one embodiment of a human keypoint identification device according to the present application;
FIG. 6 is a schematic diagram of a computer system suitable for use in implementing embodiments of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which an embodiment of the human keypoint identification method of the present application may be applied.
As shown in fig. 1, a system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium to provide communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The user may interact with the server 105 via the network 104 using the terminal devices 101, 102, 103 to receive or send messages or the like. Various communication client applications, such as an image recognition class application, an image processing class application, and the like, may be installed on the terminal devices 101, 102, 103.
The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices with display screens, including but not limited to smartphones, tablets, laptop and desktop computers, and the like. When the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices. It may be implemented as multiple software or software modules (e.g., to provide human keypoint identification) or as a single software or software module. The present invention is not particularly limited herein.
The server 105 may be a server that provides various services, for example, identifies a target image (for example, a video frame including a human body image obtained by photographing) of a human body key point to be identified uploaded by the terminal devices 101, 102, 103, inputs the target image into a human body key point detection model trained in advance, obtains a heat map of each human body key point in the target image, and determines coordinates of each human body key point according to the heat map of each human body key point in the target image.
It should be noted that, the human body key point recognition method provided in the embodiment of the present application is generally executed by the server 105, and accordingly, the device for human body key point recognition is generally disposed in the server 105.
It should be noted that, the target image of the human body key point to be identified may also be directly stored locally in the server 105, and the server 105 may directly extract the target image of the local human body key point to be identified to identify the human body key point, where the exemplary system architecture 100 may not include the terminal devices 101, 102, 103 and the network 104.
It should be further noted that the application of human body key point recognition may be installed in the terminal devices 101, 102, 103, and the terminal devices 101, 102, 103 may also perform human body key point recognition based on the target image of the human body key point to be recognized, and at this time, the human body key point recognition method may also be performed by the terminal devices 101, 102, 103, and accordingly, the human body key point recognition device may also be provided in the terminal devices 101, 102, 103. At this point, the exemplary system architecture 100 may also not include the server 105 and the network 104.
The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster formed by a plurality of servers, or as a single server. When the server is software, it may be implemented as a plurality of software or software modules (for example, to provide a human key point recognition service), or may be implemented as a single software or software module. The present invention is not particularly limited herein.
It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Fig. 2 shows a flow diagram 200 of an embodiment of a human keypoint identification method that may be applied to the present application. The human body key point identification method comprises the following steps:
step 201, obtaining a target image of a key point of a human body to be identified.
In this embodiment, the execution subject may acquire the target image of the key point of the human body to be identified from the terminal device or the local preset database.
The target image of the key point of the human body to be identified can include various images, such as video frames of video, images downloaded by a network, images acquired by an image acquisition device, and the like.
Here, the target image of the human body key point to be identified is an image including a human body region. Specifically, the execution subject may first acquire an initial image including a human body region, perform human body region detection on the initial image including the human body region, then extract the detected human body region from the initial image, and determine the extracted human body region as a target image of a human body key point to be identified.
The human body region detection may be a detection technique for detecting feature information of a human body region in the prior art or a future development technique, for example, detection of a human body region by using a human body region detection model trained in advance, which is not limited in this application.
Furthermore, it should be noted that body keypoints are used to locate body key parts of the human body, such as parts of the head, neck, shoulders, hands, legs, and/or feet; for any body critical site, one or more body critical points may be required in locating the body critical site. In different situations, specific positions and numbers of key points of the human body can be different due to different key parts of the body to be positioned, and the application is not limited to the specific positions and the number of the key points.
Step 202, inputting the target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image.
In this embodiment, the execution subject inputs the obtained target image including the human body region to a human body key point detection model trained in advance, so as to obtain a heat map of each human body key point in the target image.
The heat map of each human body key point is a probability distribution map of a possible position of the human body key point.
Here, the human body key point detection model trained in advance is obtained based on a sample image labeled with a true value heat map of each human body key point.
The network structure of the pre-trained human keypoint detection model may include a variety, and in particular, the pre-trained human keypoint detection model may include a feature extraction network for extracting image features and a heat map extraction network for generating a heat map based on the image features, which may be implemented by a convolution set, the feature extraction network may include, but is not limited to, the following: leNet, resNet, mobileNet, etc.
It should be noted that the radial range of action of the true heat map for each human critical point is determined based on the scale parameters.
Specifically, the true heat map of each human critical point satisfies a gaussian probability distribution, which can be expressed by the following heat map generation formula:
wherein,,x is the two-dimensional coordinates of the pixel points of the true heat map of the human key points, mu is the true two-dimensional coordinates of the human key points, and delta is the scale parameter.
From the above formula, it can be seen that the radial action range of the true heat map is determined based on the scale parameter δ, the larger the scale parameter δ is, the flatter the gaussian distribution is, the lower the center value is, whereas the smaller the scale parameter δ is, the steeper the gaussian distribution is, and the higher the center value is.
Typically, the scale parameter δ is a fixed value. The dimension parameter delta cannot be set too large or too small, if the dimension parameter delta is set too large, gaussian distribution between two adjacent human body key points is easy to overlap and interweave, the error becomes large, and if the dimension parameter delta is set too small, the central value of the human body key points is too large, so that regression training of a model is not facilitated.
In this embodiment, the scale parameter is determined according to a distance value of coordinates between the human body key point corresponding to the scale parameter and the rest of human body key points.
Specifically, the execution body may compare a minimum value of distance values of coordinates between the human body key points corresponding to the scale parameters and the rest of human body key points with a preset distance value, and if the minimum value is smaller than the preset distance value, that is, the human body key points corresponding to the scale parameters are in a region where the human body key points are denser, for example, a human face part, the scale parameters may be set to a smaller value according to actual experience, so as to reduce overlapping of gaussian distribution values between the human body key points; if the minimum value is greater than or equal to the preset distance value, that is, the human body key points corresponding to the scale parameters are in the region where the human body key points are sparse, such as shoulders, legs and the like, the scale parameters can be set to be larger values according to actual experience, so that regression training of the model is facilitated.
Here, the small value and the large value may be fixed values, or may be values determined based on the smallest value among the distance values of the coordinates between the human body key point corresponding to the scale parameter and the remaining human body key points, which is not limited in this application.
Further, it should be noted that, since the input image of the pre-trained human body keypoint detection model is generally a fixed size, and the size of the target image of the human body keypoint to be identified is not determined, the size of the target image may be adjusted to the size of the input image of the pre-trained human body keypoint detection model before the target image is input to the pre-trained human body keypoint detection model.
In some alternatives, the determining of the scale parameter according to the distance value of the coordinates between the corresponding human body key point and the rest of the human body key points includes: if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is larger than or equal to a preset distance value, the scale parameters are preset fixed values; if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is smaller than the preset distance value, the scale parameters are determined according to the minimum value.
In this implementation manner, if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest of the human body key points is greater than or equal to the preset distance value, that is, the human body key points corresponding to the scale parameters are in the region where the human body key points are sparse, for example, the shoulders, the legs and the like, the scale parameters can be set to a larger preset fixed value so as to enable the model to quickly return, and the training efficiency of the model is ensured.
If the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest of the human body key points is smaller than the preset distance value, that is, the human body key points corresponding to the scale parameters are located in a region where the human body key points are relatively dense, for example, a human face part, the scale parameters can be determined according to the minimum value so as to avoid overlapping and interweaving of Gaussian distribution between two adjacent human body key points, and errors become larger, so that the accuracy of human body key point detection model on human body key point identification is affected.
In particular, the scale parameter may be proportional to the minimum value in the distance, i.e. the scale parameter increases with increasing minimum value, and the proportional scaling factor may be determined according to practical experience and specific application scenario.
The implementation mode is that the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is compared with a preset distance value, and the scale parameters, of which the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is greater than or equal to the preset distance value, are set to be fixed values; the scale parameters, of which the minimum value is smaller than the preset distance value, in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points are determined according to the minimum value, so that the training efficiency of the human body key point detection model is improved, and meanwhile, the accuracy of identifying the human body key points is guaranteed.
And 203, determining coordinates of each human body key point according to the heat map of each human body key point in the target image.
In this embodiment, after the execution subject obtains the heat map of each human body key point in the target image, the coordinate of each human body key point may be determined according to the heat map of each human body key point in the target image based on a preset recognition mode.
The preset identification modes can include various modes.
For example, the coordinates of each human body key point are determined from the heat map of each human body key point in the target image according to the following predetermined first calculation formula.
Wherein the predetermined first calculation formula may include:
Ix＝hx*s1；
Iy＝hy*s2；
here, ix and Iy are respectively the abscissa and the ordinate of the human body key point I, and hx and hy are respectively the abscissa and the ordinate of the pixel point with the largest value in the heat map of the human body key point I; s1 is a reduction coefficient of the output content of the pre-trained human body key point detection model relative to the input content in the x-axis direction, and s2 is a reduction coefficient of the output content of the pre-trained human body key point detection model relative to the input content in the y-axis direction.
For another example, the coordinates of each human body key point and the like are determined according to the heat map of each human body key point in the target image according to the following predetermined first calculation formula, which is not limited in this application.
The predetermined first calculation formula corrected formula may include:
Ix＝hx*s1+R1；
Iy＝hy*s2+R2；
wherein R1 and R2 are preset correction factors.
With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the human body key point recognition method according to the present embodiment.
In the application scenario of fig. 3, an execution subject 301 acquires a target image 302 of a human body key point to be identified; inputting a target image 303 into a pre-trained human body key point detection model 303 to obtain a heat map 304 of each human body key point in the target image, wherein the pre-trained human body key point detection model 303 is obtained by training based on a sample image marked with a true value heat map of each human body key point, the radial action range of the true value heat map of each human body key point is determined based on scale parameters, the minimum value in the distance values of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points is compared with a preset distance value, and the scale parameters are determined according to the comparison result; finally, the coordinates 305 of each human body key point are determined according to the heat map 304 of each human body key point in the target image.
According to the human body key point identification method provided by the embodiment of the disclosure, the target image of the human body key point to be identified is obtained; inputting a target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image, wherein the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true heat map of each human body key point, the radial action range of the true heat map of each human body key point is determined based on scale parameters, and the scale parameters are determined according to the distance values of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points; according to the heat map of each human body key point in the target image, the coordinates of each human body key point are determined, and the accuracy of detecting the human body key points is effectively improved.
Referring further to FIG. 4, a flow 400 of one embodiment of a method of determining a truncated human image labeled with a true heat map for each human keypoint in the human keypoint identification method embodiment shown in FIG. 2 is illustrated. Here, the image of the truncated human body labeled with the true heat map for each human body keypoint is used to train the pre-trained human body keypoint detection model in fig. 2 described above. In this embodiment, the process 400 of obtaining the image of the truncated human body marked with the true heat map of each human body key point may include the following steps:
In step 401, an original image is acquired.
In this embodiment, the execution subject may acquire an original image from a terminal device or a local preset database, where the original image includes a human body image and a human body detection frame labeled with coordinates of each human body key point.
The human body detection frame is used for marking the human body image in the original image. Typically, each human detection frame includes a complete human image.
Step 402, in the original image, capturing an image of a first preset size centered on the center of the human body detection frame and bounded by the human body detection frame, to obtain a first image.
In this embodiment, the first preset size is determined according to the size of the human body detection frame, where the size of the human body detection frame may be determined according to practical experience and a specific application scenario. Specifically, the human body detection frame may be square or rectangular.
In some optional manners, before capturing, in the original image, an image of a first preset size centered on a center of the human body detection frame and bounded by the human body detection frame, and obtaining a first image, the method further includes: and rotating the human body detection frame by random probability and random angle.
In this implementation manner, before the execution subject intercepts the human body detection frame of the original image, the execution subject may further perform data enhancement on the image data in the human body detection frame by performing random probability and random angle rotation on the human body detection frame, so as to further enhance the generalization capability of the human body key point detection model.
And step 403, expanding the human body detection frame in the first image by a second preset size to obtain a second image.
In this embodiment, the execution subject expands the human body detection frame in the first image by a second preset size and fills the second preset size with a value of 0 pixels to obtain a second image.
The second preset size may be determined according to practical experience and a specific application scenario, and preferably, an area of an image area corresponding to the second preset size may be greater than or equal to an area of an image area corresponding to a human body detection frame size in the first image, so as to ensure a better cutting effect. For example, the size of the human body detection frame in the first image is d×d, and after the length and width of the human body detection frame are respectively expanded by 50%, the size of the second image is 1.5d×1.5d.
And step 404, randomly intercepting the image with the first preset size from the second image to obtain an image of the truncated human body marked with the coordinates of each human body key point.
In this embodiment, after the execution body obtains the second image, the execution body randomly intercepts the image of the first preset size to obtain an image of the truncated human body marked with the coordinates of the key point of each human body.
Here, the execution subject may further add gaussian noise or pretzel noise of random size at random to further perform data enhancement after acquiring the image of the truncated human body marked with coordinates of each human body keypoint.
Step 405, obtaining an image of the truncated human body marked with the true heat map of each human body key point according to the image of the truncated human body marked with the coordinates of each human body key point.
In this embodiment, after the execution subject obtains the image of the truncated human body labeled with the coordinates of each human body key point, the execution subject may first perform coordinate transformation on the coordinates of each human body key point in the image according to the transformation to obtain the true value coordinates of each human body key point, and then obtain the image of the truncated human body labeled with the true value heat map of each human body key point according to the true value coordinates of each human body key point.
The true heat map of each human critical point satisfies gaussian probability distribution, and can be represented by the heat map generation formula, which is not described herein.
In some alternative ways, the number of human body key points of the image of the truncated human body of the truth heat map labeled with each human body key point is greater than or equal to a preset number value.
In the implementation manner, the execution main body limits the number of the human body key points of the image of the truncated human body marked with the true value heat map of each human body key point, so that the influence of the human body key point detection model on the accuracy of human body key point detection due to the fact that the number of the human body key points of the image of the truncated human body marked with the true value heat map of each human body key point is too small can be effectively avoided.
The above embodiments of the present application mainly describe the process of acquiring the image of the truncated human body marked with the true heat map of each human body key point in the embodiment of the human body key point identification method shown in fig. 2. Here, the image of the truncated human body labeled with the true heat map for each human body keypoint is used to train the pre-trained human body keypoint detection model in fig. 2. According to the embodiment, the training data of the human body cut-off scene is increased by acquiring the image of the cut-off human body marked with the true value heat map of each human body key point, and the acquired image of the cut-off human body marked with the true value heat map of each human body key point is used for training the pre-trained human body key point detection model, so that the detection precision of the pre-trained human body key point detection model on the human body cut-off scene is improved, and the detection accuracy of the human body key points is further effectively improved.
With further reference to fig. 5, as an implementation of the method shown in the foregoing drawings, the present application provides an embodiment of a human body key point recognition apparatus, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 1, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 5, the human body key point recognition device 500 of the present embodiment includes: an acquisition module 501, a training module 502, a determination module 503.
The acquiring module 501 may be configured to acquire a target image of a key point of a human body to be identified.
The training module 502 may be configured to input the target image into a pre-trained human body key point detection model, to obtain a heat map of each human body key point in the target image, where the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true heat map of each human body key point, and the radial acting range of the true heat map of each human body key point is determined based on scale parameters, where the scale parameters are determined according to distance values of coordinates between the human body key point corresponding to the scale parameters and the rest of the human body key points.
The obtaining module 503 may be configured to determine coordinates of each human body key point according to the heat map of each human body key point in the target image.
In some optional manners of this embodiment, determining the scale parameter according to the distance value of the coordinates between the corresponding human body key point and the rest of the human body key points includes: if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is larger than or equal to a preset distance value, the scale parameters are preset fixed values; and if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is smaller than a preset distance value, determining the scale parameters according to the minimum value.
In some alternatives of this embodiment, the sample image labeled with the true heat map for each human keypoint includes: cut-off human body images marked with true value heat maps of key points of each human body; the image of the truncated human body marked with the true heat map of each human body key point is obtained by: acquiring an original image, wherein the original image comprises a human body image marked with coordinates of each human body key point and a human body detection frame; in the original image, capturing an image of a first preset size taking the center of a human body detection frame as the center and taking the human body detection frame as the boundary to obtain a first image; expanding the human body detection frame in the first image by a second preset size to obtain a second image; randomly intercepting an image with a first preset size from the second image to obtain an image of a truncated human body marked with coordinates of each human body key point; and obtaining the image of the truncated human body marked with the true heat map of each human body key point according to the image of the truncated human body marked with the coordinates of each human body key point.
In some optional manners of this embodiment, the number of human body key points of the image of the truncated human body, which is labeled with the true heat map of each human body key point, is greater than or equal to a preset number value.
In some optional manners of this embodiment, before capturing, in the original image, an image of a first preset size centered on a center of the human body detection frame and bounded by the human body detection frame, and obtaining the first image, the method further includes: and rotating the human body detection frame by random probability and random angle.
According to embodiments of the present application, an electronic device and a readable storage medium are also provided.
As shown in fig. 6, a block diagram of an electronic device according to a human body key point recognition method according to an embodiment of the present application is shown.
600 is a block diagram of an electronic device of a human body key point recognition method according to an embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 6, the electronic device includes: one or more processors 601, memory 602, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 601 is illustrated in fig. 6.
Memory 602 is a non-transitory computer-readable storage medium provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the human body key point identification method provided by the application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the human body key point recognition method provided by the present application.
The memory 602 is used as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., the acquisition module 501, the training module 502, and the determination module 503 shown in fig. 5) corresponding to the human keypoint identification method in the embodiments of the present application. The processor 601 executes various functional applications of the server and data processing, i.e., implements the human keypoint identification method in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 602.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created by the use of the face tracked electronic device, and the like. In addition, the memory 602 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 602 may optionally include memory located remotely from processor 601, which may be connected to electronic devices identified by human keypoints via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the human body key point identification method may further include: an input device 603 and an output device 604. The processor 601, memory 602, input device 603 and output device 604 may be connected by a bus or otherwise, for example in fig. 6.
The input device 603 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device for quality monitoring of the live video stream, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. input devices. The output means 604 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, the accuracy of detecting the key points of the human body is effectively improved.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (10)
1. A human body key point identification method comprises the following steps:
acquiring a target image of a key point of a human body to be identified;
inputting the target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image, wherein the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true heat map of each human body key point, the radial action range of the true heat map of each human body key point is determined based on scale parameters, and the scale parameters are determined according to the distance values of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points, and the method comprises the following steps: if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is smaller than a preset distance value, determining the scale parameters according to the minimum value, and if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is larger than or equal to the preset distance value, determining the scale parameters as a preset fixed value;
And determining the coordinates of each human body key point according to the heat map of each human body key point in the target image.
2. The method of claim 1, wherein the sample image labeled with the true heat map for each human keypoint comprises: cut-off human body images marked with true value heat maps of key points of each human body; and the image of the truncated human body marked with the true heat map of each human body key point is obtained by the following steps:
acquiring an original image, wherein the original image comprises a human body image marked with coordinates of each human body key point and a human body detection frame;
in the original image, capturing an image of a first preset size taking the center of a human body detection frame as the center and taking the human body detection frame as the boundary to obtain a first image;
expanding the human body detection frame in the first image by a second preset size to obtain a second image;
randomly intercepting an image with a first preset size from the second image to obtain an image of a truncated human body marked with coordinates of each human body key point;
and obtaining the image of the truncated human body marked with the true heat map of each human body key point according to the image of the truncated human body marked with the coordinates of each human body key point.
3. The method of claim 2, wherein the number of human keypoints of the truncated human image labeled with the true heat map for each human keypoint is greater than or equal to a preset number value.
4. The method of claim 2, wherein, in the original image, capturing an image of a first preset size centered on a center of a human detection frame and bounded by the human detection frame, and before obtaining the first image, the method further comprises:
and rotating the human body detection frame by random probability and random angle.
5. A human body keypoint identification device comprising:
the acquisition module is configured to acquire a target image of a key point of a human body to be identified;
the training module is configured to input the target image into a pre-trained human body key point detection model to obtain a heat map of each human body key point in the target image, the pre-trained human body key point detection model is obtained by training based on a sample image marked with a true heat map of each human body key point, wherein the radial action range of the true heat map of each human body key point is determined based on scale parameters, and the scale parameters are determined according to the distance values of coordinates between the human body key point corresponding to the scale parameters and the rest human body key points, and the training module comprises: if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is smaller than a preset distance value, determining the scale parameters according to the minimum value, and if the minimum value in the distance values of the coordinates between the human body key points corresponding to the scale parameters and the rest human body key points is larger than or equal to the preset distance value, determining the scale parameters as a preset fixed value;
And the determining module is configured to determine coordinates of each human body key point according to the heat map of each human body key point in the target image.
6. The apparatus of claim 5, wherein the sample image labeled with the true heat map for each human keypoint comprises: cut-off human body images marked with true value heat maps of key points of each human body; and the image of the truncated human body marked with the true heat map of each human body key point is obtained by the following steps:
acquiring an original image, wherein the original image comprises a human body image marked with coordinates of each human body key point and a human body detection frame;
in the original image, capturing an image of a first preset size taking the center of a human body detection frame as the center and taking the human body detection frame as the boundary to obtain a first image;
expanding the human body detection frame in the first image by a second preset size to obtain a second image;
randomly intercepting an image with a first preset size from the second image to obtain an image of a truncated human body marked with coordinates of each human body key point;
and obtaining the image of the truncated human body marked with the true heat map of each human body key point according to the image of the truncated human body marked with the coordinates of each human body key point.
7. The apparatus of claim 6, wherein the number of human keypoints of the truncated human image labeled with the true heat map for each human keypoint is greater than or equal to a preset number value.
8. The apparatus of claim 6, wherein, in the original image, an image of a first preset size centered on a center of a human detection frame and bordered by the human detection frame is taken, and before obtaining the first image, the apparatus further comprises:
and rotating the human body detection frame by random probability and random angle.
9. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.
10. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010327619.9A CN111523468B (en) | 2020-04-23 | 2020-04-23 | Human body key point identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010327619.9A CN111523468B (en) | 2020-04-23 | 2020-04-23 | Human body key point identification method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111523468A CN111523468A (en) | 2020-08-11 |
| CN111523468B true CN111523468B (en) | 2023-08-08 |
Family
ID=71904306
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010327619.9A Active CN111523468B (en) | 2020-04-23 | 2020-04-23 | Human body key point identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111523468B (en) |
Families Citing this family (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112073691A (en) * | 2020-09-11 | 2020-12-11 | 中国石油集团西南管道有限公司 | Building site safety monitoring system and method for pipeline engineering construction |
| CN112163479A (en) * | 2020-09-16 | 2021-01-01 | 广州华多网络科技有限公司 | Motion detection method, motion detection device, computer equipment and computer-readable storage medium |
| CN112232194A (en) * | 2020-10-15 | 2021-01-15 | 广州云从凯风科技有限公司 | Single-target human body key point detection method, system, equipment and medium |
| CN112270669B (en) * | 2020-11-09 | 2024-03-01 | 北京百度网讯科技有限公司 | Human body 3D key point detection method, model training method and related devices |
| CN112330730B (en) * | 2020-11-27 | 2024-02-20 | 北京百度网讯科技有限公司 | Image processing method, device, equipment and storage medium |
| CN112528850B (en) | 2020-12-11 | 2024-06-04 | 北京百度网讯科技有限公司 | Human body identification method, device, equipment and storage medium |
| CN112966599B (en) * | 2021-03-04 | 2023-07-28 | 北京百度网讯科技有限公司 | Training method of key point recognition model, key point recognition method and device |
| CN113177468B (en) * | 2021-04-27 | 2023-10-27 | 北京百度网讯科技有限公司 | Human behavior detection method and device, electronic equipment and storage medium |
| CN113421196B (en) * | 2021-06-08 | 2023-08-11 | 杭州逗酷软件科技有限公司 | Image processing method and related device |
| CN113435358B (en) * | 2021-06-30 | 2023-08-11 | 北京百度网讯科技有限公司 | Sample generation method, device, equipment and program product for training model |
| CN114186632B (en) * | 2021-12-10 | 2023-04-18 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for training key point detection model |
| CN114519666B (en) * | 2022-02-18 | 2023-09-19 | 广州方硅信息技术有限公司 | Live image correction method, device, equipment and storage medium |
| CN116246351B (en) * | 2023-05-11 | 2023-07-18 | 天津医科大学第二医院 | Image processing-based old person gait recognition method and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107256225A (en) * | 2017-04-28 | 2017-10-17 | 济南中维世纪科技有限公司 | A kind of temperature drawing generating method and device based on video analysis |
| US10353526B1 (en) * | 2018-02-07 | 2019-07-16 | Disney Enterprises, Inc. | Room-scale interactive and context-aware sensing |
| CN110175528A (en) * | 2019-04-29 | 2019-08-27 | 北京百度网讯科技有限公司 | Human body tracing method and device, computer equipment and readable medium |
| CN110334599A (en) * | 2019-05-31 | 2019-10-15 | 北京奇艺世纪科技有限公司 | Training method, device, equipment and the storage medium of deep learning network |
| CN110427917A (en) * | 2019-08-14 | 2019-11-08 | 北京百度网讯科技有限公司 | Method and apparatus for detecting key point |
| CN110781765A (en) * | 2019-09-30 | 2020-02-11 | 腾讯科技（深圳）有限公司 | Human body posture recognition method, device, equipment and storage medium |
| CN110909665A (en) * | 2019-11-20 | 2020-03-24 | 北京奇艺世纪科技有限公司 | Multitask image processing method and device, electronic equipment and storage medium |
| CN110929638A (en) * | 2019-11-20 | 2020-03-27 | 北京奇艺世纪科技有限公司 | Human body key point identification method and device and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| AU2016315938B2 (en) * | 2015-08-31 | 2022-02-24 | Cape Analytics, Inc. | Systems and methods for analyzing remote sensing imagery |
| US10586350B2 (en) * | 2017-12-03 | 2020-03-10 | Facebook, Inc. | Optimizations for dynamic object instance detection, segmentation, and structure mapping |
-
2020
- 2020-04-23 CN CN202010327619.9A patent/CN111523468B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107256225A (en) * | 2017-04-28 | 2017-10-17 | 济南中维世纪科技有限公司 | A kind of temperature drawing generating method and device based on video analysis |
| US10353526B1 (en) * | 2018-02-07 | 2019-07-16 | Disney Enterprises, Inc. | Room-scale interactive and context-aware sensing |
| CN110175528A (en) * | 2019-04-29 | 2019-08-27 | 北京百度网讯科技有限公司 | Human body tracing method and device, computer equipment and readable medium |
| CN110334599A (en) * | 2019-05-31 | 2019-10-15 | 北京奇艺世纪科技有限公司 | Training method, device, equipment and the storage medium of deep learning network |
| CN110427917A (en) * | 2019-08-14 | 2019-11-08 | 北京百度网讯科技有限公司 | Method and apparatus for detecting key point |
| CN110781765A (en) * | 2019-09-30 | 2020-02-11 | 腾讯科技（深圳）有限公司 | Human body posture recognition method, device, equipment and storage medium |
| CN110909665A (en) * | 2019-11-20 | 2020-03-24 | 北京奇艺世纪科技有限公司 | Multitask image processing method and device, electronic equipment and storage medium |
| CN110929638A (en) * | 2019-11-20 | 2020-03-27 | 北京奇艺世纪科技有限公司 | Human body key point identification method and device and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 基于多任务学习的人脸及关键点检测算法研究;李晓光;中国优秀硕士学位论文全文数据库 (基础科学辑);I138-914 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111523468A (en) | 2020-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111523468B (en) | Human body key point identification method and device | |
| US11756332B2 (en) | Image recognition method, apparatus, device, and computer storage medium | |
| CN111986178A (en) | Product defect detection method and device, electronic equipment and storage medium | |
| US11790553B2 (en) | Method and apparatus for detecting target object, electronic device and storage medium | |
| CN112150551B (en) | Object pose acquisition method and device and electronic equipment | |
| US20210209774A1 (en) | Image adjustment method and apparatus, electronic device and storage medium | |
| CN111612852B (en) | Method and apparatus for verifying camera parameters | |
| CN111767853B (en) | Lane line detection method and device | |
| CN111722245B (en) | Positioning method, positioning device and electronic equipment | |
| CN111260569A (en) | Method and device for correcting image inclination, electronic equipment and storage medium | |
| CN111539347B (en) | Method and device for detecting target | |
| CN111695519B (en) | Method, device, equipment and storage medium for positioning key point | |
| CN111832396B (en) | Method and device for analyzing document layout, electronic equipment and storage medium | |
| CN112241716B (en) | Training sample generation method and device | |
| JP7164589B2 (en) | Indoor positioning method, device, equipment and storage medium | |
| JP7389824B2 (en) | Object identification method and device, electronic equipment and storage medium | |
| US20210334985A1 (en) | Method and apparatus for tracking target | |
| CN111523467B (en) | Face tracking method and device | |
| CN111275827B (en) | Edge-based augmented reality three-dimensional tracking registration method and device and electronic equipment | |
| CN111462179B (en) | Three-dimensional object tracking method and device and electronic equipment | |
| CN111191619B (en) | Method, device and equipment for detecting virtual line segment of lane line and readable storage medium | |
| CN111783639A (en) | Image detection method and device, electronic equipment and readable storage medium | |
| CN112508964B (en) | Image segmentation method, device, electronic equipment and storage medium | |
| CN111767859A (en) | Image correction method and device, electronic equipment and computer-readable storage medium | |
| CN111552829A (en) | Method and apparatus for analyzing image material |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |