CN112507954B - Human body key point identification method and device, terminal equipment and storage medium - Google Patents
Human body key point identification method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN112507954B CN112507954B CN202011518131.0A CN202011518131A CN112507954B CN 112507954 B CN112507954 B CN 112507954B CN 202011518131 A CN202011518131 A CN 202011518131A CN 112507954 B CN112507954 B CN 112507954B
- Authority
- CN
- China
- Prior art keywords
- human body
- key points
- heat map
- key point
- coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000001514 detection method Methods 0.000 claims abstract description 34
- 238000010586 diagram Methods 0.000 claims abstract description 24
- 238000004590 computer program Methods 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 4
- 239000003086 colorant Substances 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 4
- 210000003127 knee Anatomy 0.000 description 10
- 210000003423 ankle Anatomy 0.000 description 8
- 230000003068 static effect Effects 0.000 description 7
- 210000000707 wrist Anatomy 0.000 description 6
- 210000001331 nose Anatomy 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 210000003739 neck Anatomy 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 210000000988 bone and bone Anatomy 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000036544 posture Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Geometry (AREA)
- General Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention is applicable to the technical field of artificial intelligence, and provides a human body key point identification method, a device, terminal equipment and a storage medium, wherein key points at N joint positions of a human body in a human body image are detected through a key point detection model, and N Zhang Retu and a background image output by the key point detection model are obtained; dividing the areas where the key points of different human bodies are located in each heat map, and determining the outline of the areas where the key points of different human bodies are located in each heat map; acquiring a heat map peak value in the outline of the key point area; obtaining coordinates of key points of different human bodies in each heat map according to coordinates of heat map peak values in the outline of the region where the key points of different human bodies are located in each heat map; according to the coordinates of the key points of each human body, the joint characteristic diagram of each human body is drawn, so that the recognition accuracy of the key points of the human body can be effectively improved.
Description
Technical Field
The invention belongs to the technical field of artificial intelligence (Artificial Intelligence, AI), and particularly relates to a human body key point identification method, a device, terminal equipment and a storage medium.
Background
With the rapid development of artificial intelligence technology, the target detection and classification technology based on neural networks is mature, and is widely applied in the industrial field, for example, the gesture of a human body is identified by using images. The human body gesture can be divided into static and dynamic, static actions (such as standing, sitting, lifting hands and the like) can be judged by means of a single frame image, and a common static action recognition method is a method based on target detection; dynamic actions (e.g., walking, jumping, running, etc.) are determined by means of a sequence of consecutive images of multiple frames, and a common dynamic action recognition method is a method based on a dual-stream network or 3D convolution, etc. The human body key point recognition is the realization basis of static action recognition, ensures the recognition precision of the human body key points, and is the key for solving the problems of low training speed, weak interpretability and the like caused by large data volume required by training, covering clothes, background and the like in the diagram in the existing static action recognition method.
Disclosure of Invention
In view of the above, the embodiments of the present invention provide a method, an apparatus, a terminal device, and a storage medium for identifying key points of a human body, which can effectively improve the accuracy of identifying key points of the human body.
A first aspect of an embodiment of the present invention provides a method for identifying key points of a human body, including:
detecting key points at N joint positions of a human body in a human body image through a key point detection model, and obtaining N Zhang Retu and a background image output by the key point detection model; wherein N is an integer greater than or equal to 2;
dividing the areas where the key points of different human bodies are located in each heat map, and determining the outline of the areas where the key points of different human bodies are located in each heat map;
acquiring a heat map peak value in the outline of the key point area;
obtaining coordinates of key points of different human bodies in each heat map according to coordinates of heat map peak values in the outline of the region where the key points of different human bodies are located in each heat map;
and drawing a joint characteristic diagram of each human body according to the coordinates of the key points of each human body.
A second aspect of an embodiment of the present invention provides a human body key point recognition apparatus, including:
the key point detection module is used for detecting key points at N joint positions of a human body in a human body image through the key point detection model and obtaining N Zhang Retu and a background image output by the key point detection model; wherein N is an integer greater than or equal to 2;
the key point segmentation module is used for segmenting out the areas where the key points of different human bodies are located in each heat map and determining the outline of the areas where the key points of different human bodies are located in each heat map;
the heat map peak value acquisition module is used for acquiring a heat map peak value in the outline of the key point area;
the key point positioning module is used for obtaining the coordinates of the key points of different human bodies in each heat map according to the coordinates of the peak values of the heat map in the outline of the area where the key points of different human bodies are located in each heat map;
and the characteristic diagram drawing module is used for drawing the joint characteristic diagram of each human body according to the coordinates of the key points of each human body.
A third aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the human body key point identification method according to the first aspect of the embodiments of the present invention when the computer program is executed by the processor.
A fourth aspect of the embodiments of the present invention provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the human keypoint identification method according to the first aspect of the embodiments of the present invention.
According to the human body key point identification method provided by the first aspect of the embodiment of the invention, key points at N joint positions of a human body in a human body image are detected through a key point detection model, and N Zhang Retu and a background image which are output by the key point detection model are obtained; dividing the areas where the key points of different human bodies are located in each heat map, and determining the outline of the areas where the key points of different human bodies are located in each heat map; acquiring a heat map peak value in the outline of the key point area; obtaining coordinates of key points of different human bodies in each heat map according to coordinates of heat map peak values in the outline of the region where the key points of different human bodies are located in each heat map; according to the coordinates of the key points of each human body, the joint feature map of each human body is drawn, so that the recognition accuracy of the key points of the human body can be effectively improved, and therefore, when the gesture of a static human body is recognized by the joint feature map drawn by the coordinates of the key points of the human body, the interferences of clothes, backgrounds and the like in the human body image can be effectively eliminated, the data volume required by training is reduced, the training speed is further improved, and the interpretation is strong.
It will be appreciated that the advantages of the second to fourth aspects may be found in the relevant description of the first aspect and are not repeated here.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a first method for identifying key points of a human body according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of 18 key point positions of a human body provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of key points of all human bodies in one human body image according to an embodiment of the present invention;
FIG. 4 is an 18-sheet heat map provided by an embodiment of the present invention;
FIG. 5 is an effect diagram obtained by connecting key points according to an embodiment of the present invention;
fig. 6 is a schematic diagram of a second flow of a human body key point recognition method according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a third flow chart of a human body key point identification method according to an embodiment of the present invention;
FIG. 8 is a diagram of 10 bones with solid background provided by an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a human body key point recognition device according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
It should be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
As used in the present description and the appended claims, the term "if" may be interpreted as "when..once" or "in response to a determination" or "in response to detection" depending on the context. Similarly, the phrase "if a determination" or "if a [ described condition or event ] is detected" may be interpreted in the context of meaning "upon determination" or "in response to determination" or "upon detection of a [ described condition or event ]" or "in response to detection of a [ described condition or event ]".
Furthermore, the terms "first," "second," "third," and the like in the description of the present specification and in the appended claims, are used for distinguishing between descriptions and not necessarily for indicating or implying a relative importance.
Reference in the specification to "one embodiment" or "some embodiments" or the like means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the invention. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," and the like in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless expressly specified otherwise. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.
The human body key point identification method provided by the embodiment of the invention can be applied to terminal equipment such as robots, mobile phones, tablet personal computers, wearable equipment, vehicle-mounted equipment, augmented reality (augmented reality, AR)/Virtual Reality (VR) equipment, notebook computers, ultra-mobile personal computer (UMPC), netbooks, personal digital assistants (personal digital assistant, PDA) and the like, and the specific types of the terminal equipment are not limited. The robot may specifically be a service robot, an underwater robot, an entertainment robot, a military robot, an agricultural robot, or the like.
As shown in fig. 1, the human body key point identification method provided by the embodiment of the invention includes the following steps S101 to S104:
step S101, detecting N key points of a human body in a human body image through a key point detection model, and obtaining N Zhang Retu and a background image which are output by the key point detection model; wherein N is an integer greater than or equal to 2.
In application, the key point detection model can be constructed based on any algorithm for detecting key points of human bones, and a human body image containing a human body is input into the key point detection model, so that N Zhang Retu and a background image output by the key point detection model can be obtained. The human body contains 18 keypoints, and the N keypoints contain at least 2 of the 18 keypoints of the human body. The human body image can be an RGB image, an infrared image or a depth image.
As shown in fig. 2, a schematic diagram illustrating 18 key point positions of a human body is exemplarily shown; wherein, 18 key points are respectively identified as: nose (Nose) 0, neck (neg) 1, right Shoulder (Right hand holder) 2, right Elbow (Right hand Elbow) 3, right Wrist (Right hand write) 4, left Shoulder (Left hand holder) 5, left Elbow (Left hand Elbow) 6, left Wrist (Left write) 7, right Hip (Right Hip) 8, right Knee (Right Knee) 9, right Ankle (Right Ankle) 10, left Hip (Left Hip) 11, left Knee (Left Knee) 12, left Ankle (Left Ankle) 13, right Eye (Right Eye) 14, left Eye (Left Eye) 15, right Ear (Right Ear) 16, and Left Ear (Left Ear) 17.
In one embodiment, the N keypoints comprise at least 14 of 18 keypoints of a human body, namely a nose, neck, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, and left ankle, and optionally at least one of a right eye, left eye, right ear, and left ear.
In the application, under the condition that the head action of the human body is not required to be identified, the N key points can only contain other key points except the head key points, so that the data volume is reduced, and the training speed is improved.
In application, only N key points of a human body to be detected in a human body image can be detected through the key point detection model, the human body to be detected does not need to be detected, each human body to be subjected to key point detection in the human body image is the human body to be detected, each human body to be subjected to key point detection is the human body to be detected, a user can set any human body in the human body image as the human body to be detected according to actual needs, for example, all human bodies in the human body image can be set as the human body to be detected. Each heat map contains the same key point of all the human bodies to be detected in the human body image, that is, the same key point of all the human bodies to be detected in the human body image corresponds to one heat map, for example, when the N key points contain 18 key points of the human bodies, 18 heat maps and one background map can be obtained, the 1 st heat map contains the noses of all the human bodies to be detected, the 2 nd heat map contains the necks of all the human bodies to be detected, … (and so on), and the N Zhang Retu th heat map contains the left ears of all the human bodies to be detected. The background image is an image which is output by the key point detection model and corresponds to the background in the human body image, and the background image which is output by the key point detection module is a blank image because no key point exists in the background of the human body image. Since the human body image does not necessarily show the complete human body, some key points in the human body image may be blocked, so some key points may not be detected, and blank images which do not contain the key points may exist in all heat maps output by the key point detection model.
As shown in fig. 3, key points of all human bodies in one human body image are exemplarily shown.
As shown in fig. 4, an exemplary 18 heat maps were output.
Step S102, dividing out the areas where the key points of different human bodies are located in each heat map, and determining the outline of the areas where the key points of different human bodies are located in each heat map;
step S103, obtaining a heat map peak value in the outline of the key point area;
step S104, obtaining coordinates of key points of different human bodies in each heat map according to coordinates of heat map peak values in the outline of the region where the key points of different human bodies are located in each heat map.
In the application, after obtaining N Zhang Retu, the coordinates of the key points in each heat map are identified, and then the coordinates of the key points belonging to the same human body are allocated to the same human body, so that the coordinates of N key points of each human body can be obtained.
In the application, the position confidence of the coordinates closer to the key points in the heat map is higher, so that the region where the key points are located in the heat map is segmented, the outline of the region where the key points are located can be identified through any outline identification method, the coordinates of the peak value of the heat map are found in the outline, the peak value of the heat map is found in the region where the key points are located, and the coordinates where the peak value of the heat map is located are used as the coordinates of the key points.
In one embodiment, step S104 includes:
and distributing the key points of different human bodies in each heat map to corresponding human bodies according to the coordinates of the key points of different human bodies in each heat map, and obtaining the coordinates of the key points of different human bodies in each heat map.
In the application, after the coordinates of each key point are obtained, the key points belonging to each human body are assigned to each human body according to the coordinates of each key point, and the coordinates of all the key points belonging to each human body are obtained.
In one embodiment, step S104 specifically includes:
and mapping the key points of different human bodies in each heat map to corresponding human bodies according to the coordinates of the key points of different human bodies in each heat map and the positions of each human body, and obtaining the coordinates of the key points of different human bodies in each heat map.
In application, the position of each human body in the human body image can be obtained by identifying the human body image through a target identification method, then the key point belonging to each human body can be obtained according to the coordinates of the key point and the position of the human body, and then each key point is mapped to the corresponding human body.
And step 105, drawing a joint feature map of each human body according to the coordinates of the key points of each human body.
In the application, after the coordinates of the key points of each human body are obtained, the obtained key points of each human body can be connected according to the connection rule of the adjacent key points in the joints of the human body, so as to obtain the joint characteristic diagram of each human body.
As shown in fig. 5, an effect diagram obtained by assigning key points to corresponding human bodies and connecting the key points is exemplarily shown.
In one embodiment, step S105 includes:
and drawing a joint characteristic diagram of each human body according to the coordinates of the key points of each human body and a preset key point connection rule.
In application, the preset key point connection rule is a rule preset according to the connection rule of adjacent key points in joints of a human body, for example, a right shoulder connection right elbow, a right elbow connection right wrist, a left shoulder connection left elbow, a left elbow connection left wrist, a neck connection right hip and left hip, a right hip connection right knee, a right knee connection right ankle, a left hip connection left knee, a left knee connection left ankle, a right ear connection right eye, a left ear connection left eye, a nose connection right eye and a left eye.
In application, the joint feature map may be an image with a solid background and only containing key points of one human body, or an image with a smooth background and containing key points of all human bodies.
As shown in fig. 6, in one embodiment, after step S105, the method further includes:
and step S601, classifying the joint feature graphs of the human bodies through a classification network to obtain gesture class labels of the human bodies.
In the application, after the joint feature graphs of each human body are drawn, the joint feature graphs of all human bodies are input into a classification network as training data, the classification network is trained, and the gesture type label of each human body, which is output after the classification network classifies the joint feature graphs of all human bodies, is obtained. The classification network may be a lightweight classification network (shufflelet-v 2) with a normalized exponential loss function (softmax loss) at the last layer.
As shown in fig. 7, in one embodiment, step S601 includes the following steps S701 to S703:
step 701, respectively identifying a left part joint and a right part joint in the joint characteristic diagram of each human body by using different colors;
step S702, adjusting the joint feature map of each identified human body to a preset size;
and step 703, training a classification network through the adjusted joint feature graphs of each human body to obtain the gesture class label of each human body output by the classification network.
In application, the left part joint and the right part joint in the joint characteristic diagram of each human body are marked by adopting different colors, so that the classification network is convenient to classify different human body postures. Since the joint feature map has no background semantics, each joint feature map does not need to be large, and therefore, the joint feature map which is too large can be reduced or cut and adjusted to a preset size, for example, 64×64 pixels. When the joint feature map is drawn based on key points of a body part of the human body other than the head, the joint feature map may be referred to as a skeleton map.
As shown in fig. 8, 10 bone maps with solid background are shown by way of example.
In one embodiment, step S702 includes:
according to the resolution ratio of the classification network, adjusting the joint feature map of each identified human body to a preset size;
step S703 includes:
training the classification network through all the adjusted joint feature graphs of the human body to obtain the posture category labels of each human body output by the normalized index loss function of the last layer of the classification network.
In the application, the size of each joint feature map can be adjusted according to the input resolution supported by the classification network, then the classification network is trained by utilizing the joint feature map, and finally the gesture class labels are output through the normalized index loss function of the classification network, namely the training of the classification network is completed.
In the application, the human body image can be an image needing to recognize the human body gesture, or can be a human body image special for training the classification network. If other human body images are to be identified, inputting the other human body images into the key point detection model, and executing the human body key point identification method provided by the embodiment of the invention again.
The human body key point identification method provided by the embodiment of the invention can effectively improve the identification precision of the key points of the human body, so that the identification precision of the key points of the human body can be effectively improved, and the interferences of clothes, backgrounds and the like in human body images can be effectively eliminated when the gesture of the static human body is identified by utilizing the joint feature map drawn by the coordinates of the key points of the human body, thereby reducing the data quantity required by training, further improving the training speed and having strong interpretability.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
The embodiment of the invention also provides a human body key point identification device which is used for executing the steps in the human body key point identification method embodiment. The human body key point recognition device may be a virtual device (virtual appliance) in the terminal device, and may be executed by a processor of the terminal device, or may be the terminal device itself.
As shown in fig. 9, the human body key point recognition device provided by the embodiment of the invention includes:
the key point detection module 101 is configured to detect key points at N joint positions of a human body in a human body image through a key point detection model, and obtain N Zhang Retu and a background image output by the key point detection model; wherein N is an integer greater than or equal to 2;
the key point segmentation module 102 is configured to segment areas where key points of different human bodies are located in each heat map, and determine outlines of the areas where the key points of different human bodies are located in each heat map;
a heat map peak value obtaining module 103, configured to obtain a heat map peak value in the outline of the keypoint region;
the key point positioning module 104 is configured to obtain coordinates of key points of different human bodies in each heat map according to coordinates of heat map peak values in the outline of the region where the key points of different human bodies are located in each heat map;
and the feature map drawing module 105 is used for drawing the joint feature map of each human body according to the coordinates of the key points of each human body.
In one embodiment, the human body key point recognition device further includes:
the classification module is used for classifying the skeleton map of each human body through a classification network to obtain the gesture category label of each human body.
In application, each module in the human body key point recognition device can be a software program module, can be realized by different logic circuits integrated in a processor, and can also be realized by a plurality of distributed processors.
As shown in fig. 10, an embodiment of the present invention further provides a terminal device 200, including: at least one processor 201 (only one processor is shown in fig. 10), a memory 202, and a computer program 203 stored in the memory 202 and executable on the at least one processor 201, the processor 201 implementing the steps in the various human keypoint identification method embodiments described above when executing the computer program 203.
In an application, the terminal device may include, but is not limited to, a memory, a processor. It will be appreciated by those skilled in the art that fig. 10 is merely an example of a terminal device and is not limiting of the terminal device, and may include more or fewer components than shown, or may combine certain components, or different components, such as may also include input and output devices, network access devices, etc.
In application, the processor may include a central processing unit and a graphics processor, which may also be other general purpose processors, digital signal processors, application specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In applications, the memory may in some embodiments be an internal storage unit of the terminal device, such as a hard disk or a memory of the terminal device. The memory may in other embodiments also be an external storage device of the terminal device, such as a plug-in hard disk provided on the terminal device, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like. Further, the memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used to store an operating system, application programs, boot Loader (Boot Loader), data, and other programs, etc., such as program code for a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.
It should be noted that, because the content of information interaction and execution process between the above devices/modules is based on the same concept as the method embodiment of the present invention, specific functions and technical effects thereof may be referred to in the method embodiment section, and will not be described herein.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to perform all or part of the functions described above. The functional modules in the embodiment may be integrated in one processing module, or each module may exist alone physically, or two or more modules may be integrated in one module, where the integrated modules may be implemented in a form of hardware or a form of software functional modules. In addition, the specific names of the functional modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working process of the modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.
The embodiment of the invention provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the human body key point identification method of any embodiment when being executed by a processor.
The embodiment of the invention provides a computer program product, which enables a terminal device to execute the human body key point identification method of any embodiment when the computer program product runs on the terminal device.
The integrated modules, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the present invention may implement all or part of the flow of the method of the above-described embodiments, and may be implemented by a computer program to instruct related hardware, and the computer program may be stored in a computer readable storage medium, where the computer program, when executed by a processor, may implement the steps of each of the method embodiments described above. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, executable files or in some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a terminal device, a recording medium, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Such as a U-disk, removable hard disk, magnetic or optical disk, etc.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
Those of ordinary skill in the art will appreciate that the various illustrative modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, terminal device and method may be implemented in other manners. For example, the apparatus, terminal device embodiments described above are merely illustrative, e.g., the division of modules is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or modules, which may be in electrical, mechanical or other forms.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (8)
1. The human body key point identification method is characterized by comprising the following steps of:
detecting key points at N joint positions of a human body in a human body image through a key point detection model, and obtaining N Zhang Retu and a background image output by the key point detection model; wherein N is an integer greater than or equal to 2;
dividing the areas where the key points of different human bodies are located in each heat map, and determining the outline of the areas where the key points of different human bodies are located in each heat map;
acquiring a heat map peak value in the outline of the key point area;
according to the coordinates of the key points of different human bodies in each heat map and the positions of each human body, mapping the key points of different human bodies in each heat map to corresponding human bodies to obtain the coordinates of the key points of different human bodies in each heat map;
and drawing a joint characteristic diagram of each human body according to the coordinates of the key points of each human body.
2. The method of claim 1, wherein the mapping the joint feature map of each of the human bodies based on coordinates of key points of each of the human bodies comprises:
and drawing a joint characteristic diagram of each human body according to the coordinates of the key points of each human body and a preset key point connection rule.
3. The method according to claim 1 or 2, wherein after the plotting the joint feature map of each human body according to the coordinates of the key points of each human body, further comprises:
and classifying the joint feature graphs of each human body through a classification network to obtain the gesture category label of each human body.
4. A method according to claim 3, wherein classifying the joint feature map of each of the human bodies through a classification network to obtain a pose class label of each of the human bodies comprises:
respectively identifying a left part joint and a right part joint in the joint characteristic diagram of each human body by using different colors;
adjusting the joint feature map of each identified human body to a preset size;
training a classification network through the adjusted joint feature graphs of each human body to obtain gesture type labels of each human body output by the classification network.
5. The method of claim 4, wherein the classification network is a lightweight classification network;
the adjusting the joint feature map of each identified human body to a preset size comprises the following steps:
according to the resolution ratio of the classification network, adjusting the joint feature map of each identified human body to a preset size;
training the classification network through the adjusted joint feature graphs of each human body to obtain gesture category labels of each human body output by the classification network, wherein the gesture category labels comprise:
training the classification network through the adjusted joint feature graphs of each human body to obtain the posture category labels of each human body output by the normalized index loss function of the last layer of the classification network.
6. A human body key point recognition device, comprising:
the key point detection module is used for detecting key points at N joint positions of a human body in a human body image through the key point detection model and obtaining N Zhang Retu and a background image output by the key point detection model; wherein N is an integer greater than or equal to 2;
the key point segmentation module is used for segmenting out the areas where the key points of different human bodies are located in each heat map and determining the outline of the areas where the key points of different human bodies are located in each heat map;
the heat map peak value acquisition module is used for acquiring a heat map peak value in the outline of the key point area;
the key point positioning module is used for mapping the key points of different human bodies in each heat map to corresponding human bodies according to the coordinates of the key points of different human bodies in each heat map and the positions of each human body, so as to obtain the coordinates of the key points of different human bodies in each heat map;
and the characteristic diagram drawing module is used for drawing the joint characteristic diagram of each human body according to the coordinates of the key points of each human body.
7. Terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the human keypoint identification method according to any one of claims 1 to 5 when the computer program is executed by the processor.
8. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the human body keypoint identification method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011518131.0A CN112507954B (en) | 2020-12-21 | 2020-12-21 | Human body key point identification method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011518131.0A CN112507954B (en) | 2020-12-21 | 2020-12-21 | Human body key point identification method and device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112507954A CN112507954A (en) | 2021-03-16 |
| CN112507954B true CN112507954B (en) | 2024-01-19 |
Family
ID=74922700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011518131.0A Active CN112507954B (en) | 2020-12-21 | 2020-12-21 | Human body key point identification method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112507954B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115966016A (en) * | 2022-12-19 | 2023-04-14 | 天翼爱音乐文化科技有限公司 | Jumping state identification method and system, electronic equipment and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108985259A (en) * | 2018-08-03 | 2018-12-11 | 百度在线网络技术（北京）有限公司 | Human motion recognition method and device |
| CN110532981A (en) * | 2019-09-03 | 2019-12-03 | 北京字节跳动网络技术有限公司 | Human body key point extracting method, device, readable storage medium storing program for executing and equipment |
| CN110942056A (en) * | 2018-09-21 | 2020-03-31 | 深圳云天励飞技术有限公司 | Clothing key point positioning method and device, electronic equipment and medium |
| CN111339903A (en) * | 2020-02-21 | 2020-06-26 | 河北工业大学 | Multi-person human body posture estimation method |
| CN111860300A (en) * | 2020-07-17 | 2020-10-30 | 广州视源电子科技股份有限公司 | Key point detection method and device, terminal equipment and storage medium |
| WO2020228217A1 (en) * | 2019-05-13 | 2020-11-19 | 河北工业大学 | Human body posture visual recognition method for transfer carrying nursing robot, and storage medium and electronic device |
| CN112101312A (en) * | 2020-11-16 | 2020-12-18 | 深圳市优必选科技股份有限公司 | Hand key point identification method and device, robot and storage medium |
-
2020
- 2020-12-21 CN CN202011518131.0A patent/CN112507954B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108985259A (en) * | 2018-08-03 | 2018-12-11 | 百度在线网络技术（北京）有限公司 | Human motion recognition method and device |
| CN110942056A (en) * | 2018-09-21 | 2020-03-31 | 深圳云天励飞技术有限公司 | Clothing key point positioning method and device, electronic equipment and medium |
| WO2020228217A1 (en) * | 2019-05-13 | 2020-11-19 | 河北工业大学 | Human body posture visual recognition method for transfer carrying nursing robot, and storage medium and electronic device |
| CN110532981A (en) * | 2019-09-03 | 2019-12-03 | 北京字节跳动网络技术有限公司 | Human body key point extracting method, device, readable storage medium storing program for executing and equipment |
| CN111339903A (en) * | 2020-02-21 | 2020-06-26 | 河北工业大学 | Multi-person human body posture estimation method |
| CN111860300A (en) * | 2020-07-17 | 2020-10-30 | 广州视源电子科技股份有限公司 | Key point detection method and device, terminal equipment and storage medium |
| CN112101312A (en) * | 2020-11-16 | 2020-12-18 | 深圳市优必选科技股份有限公司 | Hand key point identification method and device, robot and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112507954A (en) | 2021-03-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7248799B2 (en) | IMAGE PROCESSING METHOD, IMAGE PROCESSING APPARATUS, COMPUTER PROGRAM, AND IMAGE PROCESSING DEVICE | |
| US10152655B2 (en) | Deep-learning network architecture for object detection | |
| CN110874594B (en) | Human body appearance damage detection method and related equipment based on semantic segmentation network | |
| CN111178250B (en) | Object identification positioning method and device and terminal equipment | |
| CN111563502B (en) | Image text recognition method and device, electronic equipment and computer storage medium | |
| CN108734058B (en) | Obstacle type identification method, device, equipment and storage medium | |
| US20120288186A1 (en) | Synthesizing training samples for object recognition | |
| CN112084856A (en) | Face posture detection method and device, terminal equipment and storage medium | |
| CN111191582B (en) | Three-dimensional target detection method, detection device, terminal device and computer readable storage medium | |
| US20230334893A1 (en) | Method for optimizing human body posture recognition model, device and computer-readable storage medium | |
| CN110288715B (en) | Virtual necklace try-on method and device, electronic equipment and storage medium | |
| CN110852311A (en) | Three-dimensional human hand key point positioning method and device | |
| CN110363077A (en) | Sign Language Recognition Method, device, computer installation and storage medium | |
| CN112651380A (en) | Face recognition method, face recognition device, terminal equipment and storage medium | |
| CN110866469A (en) | Human face facial features recognition method, device, equipment and medium | |
| CN112507954B (en) | Human body key point identification method and device, terminal equipment and storage medium | |
| CN113971833A (en) | Multi-angle face recognition method and device, computer main equipment and storage medium | |
| CN112419326A (en) | Image segmentation data processing method, device, equipment and storage medium | |
| CN114445853A (en) | Visual gesture recognition system recognition method | |
| Jiang et al. | independent hand gesture recognition with Kinect | |
| CN115035367A (en) | Picture identification method and device and electronic equipment | |
| CN110222651A (en) | A kind of human face posture detection method, device, terminal device and readable storage medium storing program for executing | |
| CN111460910A (en) | Face type classification method and device, terminal equipment and storage medium | |
| CN112364846A (en) | Face living body identification method and device, terminal equipment and storage medium | |
| CN111598000A (en) | Face recognition method, device, server and readable storage medium based on multiple tasks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |