CN114792445A - Mining method and device for target human body posture sample, equipment and medium - Google Patents
Mining method and device for target human body posture sample, equipment and medium Download PDFInfo
- Publication number
- CN114792445A CN114792445A CN202210531488.5A CN202210531488A CN114792445A CN 114792445 A CN114792445 A CN 114792445A CN 202210531488 A CN202210531488 A CN 202210531488A CN 114792445 A CN114792445 A CN 114792445A
- Authority
- CN
- China
- Prior art keywords
- human body
- body posture
- human
- posture
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Evolutionary Computation (AREA)
- Databases & Information Systems (AREA)
- Artificial Intelligence (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a mining method, a mining device, mining equipment and mining media for target human body posture samples, relates to the technical field of artificial intelligence, and particularly relates to the technical fields of image processing, computer vision, deep learning and the like. The implementation scheme is as follows: acquiring a plurality of human body posture images, wherein the human body posture images comprise a plurality of sample human body posture images and at least one standard human body posture image; determining posture characteristic vector representation of each human body posture image based on a plurality of key points included in the human body posture image; clustering the plurality of human body posture images based on similarity between posture feature vector representations of the plurality of human body posture images to obtain a plurality of cluster clusters; selecting at least one cluster from a plurality of clusters based on a similarity criterion, wherein the human body posture images in the at least one cluster are less similar to the standard human body posture images than the human body posture images in other clusters; and determining each sample human posture image included in the at least one cluster as a target human posture sample.
Description
Technical Field
The present disclosure relates to the field of artificial intelligence technologies, and in particular, to the field of image processing, computer vision, deep learning, and the like, and in particular, to a method and an apparatus for mining a target human body posture sample, an electronic device, a computer-readable storage medium, and a computer program product.
Background
Artificial intelligence is the subject of research that makes computers simulate some human mental processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), both at the hardware level and at the software level. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises a computer vision technology, a voice recognition technology, a natural language processing technology, a machine learning/deep learning technology, a big data processing technology, a knowledge map technology and the like.
One of the difficult problems faced by the human gesture recognition technology is the diversity of human gestures, for example, the human gestures in the sports scene may include unconventional motion gestures such as yoga motions. By mining the target human posture samples meeting the requirements, the recognition effect of the human posture recognition model on the target human posture samples can be effectively improved.
The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, the problems mentioned in this section should not be considered as having been acknowledged in any prior art.
Disclosure of Invention
The disclosure provides a mining method, a mining device, an electronic device, a computer-readable storage medium and a computer program product for a target human body posture sample.
According to an aspect of the present disclosure, there is provided a mining method of a target human body posture sample, including: obtaining a plurality of human body posture images, wherein the plurality of human body posture images comprise a plurality of sample human body posture images and at least one standard human body posture image, and each human body posture image comprises a plurality of key points; determining a pose feature vector representation of each human pose image based on a plurality of key points included in each human pose image; clustering the plurality of human body posture images based on similarity between posture feature vector representations of the plurality of human body posture images to obtain a plurality of cluster clusters; selecting at least one cluster from the plurality of clusters based on a similarity criterion, wherein according to the similarity criterion, the human pose image in the at least one cluster is less similar to the at least one standard human pose image than the human pose images in other clusters of the plurality of clusters; and determining each sample human posture image included in the at least one cluster as the target human posture sample.
According to another aspect of the present disclosure, there is provided a training method of a human body posture recognition model, including: acquiring an initial human body posture recognition model; and training the initial human body posture recognition model by using a target human body posture sample, wherein the target human body posture sample is obtained by mining by using the mining method of the target human body posture sample.
According to another aspect of the present disclosure, there is provided a human body posture recognition method, including: inputting a human body posture image to be detected into a human body posture recognition model to obtain a human body posture represented by the human body posture image to be detected, which is output by the human body posture recognition model, wherein the human body posture recognition model is obtained by utilizing the training method of the human body posture recognition model.
According to another aspect of the present disclosure, there is provided a digging device of a target human body posture sample, including: a first obtaining unit configured to obtain a plurality of body pose images, the plurality of body pose images including a plurality of sample body pose images and at least one standard body pose image, each body pose image including a plurality of key points; a first determination unit configured to determine a pose feature vector representation of each human pose image based on a plurality of key points included in the each human pose image; a clustering unit configured to cluster the plurality of human body posture images based on similarity between posture feature vector representations of the plurality of human body posture images to obtain a plurality of cluster clusters; a selecting unit configured to select at least one cluster from the plurality of clusters based on a similarity criterion, wherein according to the similarity criterion, a human pose image in the at least one cluster is less similar to the at least one standard human pose image than human pose images in other clusters of the plurality of clusters; and a second determining unit configured to determine each sample human posture image included in the at least one cluster as the target human posture sample.
According to another aspect of the present disclosure, there is provided a training apparatus for a human body posture recognition model, including: a second obtaining unit configured to obtain an initial human body posture recognition model; and the training unit is configured to train the initial human body posture recognition model by using a target human body posture sample, wherein the target human body posture sample is obtained by mining by using a mining device of the target human body posture sample.
According to another aspect of the present disclosure, there is provided a human body posture recognition apparatus including: a human body posture recognition model obtained by training with a training device of the human body posture recognition model; and the input unit is configured to input the human body posture image to be detected into the human body posture recognition model so as to obtain the human body posture represented by the human body posture image to be detected output by the human body posture recognition model.
According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any of the methods described above.
According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform any of the methods described above.
According to another aspect of the disclosure, a computer program product is provided, comprising a computer program, wherein the computer program is capable of implementing any of the above methods when executed by a processor.
According to one or more embodiments of the present disclosure, target human body posture samples can be efficiently and accurately mined from a sample set.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the embodiments and, together with the description, serve to explain the exemplary implementations of the embodiments. The illustrated embodiments are for purposes of example only and do not limit the scope of the claims. Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.
FIG. 1 shows a schematic diagram of an exemplary system in which various methods described herein may be implemented, according to an exemplary embodiment of the present disclosure;
FIG. 2 shows a flow chart of a mining method of a target human pose sample according to an example embodiment of the present disclosure;
FIG. 3 shows a flowchart of a training method of a human gesture recognition model according to an example embodiment of the present disclosure;
FIG. 4 shows a flow chart of a human gesture recognition method according to an example embodiment of the present disclosure;
FIG. 5 shows a block diagram of a mining device of a target human pose sample according to an exemplary embodiment of the present disclosure;
FIG. 6 shows a block diagram of a training apparatus for a human gesture recognition model according to an exemplary embodiment of the present disclosure;
fig. 7 illustrates a block diagram of a human body gesture recognition apparatus according to an exemplary embodiment of the present disclosure;
FIG. 8 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, it will be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to define a positional relationship, a temporal relationship, or an importance relationship of the elements, and such terms are used only to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, based on the context, they may also refer to different instances.
The terminology used in the description of the various described examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or a plurality of. Furthermore, the term "and/or" as used in this disclosure is intended to encompass any and all possible combinations of the listed items.
By mining the target human posture sample meeting the requirement and training the model by using the obtained sample, the recognition effect of the human posture recognition model on the target human posture sample can be effectively improved. In the related art, one way is to label the target human body posture sample in the sample set manually, but the efficiency of manual labeling is low and the cost is high. The other method is to use a traditional data enhancement method to construct target human body posture samples, such as rotation, scaling and the like, but this method can only construct samples with higher difficulty in posture recognition through geometric transformation, and cannot fit the complexity of human body posture features.
Based on the above, the present disclosure provides a mining method for target human body posture samples, which is implemented by clustering a plurality of sample human body posture images and screening a sample set by screening the cluster, so that the target human body posture samples can be accurately and efficiently determined from the sample set.
Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.
Fig. 1 illustrates a schematic diagram of an example system 100 in which various methods and apparatus described herein may be implemented in accordance with embodiments of the present disclosure. Referring to fig. 1, the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. Client devices 101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.
In embodiments of the present disclosure, the server 120 may run one or more services or software applications that enable performing at least one of a mining method of target human pose samples, a training method of human pose recognition models, or a human pose recognition method.
In some embodiments, the server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of client devices 101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.
In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof, which may be executed by one or more processors. A user operating client devices 101, 102, 103, 104, 105, and/or 106 may, in turn, utilize one or more client applications to interact with server 120 to take advantage of the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100. Accordingly, fig. 1 is one example of a system for implementing the various methods described herein, and is not intended to be limiting.
The user may use client devices 101, 102, 103, 104, 105, and/or 106 to transmit the human body pose image. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that any number of client devices may be supported by the present disclosure.
The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization (e.g., one or more flexible pools of logical storage that may be virtualized to maintain virtual storage for the server). In various embodiments, the server 120 may run one or more services or software applications that provide the functionality described below.
The computing units in server 120 may run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. The server 120 can also run any of a variety of additional server applications and/or mid-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.
In some implementations, the server 120 can include one or more applications to analyze and consolidate data feeds and/or event updates received from users of the client devices 101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and 106.
In some embodiments, the server 120 may be a server of a distributed system, or a server incorporating a blockchain. The server 120 may also be a cloud server, or a smart cloud computing server or a smart cloud host with artificial intelligence technology. The cloud Server is a host product in a cloud computing service system, and is used for solving the defects of high management difficulty and weak service expansibility in the conventional physical host and Virtual Private Server (VPS) service.
The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The database 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The database 130 may be of different types. In certain embodiments, the database used by the server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data to and from the databases in response to the commands.
In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key-value stores, object stores, or conventional stores supported by a file system.
The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.
Fig. 2 shows a flow diagram of a mining method 200 of a target human pose sample according to an exemplary embodiment of the present disclosure. As shown in fig. 2, the method 200 includes:
step S201, obtaining a plurality of human body posture images, wherein the plurality of human body posture images comprise a plurality of sample human body posture images and at least one standard human body posture image, and each human body posture image comprises a plurality of key points;
step S202, determining posture characteristic vector representation of each human body posture image based on a plurality of key points included in each human body posture image;
step S203, clustering the human body posture images based on the similarity among posture characteristic vector representations of the human body posture images to obtain a plurality of cluster clusters;
step S204, selecting at least one cluster from the plurality of clusters based on a similarity criterion, wherein the human posture image in the at least one cluster is more dissimilar to the at least one standard human posture image than the human posture images in other clusters in the plurality of clusters according to the similarity criterion;
step S205, determining each sample human posture image included in the at least one cluster as the target human posture sample.
Therefore, the multiple sample human body posture images are clustered, at least one cluster is determined by screening based on the similarity criterion, and each sample human body posture image included in the at least one cluster is determined to be a target human body posture sample, so that the target human body posture sample can be accurately and efficiently determined from a sample set.
It should be understood that by clustering the plurality of human pose images based on the similarity between pose feature vector representations of the plurality of human pose images, the human pose images in each of the resulting plurality of clusters have similar pose features. Therefore, the clustering cluster is screened, so that the screening of the human body posture images of the samples can be realized, and the mining of the target human body posture samples can be accurately and efficiently realized.
Illustratively, the plurality of key points included in each body posture image in step S201 may refer to core key points in the body, such as shoulder joints, elbow joints, wrist joints, knee joints, five human faces, and the like. According to the positions of the key points, corresponding skeleton lines can be further determined, and therefore the human action posture can be intuitively indicated. For example, the skeleton line formed by connecting corresponding key points of the shoulder joint, the elbow joint and the wrist joint can indicate the motion postures of the big arm and the small arm of the human body.
According to some embodiments, the determining the pose feature vector representation of each human pose image in step S202 based on the plurality of key points included in each human pose image comprises: determining a plurality of skeleton lines corresponding to a plurality of key points included in the human body posture image; determining an included angle between every two adjacent bone lines in the plurality of bone lines to obtain a plurality of bone line included angles; and determining the posture characteristic vector representation of the human body posture image based on the plurality of bone line included angles.
The human body posture images can be represented as a graph structure consisting of the key points and the bone lines by determining a plurality of corresponding bone lines based on a plurality of key points included in each human body posture image.
In one example, the body pose image includes 19 body keypoints as shown in table 1:
table 1: key points of human body
Numbering | Name(s) | Numbering | Name(s) | Number of | Name (R) | Numbering | Name (R) |
0 | Nose | 5 | Left shoulder | 10 | Right wrist | 15 | Left foot |
1 | Left eye | 6 | Right shoulder | 11 | Left hip | 16 | Right foot |
2 | Right eye | 7 | Left elbow | 12 | Right hip | 17 | Top of head |
3 | Left ear | 8 | Right elbow | 13 | Left knee | 18 | Neck part |
4 | Right ear | 9 | Left wrist | 14 | Right knee |
Based on the above 19 human body key points, the corresponding 14 human body skeleton lines as shown in table 2 can be obtained, and the position number of each skeleton line can be represented by the numbers of its two end point key points:
table 2: human skeleton line
Location numbering | Name (R) | Location numbering | Name (R) | Location numbering | Name(s) |
(0,17) | Nose-top of head | (18,5) | Left shoulder | (14,16) | Right crus |
(0,18) | Nose-neck | (5,7) | Left big arm | (5,11) | Left shoulder-left hip |
(18,6) | Right shoulder | (7,9) | Left forearm | (11,13) | Left thigh |
(6,8) | Big right arm | (6,12) | Right shoulder-right hip | (13,15) | Left shank |
(8,10) | Right forearm | (12,14) | Right thigh |
For example, a plane rectangular coordinate system may be established with a specific key point in the body posture diagram structure as an origin, so that the position information of each key point can be indicated in the form of plane rectangular coordinates, and the vector coordinates of each bone line can be represented by the coordinates of the key point. Furthermore, by using the vector coordinates of each bone line, the included angle between every two adjacent bone lines can be calculated, and further, the included angles of a plurality of bone lines are obtained.
Illustratively, the pose feature vector representation of the human pose image may be an n-dimensional vector (x) composed of n bone line angles corresponding to the human pose image 1 ,x 2 ……x n ). Therefore, the posture characteristic of the human body posture image can be expressed by utilizing the included angles of a plurality of adjacent skeleton lines in the corresponding graph structure of the human body posture image, and the method is simpler, more convenient and more accurate.
In the above example, the pose feature vector representation of two human pose images ((x) 1 ,x 2 ……x n ) And (y) 1 ,y 2 ……y n ) The similarity between the two groups) can be calculated by the following formula:
further, according to some embodiments, the at least one standard human pose image has the same size reference value indicating size information of bone lines of the human pose image, wherein the determining an included angle between each two adjacent bone lines of the plurality of bone lines to obtain a plurality of bone line clip angles comprises: scaling the plurality of bone lines based on the size reference value to obtain a plurality of normalized bone lines; and determining an included angle between every two adjacent normalized bone lines in the plurality of normalized bone lines to obtain the plurality of bone line included angles. Therefore, all human body posture image structures can be converted to the same size scale through normalization processing, and accuracy of human body posture image clustering and screening is improved.
Illustratively, the size reference value may be a length value of a certain limb skeleton line in the human body posture diagram structure, that is, a length value of a reference edge. By comparing the length of the reference edge in the sample human body posture image with the length of the reference edge in the standard human body posture image, the corresponding scaling ratio of the sample human body posture image can be determined, and then scaling processing is carried out on a plurality of skeleton lines included in the sample human body posture image based on the scaling ratio, so that the accuracy of human body posture image clustering and screening is improved.
According to some embodiments, clustering the plurality of human pose images based on the similarity between pose feature vector representations of the plurality of human pose images in step S203 comprises: and clustering the plurality of human posture images by using a K-means clustering algorithm based on posture characteristic vector representation of the plurality of human posture images.
For example, the clustering the plurality of human posture images by using a K-means clustering algorithm based on the posture feature vector representations of the plurality of human posture images may include the following steps:
s1, randomly selecting k initial cluster centers;
and S2, clusters are distributed to the human body posture images, wherein when the cluster distribution result of any one human body posture image changes, the distance between each human body posture image and the center of each cluster is calculated (namely the posture characteristic vector of each human body posture image represents the similarity between the posture characteristic vector representation of the human body posture image corresponding to the center of each cluster), the human body posture image is distributed to the cluster closest to the distance, and the updated cluster center is determined based on the posture characteristic vector representation of all the human body posture images in each cluster.
For example, the plurality of body posture images may be clustered by using other methods based on the similarity between the posture feature vector representations of the plurality of body posture images, as long as the obtained posture feature vector representations of the body posture images of each of the plurality of clusters have a higher similarity, and the posture feature vector representations of the body posture images of different clusters have a relatively lower similarity, which is not limited.
For example, the target human body posture sample can be a difficult-to-recognize sample with a complex posture, and the corresponding standard human body posture image can be a human body posture sample with a conventional posture, such as a standing posture, a sitting posture and the like. In this case, the similarity criterion in step S204 may mean that the human posture features of the human posture image in the at least one cluster are more dissimilar to the human posture features of the at least one standard human posture image than the human posture images in other clusters, so that a cluster corresponding to a human posture image with a complex posture can be screened out.
According to some embodiments, the selecting at least one cluster from the plurality of cluster clusters based on the similarity criterion in step S204 comprises: calculating the similarity between the posture characteristic vector representation of the human body posture image corresponding to the cluster center and the posture characteristic vector representation of the at least one standard human body posture image aiming at the cluster center of each cluster in the plurality of clusters; and selecting a cluster corresponding to the cluster center in response to the similarity between the posture characteristic vector representation of the human posture image corresponding to the cluster center and the posture characteristic vector representation of the at least one standard human posture image being smaller than a preset threshold. Therefore, the attitude characteristics of the human body attitude images included in the whole cluster can be indicated by using the cluster center of each cluster, the clusters are screened based on the similarity of the attitude characteristics of the human body attitude images corresponding to the cluster centers and the standard human body attitude images, and the mining efficiency of the target human body attitude sample is effectively improved.
For example, at least one cluster may be selected from the plurality of clusters in other manners, e.g., in response to at least one standard human pose image being included in the cluster. Therefore, the posture characteristics of the human posture images corresponding to the whole cluster can be indicated by whether each cluster comprises the standard human posture image or not, the similarity of the posture characteristics among the human posture images in each cluster is fully utilized, and the cluster screening is realized more simply and conveniently.
According to another aspect of the present disclosure, a training method of a human body posture recognition model is also provided. Fig. 3 shows a flowchart of a training method 300 of a human body posture recognition model according to an exemplary embodiment of the present disclosure, as shown in fig. 3, the method 300 includes:
s301, acquiring an initial human body posture recognition model;
step S302, training the initial human body posture recognition model by using a target human body posture sample, wherein the target human body posture sample is obtained by mining by using the method 200 described above.
Therefore, the initial human body posture recognition model is trained by utilizing the target human body posture sample, and the recognition effect of the model on the target human body posture sample can be improved. For example, the recognition effect of the model on the difficultly recognized sample with a complex gesture can be improved.
For example, training the initial human posture recognition model by using the target human posture sample in step S302 may include the following processes: obtaining a target human body posture sample, wherein the target human body posture sample comprises a plurality of key points; determining a real human body posture represented by the target human body posture sample based on a plurality of key points included by the target human body posture sample; inputting the target human body posture sample into the initial human body posture recognition model to obtain a predicted human body posture represented by the target human body posture sample output by the initial human body posture recognition model; and adjusting parameters of the human body posture recognition model based on the real human body posture and the predicted human body posture.
According to another aspect of the present disclosure, a human body posture recognition method is also provided. Fig. 4 shows a flowchart of a human gesture recognition method 400 according to an exemplary embodiment of the present disclosure, and as shown in fig. 4, the method 400 includes:
step S401, inputting the human body posture image to be detected into a human body posture recognition model to obtain the human body posture represented by the human body posture image to be detected output by the human body posture recognition model, wherein the human body posture recognition model is obtained by training using the method 300 described above. Therefore, the human body posture recognition effect can be improved by performing human body posture recognition by using the human body posture recognition model with optimized performance.
According to another aspect of the present disclosure, an excavating device for a target human body posture sample is also provided. Fig. 5 shows a block diagram of a mining apparatus 500 for a target human body posture sample according to an exemplary embodiment of the present disclosure, and as shown in fig. 5, the apparatus 500 includes:
a first obtaining unit 501, configured to obtain a plurality of body posture images, including a plurality of sample body posture images and at least one standard body posture image, each body posture image including a plurality of key points;
a first determining unit 502 configured to determine a pose feature vector representation of each human pose image based on a plurality of key points included in the each human pose image;
a clustering unit 503 configured to cluster the plurality of human posture images based on similarity between posture feature vector representations of the plurality of human posture images to obtain a plurality of cluster clusters;
a selecting unit 504 configured to select at least one cluster from the plurality of clusters based on a similarity criterion, wherein according to the similarity criterion, a body posture image in the at least one cluster is less similar to the at least one standard body posture image than body posture images in other clusters of the plurality of clusters;
a second determining unit 505 configured to determine each sample human posture image included in the at least one cluster as the target human posture sample.
According to some embodiments, the selection unit 504 comprises: a calculating subunit configured to calculate, for a cluster center of each of the plurality of cluster clusters, a similarity between a pose feature vector representation of a human pose image corresponding to the cluster center and a pose feature vector representation of the at least one standard human pose image; and the first determining subunit is configured to select a cluster corresponding to the cluster center in response to the similarity between the posture characteristic vector representation of the human posture image corresponding to the cluster center and the posture characteristic vector representation of the at least one standard human posture image being less than a preset threshold.
According to some embodiments, the first determining unit 202 comprises: the second determining subunit is configured to determine a plurality of bone lines corresponding to a plurality of key points included in the human body posture image; a third determining subunit, configured to determine an included angle between every two adjacent bone lines in the plurality of bone lines to obtain a plurality of bone line included angles; and a fourth determining subunit configured to determine a pose feature vector representation of the human pose image based on the plurality of bone line angles.
According to some embodiments, the at least one standard human pose image has the same size reference value, the size reference value being capable of indicating size information of a bone line of the human pose image, the third determining subunit includes: a scaling submodule configured to scale the plurality of bone lines based on the size reference value to obtain a plurality of normalized bone lines; and the determining submodule is configured to determine an included angle of the adjacent normalized bone lines for every two adjacent normalized bone lines in the plurality of normalized bone lines to obtain a plurality of bone line included angles.
According to some embodiments, clustering unit 505 is configured to cluster the plurality of human pose images using a K-means clustering algorithm based on similarities between pose feature vector representations of the plurality of human pose images.
According to another aspect of the present disclosure, a training device of a human body posture recognition model is also provided. Fig. 6 shows a block diagram of a training apparatus 600 for human body posture recognition model according to an exemplary embodiment of the present disclosure, and as shown in fig. 6, the apparatus 600 includes:
a second obtaining unit 601 configured to obtain an initial human body posture recognition model;
a training unit 602 configured to train the initial human body posture recognition model by using the target human body posture sample obtained by mining by the mining device 500 of the target human body posture sample described above.
According to another aspect of the present disclosure, a human body posture recognition device is also provided. Fig. 7 shows a block diagram of a human gesture recognition apparatus 700 according to an exemplary embodiment of the present disclosure, and as shown in fig. 7, the apparatus 700 includes:
a human body posture recognition model 701 obtained by training with the training device 600 for human body posture recognition models described above;
an input unit 702, configured to input a human body posture image to be detected into the human body posture recognition model, so as to obtain a human body posture represented by the human body posture image to be detected output by the human body posture recognition model.
According to another aspect of the present disclosure, there is also provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any of the methods described above.
According to another aspect of the present disclosure, there is also provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform any of the above methods.
According to another aspect of the present disclosure, there is also provided a computer program product comprising a computer program, wherein the computer program when executed by a processor implements any of the methods described above.
Referring to fig. 8, a block diagram of a structure of an electronic device 800, which may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. Electronic device is intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.
A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, an output unit 807, a storage unit 808, and a communication unit 809. The input unit 806 may be any type of device capable of inputting information to the device 800, and the input unit 806 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and/or a remote control. Output unit 807 can be any type of device capable of presenting information and can include, but is not limited to, a display, speakers, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 808 may include, but is not limited to, a magnetic disk, an optical disk. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth (TM) devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and/or the like.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be performed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical aspects of the present disclosure can be achieved.
Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the above-described methods, systems and apparatus are merely exemplary embodiments or examples and that the scope of the present invention is not limited by these embodiments or examples, but only by the claims as issued and their equivalents. Various elements in the embodiments or examples may be omitted or may be replaced with equivalents thereof. Further, the steps may be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced with equivalent elements that appear after the present disclosure.
Claims (17)
1. A mining method of a target human body posture sample comprises the following steps:
obtaining a plurality of human body posture images, wherein the plurality of human body posture images comprise a plurality of sample human body posture images and at least one standard human body posture image, and each human body posture image comprises a plurality of key points;
determining a pose feature vector representation of each human pose image based on a plurality of key points included in each human pose image;
clustering the plurality of human body posture images based on similarity between posture feature vector representations of the plurality of human body posture images to obtain a plurality of cluster clusters;
selecting at least one cluster from the plurality of clusters based on a similarity criterion, wherein according to the similarity criterion, a body pose image in the at least one cluster is less similar to the at least one standard body pose image than body pose images in other clusters of the plurality of clusters; and
and determining each sample human posture image included in the at least one cluster as the target human posture sample.
2. The method of claim 1, wherein the selecting at least one cluster from the plurality of cluster clusters based on a similarity criterion comprises:
calculating, for a cluster center of each of the plurality of cluster clusters, a similarity between a pose feature vector representation of a human pose image corresponding to the cluster center and a pose feature vector representation of the at least one standard human pose image; and
and selecting a cluster corresponding to the cluster center in response to the similarity between the posture characteristic vector representation of the human body posture image corresponding to the cluster center and the posture characteristic vector representation of the at least one standard human body posture image being smaller than a preset threshold value.
3. The method of claim 1 or 2, wherein the determining a pose feature vector representation of each human pose image based on the plurality of keypoints included in the each human pose image comprises:
determining a plurality of skeleton lines corresponding to a plurality of key points included in the human body posture image;
determining an included angle between every two adjacent bone lines in the plurality of bone lines to obtain a plurality of bone line included angles; and
and determining the posture characteristic vector representation of the human body posture image based on the plurality of bone line included angles.
4. The method of claim 3, wherein the at least one standard body pose image has a same size reference value indicating size information of bone lines of the body pose image, the determining an included angle between each two adjacent bone lines of the plurality of bone lines to obtain a plurality of bone line clip angles comprising:
scaling the plurality of bone lines based on the size reference value to obtain a plurality of normalized bone lines; and
determining an included angle between every two adjacent normalized bone lines in the plurality of normalized bone lines to obtain the plurality of bone line included angles.
5. The method of any one of claims 1-4, wherein the clustering the plurality of human pose images based on similarities between pose feature vector representations of the plurality of human pose images comprises:
and clustering the plurality of human posture images by using a K-means clustering algorithm based on posture characteristic vector representation of the plurality of human posture images.
6. A training method of a human body posture recognition model comprises the following steps:
acquiring an initial human body posture recognition model; and
training the initial body pose recognition model using a target body pose sample mined using the method of any one of claims 1-5.
7. A human body posture recognition method comprises the following steps:
inputting a human body posture image to be detected into a human body posture recognition model to obtain a human body posture represented by the human body posture image to be detected, wherein the human body posture recognition model is obtained by training according to the method of claim 6.
8. An excavation apparatus for a target body pose sample, comprising:
a first acquisition unit configured to acquire a plurality of human body posture images including a plurality of sample human body posture images and at least one standard human body posture image, each human body posture image including a plurality of key points;
a first determination unit configured to determine a pose feature vector representation of each human pose image based on a plurality of key points included in the each human pose image;
a clustering unit configured to cluster the plurality of human posture images based on similarity between posture feature vector representations of the plurality of human posture images to obtain a plurality of cluster clusters;
a selecting unit configured to select at least one cluster from the plurality of clusters based on a similarity criterion, wherein according to the similarity criterion, a human pose image in the at least one cluster is less similar to the at least one standard human pose image than human pose images in other clusters of the plurality of clusters; and
a second determining unit configured to determine each sample human posture image included in the at least one cluster as the target human posture sample.
9. The apparatus of claim 8, wherein the selection unit comprises:
a calculating subunit configured to calculate, for a cluster center of each of the plurality of cluster clusters, a similarity between a pose feature vector representation of a human pose image corresponding to the cluster center and a pose feature vector representation of the at least one standard human pose image; and
the first determining subunit is configured to select a cluster corresponding to the cluster center in response to the similarity between the posture characteristic vector representation of the human posture image corresponding to the cluster center and the posture characteristic vector representation of the at least one standard human posture image being less than a preset threshold.
10. The apparatus of claim 8 or 9, wherein the first determining unit comprises:
the second determining subunit is configured to determine a plurality of bone lines corresponding to a plurality of key points included in the human body posture image;
a third determining subunit, configured to determine an included angle between every two adjacent bone lines in the plurality of bone lines to obtain a plurality of bone line included angles; and
a fourth determining subunit configured to determine a pose feature vector representation of the human pose image based on the plurality of bone line angles.
11. The apparatus of claim 10, wherein the at least one standard human pose image has a same size reference value capable of indicating size information of a bone line of the human pose image, the third determining subunit comprises:
a scaling submodule configured to scale the plurality of bone lines based on the size reference value to obtain a plurality of normalized bone lines; and
a determination submodule configured to determine, for each two adjacent normalized bone lines of the plurality of normalized bone lines, an included angle of the adjacent normalized bone lines to obtain a plurality of bone line included angles.
12. The apparatus according to any one of claims 8-11, wherein the clustering unit is configured to cluster the plurality of human pose images using a K-means clustering algorithm based on similarities between pose feature vector representations of the plurality of human pose images.
13. A training device for a human body posture recognition model comprises:
a second obtaining unit configured to obtain an initial human body posture recognition model; and
a training unit configured to train the initial human body posture recognition model with a target human body posture sample mined with the apparatus of any one of claims 8-12.
14. A human body posture identifying apparatus comprising:
training the obtained human posture recognition model by using the device of claim 13; and
and the input unit is configured to input the human body posture image to be detected into the human body posture recognition model so as to obtain the human body posture represented by the human body posture image to be detected output by the human body posture recognition model.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.
16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to perform the method of any one of claims 1-7.
17. A computer program product comprising a computer program, wherein the computer program realizes the method according to any of claims 1-7 when executed by a processor.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210531488.5A CN114792445A (en) | 2022-05-16 | 2022-05-16 | Mining method and device for target human body posture sample, equipment and medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210531488.5A CN114792445A (en) | 2022-05-16 | 2022-05-16 | Mining method and device for target human body posture sample, equipment and medium |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114792445A true CN114792445A (en) | 2022-07-26 |
Family
ID=82463246
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210531488.5A Pending CN114792445A (en) | 2022-05-16 | 2022-05-16 | Mining method and device for target human body posture sample, equipment and medium |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114792445A (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116228867A (en) * | 2023-03-15 | 2023-06-06 | 北京百度网讯科技有限公司 | Pose determination method, pose determination device, electronic equipment and medium |
CN116881485A (en) * | 2023-06-19 | 2023-10-13 | 北京百度网讯科技有限公司 | Method and device for generating image retrieval index, electronic equipment and medium |
-
2022
- 2022-05-16 CN CN202210531488.5A patent/CN114792445A/en active Pending
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116228867A (en) * | 2023-03-15 | 2023-06-06 | 北京百度网讯科技有限公司 | Pose determination method, pose determination device, electronic equipment and medium |
CN116228867B (en) * | 2023-03-15 | 2024-04-05 | 北京百度网讯科技有限公司 | Pose determination method, pose determination device, electronic equipment and medium |
CN116881485A (en) * | 2023-06-19 | 2023-10-13 | 北京百度网讯科技有限公司 | Method and device for generating image retrieval index, electronic equipment and medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3913532A2 (en) | Object area measurement method, apparatus, storage medium and computer product | |
CN114792445A (en) | Mining method and device for target human body posture sample, equipment and medium | |
CN112749758A (en) | Image processing method, neural network training method, device, equipment and medium | |
CN114612749A (en) | Neural network model training method and device, electronic device and medium | |
CN113656668B (en) | Retrieval method, management method, device, equipment and medium of multi-modal information base | |
US20230052389A1 (en) | Human-object interaction detection | |
CN116311519B (en) | Action recognition method, model training method and device | |
CN116228867B (en) | Pose determination method, pose determination device, electronic equipment and medium | |
CN114723949A (en) | Three-dimensional scene segmentation method and method for training segmentation model | |
CN115511779A (en) | Image detection method, device, electronic equipment and storage medium | |
CN112488126A (en) | Feature map processing method, device, equipment and storage medium | |
CN117274370A (en) | Three-dimensional pose determining method, three-dimensional pose determining device, electronic equipment and medium | |
CN114120448B (en) | Image processing method and device | |
CN116152607A (en) | Target detection method, method and device for training target detection model | |
CN115761855A (en) | Face key point information generation, neural network training and three-dimensional face reconstruction method | |
CN114882587A (en) | Method, apparatus, electronic device, and medium for generating countermeasure sample | |
CN114693977A (en) | Image processing method, model training method, device, equipment and medium | |
CN114494797A (en) | Method and apparatus for training image detection model | |
CN114596476A (en) | Key point detection model training method, key point detection method and device | |
CN114092556A (en) | Method, apparatus, electronic device, medium for determining human body posture | |
CN112860681A (en) | Data cleaning method and device, computer equipment and medium | |
CN115578451B (en) | Image processing method, training method and device of image processing model | |
CN115345981B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
CN114117046B (en) | Data processing method, device, electronic equipment and medium | |
CN115512131B (en) | Image detection method and training method of image detection model |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |