CN112733761B - Human body state matching method based on machine learning - Google Patents

Human body state matching method based on machine learning Download PDF

Info

Publication number
CN112733761B
CN112733761B CN202110054577.0A CN202110054577A CN112733761B CN 112733761 B CN112733761 B CN 112733761B CN 202110054577 A CN202110054577 A CN 202110054577A CN 112733761 B CN112733761 B CN 112733761B
Authority
CN
China
Prior art keywords
shape
gesture
human body
pose
standard
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110054577.0A
Other languages
Chinese (zh)
Other versions
CN112733761A (en
Inventor
卢书芳
王宏升
高飞
丁雪峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University of Technology ZJUT
Original Assignee
Zhejiang University of Technology ZJUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University of Technology ZJUT filed Critical Zhejiang University of Technology ZJUT
Priority to CN202110054577.0A priority Critical patent/CN112733761B/en
Publication of CN112733761A publication Critical patent/CN112733761A/en
Application granted granted Critical
Publication of CN112733761B publication Critical patent/CN112733761B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • G06V40/23Recognition of whole body movements, e.g. for sport training
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103Static body considered as a whole, e.g. static pedestrian or occupant recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a human body state matching method based on machine learning, which comprises the following steps: (1) sample set pretreatment; (2) Open Pose detects human body key points; (3) training sample set alignment; (4) establishing a human body standard posture model; (5) PCA analysis dimension reduction; (6) Training SVM classifier, calculating deformation factor b of each item in data set C Calculating b C A threshold value of (2); (7) Calculating a deformation factor b of the human body posture shape to be verified C If b C The gesture shape to be verified is the predefined standard gesture if the value is within the threshold range, if b C If the value is beyond the threshold range, the gesture shape to be verified is not a standard gesture. By using the method and the device, whether the new gesture is the predefined standard gesture can be quickly determined, and the matching accuracy is high.

Description

Human body state matching method based on machine learning
Technical Field
The invention belongs to the technical field of human body posture matching, and particularly relates to a human body state matching method based on machine learning.
Background
Human body pose matching techniques, i.e., determining whether a new pose is a predefined standard pose. This technique is required for many applications, such as virtual fitting, somatosensory games, etc.
The prior method is based on a template matching method, the template matching method firstly needs to establish a template library, and then compares the similarity of each sample in the template library and a human body target to be identified.
The Chinese patent document with publication number CN112101243A discloses a human motion recognition method based on key gestures and DTW, which comprises the following steps: s10, acquiring joint coordinate data of a 3D skeleton of human motion by using a depth sensor, and describing a static posture by using a characteristic joint relative distance; s20, extracting key gesture frames from an original action gesture sequence by using an X-Means clustering algorithm based on time constraint to describe the key gesture frames; s30, establishing a standard action template library, calculating the similarity distance between a key gesture frame sequence forming the action to be recognized and the action key gesture frame sequence in the standard action template library based on a DTW algorithm, and dividing the action to be recognized into action types with the minimum similarity distance.
The chinese patent document with publication number CN110598556a discloses a human body shape and posture matching method and apparatus for matching human body shape and posture of RGBD images, comprising: extracting two-dimensional human mask and two-dimensional key point information of RGBD images through a convolutional neural network; extracting depth information of the RGBD image using a depth map of the RGBD image; fusing the two-dimensional human mask information and the depth information to obtain three-dimensional human mask information; fusing the two-dimensional key point information and the depth information to obtain three-dimensional key point information; and comparing the three-dimensional human mask information and the three-dimensional key point information with information in a standard library to obtain the matching degree of the human body shape and posture.
However, the template-based method is rough, each gesture in the template library is a rigid gesture, and when performing gesture matching, the user is required to be as the preset gesture, but the same gesture may also have great difference in space in consideration of the diversity of human gesture actions and the multi-scale structure of sample data. The accuracy of template-based methods is very limited.
Disclosure of Invention
The invention provides a human body state matching method based on machine learning, which can quickly determine whether a new gesture is a predefined standard gesture, and has high matching accuracy.
A human body state matching method based on machine learning comprises the following steps:
(1) Collecting front views of human bodies with a plurality of standard postures as positive samples, and establishing a standard posture sample set D; collecting the front view of the human body with nonstandard posture on the same scale as a negative sample; detecting and storing key point information of all positive samples and negative samples;
(2) Training the alignment of the standard posture sample set to obtain a pairA set of aligned pose shape data
(3) Computing aligned pose shape data setsMean gesture shape ∈>And +.>Is +.>And average gesture shape->The change between is expressed as->Calculating a covariance matrix S;
(4) Performing feature decomposition on the covariance matrix S by using principal component analysis PCA to find out principal components of the deformation of the attitude shape; the transformed pose shape is approximately expressed as:
wherein Q is C Representing the feature vector corresponding to the maximum feature value C, b C As a deformation factor, b C The smaller the size of the product,relative to->The smaller the deformation of (2);
(5) Preparation ofA training data set comprising positive and negative samples, an SVM classifier is trained, and a deformation factor b for each term in the data set is calculated C Marking qualified gesture as 1, marking nonstandard gesture as 0, and calculating b C A threshold value of (2);
(6) For the human body posture shape p to be verified new First, it is combined with the average pose shapeAlignment, the shape after alignment is expressed as +.>
(7) Calculating a deformation factor b of the human body posture shape to be verified C If b C The gesture shape to be verified is the predefined standard gesture if the value is within the threshold range, if b C If the value is beyond the threshold range, the gesture shape to be verified is not a standard gesture.
In the step (1), the detecting and saving the key point information of all positive samples and negative samples includes: firstly, compressing each picture to be below 20K, then using Open Pose to estimate key points of a human body, providing 25 body key points in total, and storing key point information of the corresponding picture in a json format with the same name after the key point estimation is completed.
The specific process of the step (2) is as follows:
(2-1) representing the standard pose sample set as follows:
D={p i |0≤i≤n}
where n is the total number of samples, p i Representing the i-th sample;
(2-2) randomly selecting an initialization gesture shape p m E D, dividing the sample set by p m Other posture shapes are sequentially connected with p m Alignment, finally obtaining an aligned gesture shape data set
In the step (2-2), p is removed from the sample set m Other posture shapes are sequentially connected with p m The specific process of alignment is as follows:
(2-2-1) dividing p in the Standard gesture sample set D m External p t Carrying out affine transformation, wherein t is more than or equal to 0 and less than or equal to n, and t is not equal to m, A is an affine transformation matrix,is the transformed pose shape, E represents the transformed shape +.>And a selected shape p m Is a distinction between (a);
(2-2-2) finding an affine transformation matrix a such that the difference E is sufficiently small;
(2-2-3) passing throughObtaining transformed shape->At this time, a->And p is as follows m Has been aligned.
In step (3), an aligned pose shape dataset is computedMean gesture shape ∈>The formula of (2) is:
in [ of ]] T Representing the transpose of the matrix,representation->X corresponding to all gesture shapes in the model k Average value->Representation +.>Y corresponding to all the gesture shapes in the model k Average value.
The specific process of the step (4) is as follows:
the process of principal component analysis PCA is represented as follows:
Sq k =λ k q k k=0,1,2,…,2n-1
wherein q k Is a feature vector lambda k Is a characteristic value; the new spatial basis is expressed as q= [ Q ] 0 q 1 … q 2n-1 ]Thus any transformed pose shapeExpressed as:
by lambda k Characteristic value pair q k Ordering to lambda 1 ≥λ 2 ≥λ 3 ≥…≥λ 2n Feature vector Q corresponding to maximum feature value C C The transformed pose shape is approximately expressed as:
wherein the average shapeAnd feature vector Q C Is a gesture shape model->Parameters b of (b) C Is a deformation factor.
Compared with the prior art, the invention has the following beneficial effects:
the invention establishes a model with higher latitude, has high matching accuracy, and simultaneously has simple algorithm realization, low requirement on machine operation and higher speed.
Drawings
FIG. 1 is a schematic flow chart of the method of the present invention;
fig. 2 is a schematic diagram of key points of a human body according to an embodiment of the present invention.
Detailed Description
The invention will be described in further detail with reference to the drawings and examples, it being noted that the examples described below are intended to facilitate the understanding of the invention and are not intended to limit the invention in any way.
As shown in fig. 1, a human body state matching method based on machine learning includes the following steps:
1) Preprocessing the sample human body posture picture.
(1.1) the invention collects the front view of the human body with a plurality of standard postures as a positive sample for establishing a standard posture sample set. The persons in these photographs have different body proportions but all exhibit a standard pose shape; at the same time, the same scale of non-standard human front view is also collected as a negative sample.
(1.2) to save computer resources, each picture is compressed below 20K.
(1.3) estimating key points of a human body using Open Pose, which is a powerful library for detecting key points of a human body in real time. It can provide 25 body keypoints, including the head and the foot. After the key point estimation is completed, the key point information of the corresponding picture is stored in a json format with the same name, and the key point of the gesture estimation is shown in fig. 2.
2) Training the alignment of the standard posture sample set.
(2.1) representing the human body standard posture sample set as follows:
D={p i |0≤i≤n}
where n is the total number of samples, p i Representing the i-th sample.
(2.2) aligning the two poses. Randomly selecting an initialization gesture shape p m E D, p t E D (m not equal to t) and p m Alignment.
(2.2.1) p pair t Performing affine transformation, wherein A is an affine transformation matrix,is the transformed pose shape, E represents the transformed shape +.>And a selected shape p m Is a distinction between (a) and (b).
(2.2.2) find the affine transformation matrix a so that the difference E is small enough.
(2.2.3) passing throughObtaining transformed shape->At this time, the person is strapped with (his/her own)>And p is as follows m Has been aligned.
(2.3) sequentially combining other gesture shapes in the sample set with the initialized gesture shape p in step (2.2) m Alignment, finally obtaining an aligned gesture shape data set
3) Calculating an average pose shape
4)、Is of the shape and the average shape>The variation between them is expressed asCalculating a covariance matrix S:
the matrix is used to representThe difference between the middle pose shape and the average shape.
5) And performing feature decomposition on S by using Principal Component Analysis (PCA) to find out principal components of the deformation of the posture shape. The process of PCA can be expressed as follows:
Sq k =λ k q k k=0,1,2,…,2n-1
wherein q k Is a feature vector lambda k Is a characteristic value. The new spatial basis is expressed as q= [ Q ] 0 q 1 … q 2n-1 ]Thus any transformed pose shapeCan be expressed as:
by lambda k Characteristic value pair q k Ordering to lambda 1 ≥λ 2 ≥λ 3 ≥…≥λ 2n The largest change in pose shape can be described by the first few feature vectors. In the method, only the Cmax characteristic value is considered, and the transformed gesture shape can be approximately expressed as:
wherein the average shapeAnd feature vector Q C Is a gesture shape model->Is a parameter of (a).
6) Verifying whether the new human body posture shape matches the standard posture shape. The specific method comprises the following steps:
(6.1) for a new shape pose p new First, it is combined with the average shapeAlignment. The aligned shape is denoted +.>
(6.2)Can be approximately formed by the average shape +.>And feature vector Q C The representation is:
adjusting the parameter positions:
b in the formula C As a deformation factor, b C The smaller the size of the product,relative to->The smaller the deformation of (c).
(6.3) training the SVM classifier to set b C The specific method is as follows:
a training dataset is prepared comprising two types of human body pose picture data, namely a standard pose map and a non-standard pose map. Calculating a deformation factor b for each item in the dataset C Marking qualified gesture as 1, marking nonstandard gesture as 0, and calculating b C Is set to a threshold value of (2).
(6.4) if b of new shape C The new pose shape is the predefined standard pose if the value is within the range, b of the new shape C The value is out of range and the new pose shape is not a standard pose.
The foregoing embodiments have described in detail the technical solution and the advantages of the present invention, it should be understood that the foregoing embodiments are merely illustrative of the present invention and are not intended to limit the invention, and any modifications, additions and equivalents made within the scope of the principles of the present invention should be included in the scope of the invention.

Claims (6)

1. The human body state matching method based on machine learning is characterized by comprising the following steps of:
(1) Collecting front views of human bodies with a plurality of standard postures as positive samples, and establishing a standard posture sample set D; collecting the front view of the human body with nonstandard posture on the same scale as a negative sample; detecting and storing key point information of all positive samples and negative samples;
(2) Training alignment of standard gesture sample sets to obtain aligned gesture shape data sets
(3) Computing aligned pose shape data setsMean gesture shape ∈>And +.>Each of the gesture shapesAnd average gesture shape->The change between is expressed as->Calculating a covariance matrix S;
(4) Performing feature decomposition on the covariance matrix S by using principal component analysis PCA to find out principal components of the deformation of the attitude shape; the transformed pose shape is approximately expressed as:
wherein Q is C Representing the feature vector corresponding to the maximum feature value C, b C As a deformation factor, b C The smaller the size of the product,relative to->The smaller the deformation of (2);
(5) Preparing a training data set containing positive and negative samples, training SVM classifier, and countingDeformation factor b for each item in the dataset C Marking qualified gesture as 1, marking nonstandard gesture as 0, and calculating b C A threshold value of (2);
(6) For the human body posture shape p to be verified new First, it is combined with the average pose shapeAlignment, the shape after alignment is expressed as +.>
(7) Calculating a deformation factor b of the human body posture shape to be verified C If b C The gesture shape to be verified is the predefined standard gesture if the value is within the threshold range, if b C If the value is beyond the threshold range, the gesture shape to be verified is not a standard gesture.
2. The machine learning based human body state matching method according to claim 1, wherein in the step (1), the detecting and saving key point information of all positive samples and negative samples comprises: firstly, compressing each picture to be below 20K, then using Open Pose to estimate key points of a human body, providing 25 body key points in total, and storing key point information of the corresponding picture in a json format with the same name after the key point estimation is completed.
3. The machine learning based human body state matching method of claim 1, wherein the specific process of step (2) is:
(2-1) representing the standard pose sample set as follows:
D={p i |0≤i≤n}
where n is the total number of samples, p i Representing the i-th sample;
(2-2) randomly selecting an initialization gesture shape p m E D, dividing the sample set by p m Other posture shapes are sequentially connected with p m Alignment, finally obtaining an aligned gesture shape data set
4. The machine learning based human body state matching method of claim 3, wherein in step (2-2), p is divided from the sample set m Other posture shapes are sequentially connected with p m The specific process of alignment is as follows:
(2-2-1) dividing p in the Standard gesture sample set D m External p t Carrying out affine transformation, wherein t is more than or equal to 0 and less than or equal to n, and t is not equal to m, A is an affine transformation matrix,is the transformed pose shape, E represents the transformed shape +.>And a selected shape p m Is a distinction between (a);
(2-2-2) finding an affine transformation matrix a such that the difference E is sufficiently small;
(2-2-3) passing throughObtaining transformed shape->At this time, a->And p is as follows m Has been aligned.
5. The machine learning based human state matching method of claim 3, wherein in step (3), an aligned pose shape dataset is calculatedMean gesture shape ∈>The formula of (2) is:
in [ of ]] T Representing the transpose of the matrix,representation->X corresponding to all gesture shapes in the model k Average value->Representation +.>Y corresponding to all the gesture shapes in the model k Average value.
6. The machine learning based human body state matching method of claim 5, wherein the specific process of step (4) is as follows:
the process of principal component analysis PCA is represented as follows:
Sq k =λ k q k k=0,1,2,...,2n-1
wherein q k Is a feature vector lambda k Is a characteristic value; the new spatial basis is expressed as q= [ Q ] 0 q 1 …q 2n-1 ]Thus any transformed pose shapeExpressed as:
by lambda k Characteristic value pair q k Ordering to lambda 1 ≥λ 2 ≥λ 3 ≥…≥λ 2n Feature vector Q corresponding to maximum feature value C C The transformed pose shape is approximately expressed as:
wherein the average shapeAnd feature vector Q C Is a gesture shape model->Parameters b of (b) C Is a deformation factor.
CN202110054577.0A 2021-01-15 2021-01-15 Human body state matching method based on machine learning Active CN112733761B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110054577.0A CN112733761B (en) 2021-01-15 2021-01-15 Human body state matching method based on machine learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110054577.0A CN112733761B (en) 2021-01-15 2021-01-15 Human body state matching method based on machine learning

Publications (2)

Publication Number Publication Date
CN112733761A CN112733761A (en) 2021-04-30
CN112733761B true CN112733761B (en) 2024-03-19

Family

ID=75591641

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110054577.0A Active CN112733761B (en) 2021-01-15 2021-01-15 Human body state matching method based on machine learning

Country Status (1)

Country Link
CN (1) CN112733761B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109035329A (en) * 2018-08-03 2018-12-18 厦门大学 Camera Attitude estimation optimization method based on depth characteristic
CN109949368A (en) * 2019-03-14 2019-06-28 郑州大学 A kind of human body three-dimensional Attitude estimation method based on image retrieval
CN110147767A (en) * 2019-05-22 2019-08-20 深圳市凌云视迅科技有限责任公司 Three-dimension gesture attitude prediction method based on two dimensional image
CN111598995A (en) * 2020-04-23 2020-08-28 浙江工商大学 Self-supervision multi-view three-dimensional human body posture estimation method based on prototype analysis

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109035329A (en) * 2018-08-03 2018-12-18 厦门大学 Camera Attitude estimation optimization method based on depth characteristic
CN109949368A (en) * 2019-03-14 2019-06-28 郑州大学 A kind of human body three-dimensional Attitude estimation method based on image retrieval
CN110147767A (en) * 2019-05-22 2019-08-20 深圳市凌云视迅科技有限责任公司 Three-dimension gesture attitude prediction method based on two dimensional image
CN111598995A (en) * 2020-04-23 2020-08-28 浙江工商大学 Self-supervision multi-view three-dimensional human body posture estimation method based on prototype analysis

Also Published As

Publication number Publication date
CN112733761A (en) 2021-04-30

Similar Documents

Publication Publication Date Title
CN106682598B (en) Multi-pose face feature point detection method based on cascade regression
CN109815826B (en) Method and device for generating face attribute model
US7876931B2 (en) Face recognition system and method
Alnajar et al. Calibration-free gaze estimation using human gaze patterns
Lee et al. Collaborative expression representation using peak expression and intra class variation face images for practical subject-independent emotion recognition in videos
CN108171133B (en) Dynamic gesture recognition method based on characteristic covariance matrix
CN108182397B (en) Multi-pose multi-scale human face verification method
CN110738161A (en) face image correction method based on improved generation type confrontation network
CN106570480A (en) Posture-recognition-based method for human movement classification
CN113570684A (en) Image processing method, image processing device, computer equipment and storage medium
CN111445426B (en) Target clothing image processing method based on generation of countermeasure network model
CN110135277B (en) Human behavior recognition method based on convolutional neural network
CN111815768B (en) Three-dimensional face reconstruction method and device
CN109858433B (en) Method and device for identifying two-dimensional face picture based on three-dimensional face model
CN111209811A (en) Method and system for detecting eyeball attention position in real time
CN111598995B (en) Prototype analysis-based self-supervision multi-view three-dimensional human body posture estimation method
CN108174141B (en) Video communication method and mobile device
CN111222469B (en) Coarse-to-fine human face posture quantitative estimation method
CN115205933A (en) Facial expression recognition method, device, equipment and readable storage medium
Jin et al. Emotion information visualization through learning of 3D morphable face model
CN116758621B (en) Self-attention mechanism-based face expression depth convolution identification method for shielding people
CN106778579B (en) Head posture estimation method based on accumulated attributes
CN112733761B (en) Human body state matching method based on machine learning
CN111275610A (en) Method and system for processing face aging image
CN108694348B (en) Tracking registration method and device based on natural features

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant