CN110688933A - Novel convolutional neural network and weighted assignment human body posture estimation algorithm - Google Patents

Novel convolutional neural network and weighted assignment human body posture estimation algorithm Download PDF

Info

Publication number
CN110688933A
CN110688933A CN201910897352.4A CN201910897352A CN110688933A CN 110688933 A CN110688933 A CN 110688933A CN 201910897352 A CN201910897352 A CN 201910897352A CN 110688933 A CN110688933 A CN 110688933A
Authority
CN
China
Prior art keywords
joint
neural network
convolutional neural
weighted
algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN201910897352.4A
Other languages
Chinese (zh)
Inventor
陈亮
余益军
黄帅
金尚忠
徐时清
张淑琴
杨凯
谷振寰
杨家军
祝晓明
徐瑞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Metrology
Original Assignee
China University of Metrology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Metrology filed Critical China University of Metrology
Priority to CN201910897352.4A priority Critical patent/CN110688933A/en
Publication of CN110688933A publication Critical patent/CN110688933A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a novel convolutional neural network and weighted assignment human posture estimation algorithm, which belongs to the field of human posture recognition. The invention provides a novel joint appearance model of a convolutional neural network and weighted support vector data description algorithm, which can effectively reflect the difference between different image areas and different abstract characteristics when image characteristics are extracted, and a human body posture estimation algorithm is designed by using the joint appearance model, so that the estimation accuracy is higher.

Description

Novel convolutional neural network and weighted assignment human body posture estimation algorithm
Technical Field
The invention belongs to the technical field of human body posture identification, and particularly relates to a novel convolutional neural network and weighted assignment human body posture estimation algorithm.
Background
At present, with the rapid development of human-computer interaction technology, human posture recognition technology is more and more emphasized. Gesture recognition, an important component of human behavior recognition, has recently become an important research hotspot in the field of computer vision. The main method of research is through analyzing the parameters of the whole or partial limbs of the input human, such as human body contour, joint point position, gesture limbs and the like. In the existing joint appearance model established based on the convolutional neural network, abstract features obtained by different regions and different convolutional kernels of a joint sample image block are treated equally, but actually, the roles played by different features are not completely the same. Therefore, in order to solve the two defects, the invention provides a novel convolutional neural network and a human body posture estimation algorithm of weighted assignment.
Disclosure of Invention
The invention aims to provide a novel convolutional neural network and a weighted assignment human posture estimation algorithm to solve the problems in the background technology.
In order to achieve the purpose, the invention provides the following technical scheme:
a novel convolutional neural network and weighted assignment human posture estimation algorithm comprises the following steps:
s1, carrying out convolution operation on the image area by utilizing a convolution neural network convolution algorithm:
the convolutional neural network comprises 3 convolutional layers, 4 pooling layers, 1 lead-in layer and 3 full-connection layers; the convolution operation comprises the following steps:
s101, extracting local image information in the first 3 pooling layers by adopting a maximum pooling method, and extracting global image information in the 4 th pooling layer by adopting an average pooling method;
s102, activating the first 2 full-connection layers by adopting a modified linear unit, and outputting a joint label by adopting a generalized linear regression function for the 3 rd full-connection layer;
and S2, giving a weight to the image area as a parameter of the convolutional neural network, and carrying out area positioning on the human shoulder joint by utilizing a joint positioning algorithm through constructing a joint appearance model.
As a preferred embodiment, in S102, the generalized linear regression function is:
Figure BDA0002210706930000021
wherein x is a parameter of the sample region, and f (x) is a joint label value output by the sample region.
As a preferred embodiment, in S2, the introducing the sample weight coefficients into the joint appearance model through a weighted assignment algorithm specifically includes the following steps:
constructing a sphere:
Figure BDA0002210706930000022
Figure BDA0002210706930000023
wherein, CjIs the coordinate of the sphere center of the sphere, xijiA penalty coefficient is introduced for the sample falling outside the sphere; radius of the sphere: rj＝||xjk-cj||；
sjiFor the sample weight coefficients:
Figure BDA0002210706930000024
wherein, D (x)ji) Is a sample xjiEuclidean distance to the center of the sample, DminAt a maximum distance, DminIs a minimum distance, DavrP is a positive integer greater than or equal to 2, and epsilon is a very small positive number;
by constructing the lagrange function:
Figure BDA0002210706930000025
wherein, ajiNot less than 0 and etajiLagrange multiplier is more than or equal to 0;
handle sjiThe optimization problem of the sample weight coefficient is converted into the following steps:
Figure BDA0002210706930000031
can be solved to obtain:
Figure BDA0002210706930000032
the sphere center of the sphere:
Figure BDA0002210706930000033
radius of the sphere: rj＝||xjk-cj||；
The joint appearance model is as follows:
Figure BDA0002210706930000034
where d is the sample xjkAnd (3) carrying out linear combination on all the sub-appearance models according to different weights to construct a joint appearance model according to the Euclidean distance from the sphere center of the sphere:
Figure BDA0002210706930000035
wherein, wjIs a weight coefficient, xjFor the j abstract feature, fj(xj) Is a joint appearance model.
As a preferred embodiment, in S2, the joint positioning algorithm specifically includes the following steps:
reducing joint search space;
calculating joint positioning probability;
determining the final positioning probability of the joint in the image to be processed for specific positioning:
Figure BDA0002210706930000036
in a preferred embodiment, the positioning joints include elbow joints, wrist joints and shoulder joints.
In the above scheme, it should be noted that the learning algorithm of the convolutional neural network according to the present invention adopts a random gradient descent algorithm:
Figure BDA0002210706930000041
wherein N is the number of samples, fi(W) is the convolutional neural network output, diFor the sample class label, the positive and negative samples are 1 and 0, respectively.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a novel joint appearance model of a convolutional neural network and weighted support vector data description algorithm, which can effectively reflect the difference between different image areas and different abstract characteristics when image characteristics are extracted, and a human body posture estimation algorithm is designed by using the joint appearance model, so that the estimation accuracy is higher.
Drawings
FIG. 1 is a schematic diagram of a human body posture estimation process according to the present invention;
FIG. 2 is a schematic diagram of a convolutional neural network structure according to the present invention;
FIG. 3 is a flow chart illustrating the convolution operation of the present invention;
FIG. 4 is a diagram illustrating a range of values of weights of convolution operations according to the present invention.
Detailed Description
The present invention will be further described with reference to the following examples.
The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention. The conditions in the embodiments can be further adjusted according to specific conditions, and simple modifications of the method of the present invention based on the concept of the present invention are within the scope of the claimed invention.
The invention provides a novel convolutional neural network and a weighted assignment human posture estimation algorithm, please refer to fig. 1, the method comprises estimating the human posture by using a joint appearance model of the convolutional neural network, the novel convolutional neural network and the weighted SVDD algorithm and a human posture estimation algorithm; the method specifically comprises the following steps:
s1, giving different weight coefficients to the convolution operation of different image areas to show different functions of the convolution operation;
s2, constructing a joint appearance model by adopting a weighted assignment algorithm, and carrying out linear combination according to different weights to establish a new joint appearance model;
wherein, the difference of the weight can embody different functions of the abstract characteristics.
Referring to fig. 2-4 and the following table, the structure of the convolutional neural network includes 3 convolutional layers, 4 pooling layers, 1 lead-in layer, and 3 fully-connected layers.
Wherein, the convolutional layer activation function adopts a modified linear unit (ReLU) with strong overfitting prevention capability. The invention defines new convolution operation, different weights are assigned to different regions of the image through the convolution operation, and the weights are used as parameters of a convolution neural network;
in order to better extract image local information, the first 3 pooling layers adopt a maximum pooling method, and the pooling layer 4 adopts an average pooling method to better extract image global information;
the front of the full-connection layer is a lead-in layer, the activation functions of the first two full-connection layers adopt modified linear units, and the full-connection layer 3 adopts a generalized linear regression function:
Figure BDA0002210706930000051
the output is a joint label;
in this embodiment:
the learning algorithm of the improved convolutional neural network adopts a random gradient descent algorithm:
Figure BDA0002210706930000052
wherein N is the number of samples, fi(W) is the convolutional neural network output, diFor the sample class label, the positive and negative samples are 1 and 0, respectively.
Figure BDA0002210706930000053
Figure BDA0002210706930000061
By introducing a sample weight coefficient into a weighting assignment algorithm, the method comprises the following specific steps:
constructing a sphere:
Figure BDA0002210706930000062
Figure BDA0002210706930000063
wherein, CjIs the coordinate of the sphere center of the sphere, xijiA penalty coefficient is introduced for the sample falling outside the sphere;
sjifor the sample weight coefficients:
Figure BDA0002210706930000064
wherein, D (x)ji) Is a sample xjiEuclidean distance to the center of the sample, DminAt a maximum distance, DminIs a minimum distance, DavrP is a positive integer greater than or equal to 2, and epsilon is a very small positive number;
by constructing the lagrange function:
Figure BDA0002210706930000071
wherein, ajiNot less than 0 and etajiLagrange multiplier is more than or equal to 0;
handle sjiThe optimization problem of the sample weight coefficient is converted into the following steps:
can be solved to obtain:
Figure BDA0002210706930000073
the sphere center of the sphere:
Figure BDA0002210706930000074
radius of the sphere: rj＝||xjk-cj||；
The sub-appearance model:
Figure BDA0002210706930000075
where d is the sample xjkAnd (3) carrying out linear combination on all the sub-appearance models according to different weights to construct a joint appearance model according to the Euclidean distance from the sphere center of the sphere:
Figure BDA0002210706930000076
wherein, wjIs a weight coefficient, xjAs the j abstractCharacteristic fj(xj) Is a joint appearance model.
The human body posture estimation algorithm specifically comprises the following steps:
reducing joint search space;
calculating joint positioning probability;
determining the final positioning probability of the specific positioning shoulder joint of the joint in the image to be processed:
Figure BDA0002210706930000077
the elbow and wrist joints are the same as the shoulder joint.
The invention provides a novel joint appearance model of a convolutional neural network and weighted support vector data description algorithm, which can effectively reflect the difference between different image areas and different abstract characteristics when image characteristics are extracted, and a human body posture estimation algorithm is designed by using the joint appearance model, so that the estimation accuracy is higher.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (5)

1. A novel convolutional neural network and weighted assignment human posture estimation algorithm is characterized by comprising the following steps:
s1, carrying out convolution operation on the image area by using a convolution neural network:
the convolutional neural network comprises 3 convolutional layers, 4 pooling layers, 1 lead-in layer and 3 full-connection layers; the convolution operation comprises the following steps:
s101, extracting local image information in the first 3 pooling layers by adopting a maximum pooling method, and extracting global image information in the 4 th pooling layer by adopting an average pooling method;
s102, activating the first 2 full-connection layers by adopting a modified linear unit, and outputting a joint label by adopting a generalized linear regression function for the 3 rd full-connection layer;
and S2, giving a weight to the image area, using the weight as a parameter of the convolutional neural network, and carrying out area positioning on the shoulder joint of the human body by constructing a joint appearance model and utilizing a joint positioning algorithm.
2. The novel convolutional neural network and weighted human pose estimation algorithm of claim 1, wherein in step S102, the generalized linear regression function is:
Figure FDA0002210706920000011
where x is the convolution parameter of the sample region, and f (x) is the joint label of the sample region.
3. The novel convolutional neural network and weighted-assignment human pose estimation algorithm as claimed in claim 1, wherein in S2, the joint appearance model introduces sample weight coefficients through the weighted-assignment algorithm, comprising the following steps:
(1) structural ball
(2) By constructing the Lagrangian function, sjiSimplifying the optimization problem of sample weight coefficients
(3) And carrying out linear combination on the joint appearance models according to different weights.
4. The novel convolutional neural network and weighted human pose estimation algorithm of claim 1, wherein in S2, the joint localization algorithm specifically comprises the following steps:
(1) reducing joint search space;
(2) calculating joint positioning probability;
(3) and determining the final positioning probability of the specific positioning joint of the joint in the image to be processed.
5. The novel convolutional neural network and weighted human pose estimation algorithm of claim 4, wherein said positioning joints comprise elbow joint, wrist joint and shoulder joint.
CN201910897352.4A 2019-09-23 2019-09-23 Novel convolutional neural network and weighted assignment human body posture estimation algorithm Withdrawn CN110688933A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910897352.4A CN110688933A (en) 2019-09-23 2019-09-23 Novel convolutional neural network and weighted assignment human body posture estimation algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910897352.4A CN110688933A (en) 2019-09-23 2019-09-23 Novel convolutional neural network and weighted assignment human body posture estimation algorithm

Publications (1)

Publication Number Publication Date
CN110688933A true CN110688933A (en) 2020-01-14

Family

ID=69109896

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910897352.4A Withdrawn CN110688933A (en) 2019-09-23 2019-09-23 Novel convolutional neural network and weighted assignment human body posture estimation algorithm

Country Status (1)

Country Link
CN (1) CN110688933A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113219926A (en) * 2021-05-13 2021-08-06 中国计量大学 Human-machine co-fusion manufacturing unit safety risk assessment method based on digital twin system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108647663A (en) * 2018-05-17 2018-10-12 西安电子科技大学 Estimation method of human posture based on deep learning and multi-level graph structure model
CN110222634A (en) * 2019-06-04 2019-09-10 河海大学常州校区 A kind of human posture recognition method based on convolutional neural networks

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108647663A (en) * 2018-05-17 2018-10-12 西安电子科技大学 Estimation method of human posture based on deep learning and multi-level graph structure model
CN110222634A (en) * 2019-06-04 2019-09-10 河海大学常州校区 A kind of human posture recognition method based on convolutional neural networks

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
韩贵金: "基于改进CNN和加权SVDD算法的人体姿态估计", 《计算机工程与应用》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113219926A (en) * 2021-05-13 2021-08-06 中国计量大学 Human-machine co-fusion manufacturing unit safety risk assessment method based on digital twin system

Similar Documents

Publication Publication Date Title
CN110009052B (en) Image recognition method, image recognition model training method and device
CN108986801A (en) A kind of man-machine interaction method, device and human-computer interaction terminal
CN108875836B (en) Simple-complex activity collaborative recognition method based on deep multitask learning
CN110458235B (en) Motion posture similarity comparison method in video
Santhalingam et al. Sign language recognition analysis using multimodal data
CN111680550B (en) Emotion information identification method and device, storage medium and computer equipment
CN110046544A (en) Digital gesture identification method based on convolutional neural networks
CN110751039A (en) Multi-view 3D human body posture estimation method and related device
CN110968235B (en) Signal processing device and related product
CN111178288A (en) Human body posture recognition method and device based on local error layer-by-layer training
CN113901891A (en) Parkinson's disease fist making task evaluation method and system, storage medium and terminal
CN111108508A (en) Facial emotion recognition method, intelligent device and computer-readable storage medium
CN114882305A (en) Image key point detection method, computing device and computer-readable storage medium
CN116469175B (en) Visual interaction method and system for infant education
CN110688933A (en) Novel convolutional neural network and weighted assignment human body posture estimation algorithm
CN111695507B (en) Static gesture recognition method based on improved VGGNet network and PCA
Dong et al. An improved deep neural network method for an athlete's human motion posture recognition
CN116503654A (en) Multimode feature fusion method for carrying out character interaction detection based on bipartite graph structure
Rubin Bose et al. In-situ identification and recognition of multi-hand gestures using optimized deep residual network
Tang An action recognition method for volleyball players using deep learning
CN110309816B (en) Method and system for detecting face of newborn from rough to fine
Karthik et al. Survey on Gestures Translation System for Hearing Impaired People in Emergency Situation using Deep Learning Approach
CN114821632A (en) Method for re-identifying blocked pedestrians
CN110826726B (en) Target processing method, target processing device, target processing apparatus, and medium
Li et al. Research on continuous dynamic gesture recognition of chinese sign language based on multi-mode fusion

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication
WW01 Invention patent application withdrawn after publication

Application publication date: 20200114