CN107818318B - Humanoid robot simulation similarity evaluation method - Google Patents

Humanoid robot simulation similarity evaluation method Download PDF

Info

Publication number
CN107818318B
CN107818318B CN201711202768.7A CN201711202768A CN107818318B CN 107818318 B CN107818318 B CN 107818318B CN 201711202768 A CN201711202768 A CN 201711202768A CN 107818318 B CN107818318 B CN 107818318B
Authority
CN
China
Prior art keywords
connecting rod
joint
coordinate system
human
vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711202768.7A
Other languages
Chinese (zh)
Other versions
CN107818318A (en
Inventor
张智军
牛雅儒
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN201711202768.7A priority Critical patent/CN107818318B/en
Publication of CN107818318A publication Critical patent/CN107818318A/en
Application granted granted Critical
Publication of CN107818318B publication Critical patent/CN107818318B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J11/00—Manipulators not otherwise provided for
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/008—Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Mechanical Engineering (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Robotics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Manipulator (AREA)

Abstract

The invention discloses a humanoid robot simulation similarity evaluation method, which comprises the following steps: 1) acquiring three-dimensional position information of human skeleton nodes; 2) constructing a human skeleton vector, and establishing a human virtual joint according to the human skeleton vector and a robot joint structure to form a human skeleton model; 3) establishing a connecting rod reference coordinate system based on each connecting rod of the human skeleton model; 4) converting the connecting rod skeleton vector of the human skeleton model into a parent connecting rod reference coordinate system; 5) calculating the vector coordinates of each connecting rod of the robot in a parent connecting rod coordinate system; 6) and calculating the similarity of the human body posture and the human body posture according to the skeleton vector coordinates of the human body skeleton model connecting rod in the parent connecting rod coordinate system and the vector coordinates of the robot connecting rod in the parent connecting rod coordinate system. The method utilizes the link vector quantization of the link in the parent link coordinate system to evaluate the attitude similarity, and has the characteristics of full representation, local emphasis and accurate evaluation.

Description

Humanoid robot simulation similarity evaluation method
Technical Field
The invention relates to the technical field of human-computer interaction, in particular to a humanoid robot simulation similarity evaluation method.
Background
In recent years, robotics has been rapidly developed and is increasingly widely used in the fields of industry, medical care, scientific research, educational training, and household daily life. Meanwhile, more and more complex and various application environments put higher requirements on the adaptability and the intellectualization of the robot. The robot simulation learning technology can improve the robot learning efficiency, improve the robot intelligentization degree and release developers from heavy programming work. Therefore, the similarity between the robot simulation posture and the human body posture is properly evaluated, and the method plays an important role in optimizing the robot simulation system. Some current researches use a subjective evaluation method, which needs to issue a questionnaire to collect information, and has a tedious process and difficulty in obtaining accurate quantitative evaluation indexes. Other studies use quantitative evaluation methods to evaluate the similarity of robot simulations, which are mostly based on joint angles or body key nodes. And joint angles or body key nodes cannot directly represent the simulated or demonstrated postures, so that the similarity of the humanoid robot action simulation is difficult to effectively evaluate.
Disclosure of Invention
In order to solve the defects that the simulation or demonstration postures of the robot cannot be directly represented based on joint angles or body key nodes in the prior art and the simulation similarity of the actions of the humanoid robot is difficult to effectively evaluate, the invention provides a humanoid robot simulation similarity evaluation method for representing the postures of the robot and the human body by using connecting rod vectors.
The purpose of the invention can be realized by the following technical scheme:
a humanoid robot simulation similarity evaluation method comprises the following steps:
1. acquiring three-dimensional position information of a human skeleton node through a depth camera;
2. constructing a human skeleton vector according to the three-dimensional position information of the human skeleton node, and establishing a human virtual joint according to the human skeleton vector and a robot joint structure to form a human skeleton model;
3. establishing a connecting rod reference coordinate system based on each connecting rod of the human skeleton model;
4. transferring the connecting rod skeleton vector of the human skeleton model to a mother connecting rod reference coordinate system by using a conversion matrix;
5. calculating the vector coordinates of each connecting rod of the robot in a parent connecting rod coordinate system according to each joint angle and the corresponding rotation matrix of the robot;
6. and calculating the similarity of the human body posture and the human body posture according to the skeleton vector coordinates of the human body skeleton model connecting rod in the parent connecting rod coordinate system and the vector coordinates of the robot connecting rod in the parent connecting rod coordinate system.
Further, the human skeleton node refers to a joint point capable of rotating in a skeleton and a skeleton end node. The method specifically comprises the following steps: the spine comprises a spine bottom, a spine middle, a spine top, a neck, a head, a left shoulder, a left elbow, a left wrist, a left hand tip, a left thumb, a right shoulder, a right elbow, a right wrist, a right hand tip, a right thumb, a left hip, a left knee, a left ankle, a left foot, a right hip, a right knee, a right ankle, a right foot and a right ankle. Further, the robot joint structure refers to the number and position characteristics of joint rotating shafts capable of rotating in the humanoid robot structure, and the establishment of the human body virtual joints refers to the establishment of virtual joints corresponding to the number and types of the robot joints on the basis of human body skeleton nodes, wherein the virtual joints form joints in the human body skeleton model. Thereby facilitating the comparison of the similarity of subsequent postures.
Further, the human skeletal model is defined as follows: the trunk, the head, the left upper arm, the left lower arm, the left hand, the left thigh, the left calf, the left foot, the right upper arm, the right lower arm, the right hand, the right thigh, the right calf and the right foot are defined as connecting rods; defining a link near the torso as a parent link away from the torso link; defining links further from the torso as child links closer to the torso link; defining the neck, the left shoulder, the left elbow, the left wrist, the left hip, the left knee, the left ankle, the right shoulder, the right elbow, the right wrist, the right hip, the right knee and the right ankle as main joints; defining different degrees of freedom of a main joint of the humanoid robot as a secondary joint of the main joint; defining a primary joint close to the trunk as a primary joint of a primary joint far away from the trunk, and defining a secondary joint close to the trunk as a primary joint of a secondary joint far away from the trunk; defining a main joint far away from the trunk as a sub-joint close to the main joint of the trunk, and defining a sub-joint far away from the trunk as a sub-joint close to a sub-joint of the trunk, wherein two sub-joints which are a main joint and a sub-joint can belong to the same main joint or different main joints; the link which is close to the trunk in the two links connected with one joint is defined as a female link of the joint, and the link which is far away from the trunk is defined as a sub link of the joint; the main joint close to the trunk of the two main joints connected with one connecting rod is defined as the female joint of the connecting rod, and the main joint far away from the trunk is defined as the child joint of the connecting rod.
Further, the establishing of the human body virtual joint according to the human body bone vector and the robot joint structure to form the human body bone model specifically comprises: virtual joints with the same number and types as the humanoid robots are established in the human skeleton model. According to the joint characteristics of a common humanoid robot, in an initial posture, a secondary joint rotating shaft of a virtual joint is collinear or vertical to a skeleton vector of a mother connecting rod of the joint, and if two secondary joints belong to the same main joint, the two secondary joints are vertical to each other. The present invention considers only the case where there are one or two secondary joints of the primary joint.
Further, the virtual joint types include: rolling, pitching and yawing; the initial posture is defined as an upright posture with both arms down.
Further, the establishing of the reference coordinate system of the connecting rod based on each connecting rod of the human skeleton model specifically includes: for a connecting rod of which the sub joint is a main joint containing two sub joints, two axes of a reference coordinate system of the connecting rod are respectively collinear with the rotating axes of the two sub joints at the initial joint angle, and the other rotating axis is vertical to a plane formed by the two axes to form a right-hand rectangular coordinate system; in the initial posture, the X axis points to the front of the human skeleton model, the Y axis points to the left of the human skeleton model, and the Z axis points to the upper part of the human skeleton model. For a connecting rod of which the sub joint is a main joint containing a sub joint, one coordinate axis of a reference coordinate system of the connecting rod is collinear with a rotating axis of the sub joint, one coordinate axis is collinear or vertical with a skeleton vector of the connecting rod, and the other rotating axis is vertical to a plane formed by the two axes to form a right-hand rectangular coordinate system; in the initial posture, the X axis points to the right upper part of the human skeleton model, the Y axis points to the right left part of the human skeleton model, and the Z axis points to the right upper part of the human skeleton model. The coordinate origin of the connecting rod coordinate system is located at the tail end point of the connecting rod, namely the center point of the sub-joint of the connecting rod.
Further, the transferring the link skeleton vector of the human skeleton model into the parent link reference coordinate system specifically includes: using a transformation matrix RmConverting the link skeleton vector in the depth camera coordinate system into the link coordinate system of its parent link, converting the matrix RmComprises the following steps:
Figure GDA0002356386060000031
wherein
Figure GDA0002356386060000032
Respectively are vectors which are in the same direction with the positive directions of the X axis, the Y axis and the Z axis of the depth camera coordinate system in the depth camera coordinate system,
Figure GDA0002356386060000033
respectively are vectors which are in the same direction with the positive directions of an X axis, a Y axis and a Z axis of a connecting rod reference coordinate system of a target connecting rod female connecting rod in a depth camera coordinate system;
for the k-th human skeletal model link used for calculating pose similarity, the link skeletal vector in the link reference system of its parent link
Figure GDA0002356386060000034
(column vector) is:
Figure GDA0002356386060000035
wherein R ismkFor the purpose of the corresponding transformation matrix,
Figure GDA0002356386060000036
is the connecting rod skeleton vector in the depth camera coordinate system.
Further, the calculating the vector coordinates of each link of the robot in the parent link coordinate system according to the angle of each joint of the robot specifically includes: the vector coordinates of the initial posture connecting rods of each connecting rod of the robot in the coordinate system of the mother connecting rod are all (0,0,1)TAccording to the rotation angle theta of each secondary joint of the primary jointmkiUsing a corresponding rotation matrix R (theta)mki) Obtaining the connecting rod vector coordinates of the current attitude
Figure GDA0002356386060000041
Figure GDA0002356386060000042
Wherein n is the number of secondary joints of the target connecting rod female joint, and the smaller the numerical value of i is, the closer the secondary joint is to the female connecting rod in the connecting rod-joint chain.
Further, the calculating the similarity between the humanoid robot and the humanoid posture according to the skeleton vector coordinates of the human skeleton model connecting rod in the parent connecting rod coordinate system and the vector coordinates of the robot connecting rod in the parent connecting rod coordinate system specifically comprises: and calculating the correlation coefficient of the postures of the two connecting rods by utilizing the cosine value of the included angle between the skeleton vector of the human skeleton model connecting rod and the corresponding robot connecting rod vector according to the connecting rod vector after the coordinate system is converted, and then calculating the arithmetic mean value of the correlation coefficients of all groups of connecting rods to obtain the similarity index simulated by the robot action.
Compared with the prior art, the invention has the following beneficial effects:
1. a connecting rod reference coordinate system of each connecting rod of the human skeleton model is established, and the postures of the connecting rods of the human body can be accurately determined.
2. By using the quantitative evaluation method, the accurate quantitative index of the similarity between the robot posture and the human body posture can be obtained.
3. The gesture of the robot and the human body is more fully and effectively represented by using the connecting rod vector, and more accurate gesture similarity evaluation can be obtained.
4. The local attitude similarity can be effectively evaluated by utilizing the connecting rod vector of the connecting rod in the parent connecting rod coordinate system.
Drawings
FIG. 1 is a flow chart of a humanoid robot simulation similarity evaluation method of the present invention;
FIG. 2 is a schematic diagram of the names and numbers of human skeleton nodes obtained by Kinect II;
fig. 3 is a schematic diagram of a coordinate system of a whole body joint and a connecting rod of the Nao robot and a human skeleton model.
Detailed Description
The invention will be further described with reference to embodiments of the invention and the accompanying drawings, but the invention is not limited thereto:
examples
The invention discloses an evaluation method of anthropomorphic robot simulation similarity, which uses a Kinect II as a depth camera and uses a Nao robot as an anthropomorphic robot.
The flowchart of the steps implemented in the specific implementation of the humanoid robot simulation similarity evaluation method is shown in fig. 1, and specifically includes the following steps:
s1, acquiring three-dimensional position information of the human skeleton node through a depth camera;
three-dimensional information of human skeleton nodes is acquired by using a Kinect II camera, and the names and the numbers of the acquired skeleton nodes are shown in FIG. 2.
S2, constructing a human skeleton vector according to the human skeleton nodes, and establishing a human virtual joint according to the human skeleton vector and the robot joint structure to form a human skeleton model;
and establishing a human skeleton vector according to the three-dimensional position information of the human skeleton node. A bone vector quantization of one bone node to another bone node set in the depth camera coordinate system
Figure GDA00023563860600000513
Wherein a is the number of the bone node at the beginning of the vector, b is the number of the bone node at the end of the vector, and part of the bone vectors are shown in fig. 3.
According to the joint structure characteristics of the Nao robot, virtual joints with the number and the types consistent with those of the Nao robot joints are constructed in a human body skeleton model, and yaw pitching joints at the lower part of a trunk of the Nao robot are driven by a motor and are symmetrical left and right, so that the joint structure characteristics are not considered in the link. Defining the initial state of the robot and human skeleton model as the vertical posture with two arms drooping, defining the corresponding joint angle as the initial joint angle, wherein the shoulder pitch angle is
Figure GDA0002356386060000051
The remaining joint angles are all 0. The axes of rotation of the virtual joints in the initial pose are all collinear or perpendicular to the bone vector.
Taking the left half body part joint as an example, as shown in fig. 3, a human skeleton model virtual joint is constructed, and the rest joints have the same principle.
The left shoulder pitching joint is a female joint of the left shoulder rolling joint, and the rotating shaft and the vector of the left shoulder rolling joint
Figure GDA0002356386060000052
Collinear, the position of the rotating shaft of the left shoulder rolling joint changes along with the change of the left shoulder pitching joint, and the normal vector of the rotating shaft and the plane 2-5-6
Figure GDA0002356386060000053
Are collinear with each other and are arranged in a straight line,
Figure GDA0002356386060000054
comprises the following steps:
Figure GDA0002356386060000055
when the left shoulder pitch joint is at an initial angle, the normal vector of the rotating shaft of the left shoulder roll joint and the reference plane of the upper left trunk
Figure GDA0002356386060000056
Are collinear with each other and are arranged in a straight line,
Figure GDA0002356386060000057
comprises the following steps:
Figure GDA0002356386060000058
the left elbow yaw joint is the female joint of the left elbow roll joint, and the rotating shaft and the amount thereof
Figure GDA0002356386060000059
Co-linear. The position of the rotating shaft of the left elbow roll joint changes along with the change of the left elbow yaw joint, and the rotating shaft of the left elbow roll joint is a normal vector of the plane 5-6-7
Figure GDA00023563860600000510
Are collinear with each other and are arranged in a straight line,
Figure GDA00023563860600000511
comprises the following steps:
Figure GDA00023563860600000512
when the left elbow yaw joint is at an initial angle, the rotating shaft of the left elbow rolling joint is a normal vector of 2-5-6 of the plane
Figure GDA0002356386060000061
Co-linear.
The left hip rolling joint is a female joint of a left hip pitching joint, and the rotating shaft of the left hip rolling joint is a normal vector of a reference plane of a left lower trunk
Figure GDA0002356386060000062
Are collinear with each other and are arranged in a straight line,
Figure GDA0002356386060000063
comprises the following steps:
Figure GDA0002356386060000064
the position of the rotating shaft of the left hip pitch joint changes along with the change of the left hip roll joint, and the rotating shaft of the left hip pitch joint and the rotating shaft of the left hip roll joint
Figure GDA0002356386060000065
And
Figure GDA0002356386060000066
the normal vectors of the formed planes are collinear. When the left hip rolling joint is at an initial angle, the rotating shaft and the vector of the left hip pitching joint
Figure GDA0002356386060000067
Co-linear.
The left knee pitch joint is the only secondary joint of the left knee, the rotation axis of which is normal to the plane 17-18-19
Figure GDA0002356386060000068
Are collinear with each other and are arranged in a straight line,
Figure GDA0002356386060000069
comprises the following steps:
Figure GDA00023563860600000610
s3, establishing a connecting rod reference coordinate system based on each connecting rod of the human skeleton model;
taking the left half body part connecting rod as an example, as shown in fig. 3, a connecting rod reference system of the human skeleton model connecting rod is constructed, and the rest connecting rods have the same principle.
The X axis of the reference system of the upper left body connecting rod and the rotating shaft of the left shoulder rolling joint when the left shoulder pitching joint is at an initial angle are collinear and point to the right front of the connecting rod; the Y axis is collinear with the rotating shaft of the left shoulder pitching joint and points to the right left of the connecting rod; the Z axis is vertical to the X-Y plane and points to the right upper part of the connecting rod. With its origin of coordinates located at the bone node 5.
The X axis of the reference system of the left big arm connecting rod and the rotating shaft of the left elbow rolling joint when the left elbow yaw joint is at the initial angle are collinear and point to the right front of the connecting rod; the Z axis is collinear with the rotating shaft of the left elbow yaw joint and points to the right upper part of the connecting rod; the Y axis is perpendicular to the X-Z plane and points to the right and left of the connecting rod. With its origin of coordinates located at the bone node 6.
The X axis of the reference system of the left lower body connecting rod is collinear with the rotating shaft of the left hip rolling joint and points to the right front of the connecting rod; the Y axis is collinear with a rotating shaft of the left hip pitching joint when the left hip rolling joint is at an initial angle and points to the right left of the connecting rod; the Z axis is perpendicular to the X-Y plane and points to the right and left of the connecting rod. With its origin of coordinates located at the bone node 17.
The Y axis of the reference system of the left thigh connecting rod and the rotating shaft of the left knee pitching joint are collinear and point to the right left of the connecting rod; y axis and left thigh link skeletal vector
Figure GDA00023563860600000611
The collinear points to the right upper part of the connecting rod; the X axis is perpendicular to the Y-Z plane and points directly in front of the connecting rod. With its origin of coordinates located at the bone node 18.
The positions of the right front, right left and right upper parts of the connecting rod in the connecting rod are respectively kept consistent with the positions of the right front, right left and right upper parts of the connecting rod in the initial posture.
S4, converting the link skeleton vector of the human skeleton model into a parent link reference coordinate system;
in order to convert the link skeleton vectors in the depth camera coordinate system into the link coordinate system of its parent link, a conversion matrix R is constructedmComprises the following steps:
Figure GDA0002356386060000071
wherein
Figure GDA0002356386060000072
Respectively are vectors which are in the same direction with the positive directions of the X axis, the Y axis and the Z axis of the depth camera coordinate system in the depth camera coordinate system,
Figure GDA0002356386060000073
and vectors in the depth camera coordinate system in the same direction as the X-axis, Y-axis and Z-axis of the connecting rod reference coordinate system of the target connecting rod female connecting rod in the positive direction are respectively.
Then for the k-th manikin link used to compute pose similarity, the link skeleton vector in its parent link's link reference frame
Figure GDA0002356386060000074
(column vector) is:
Figure GDA0002356386060000075
wherein R ismkFor the purpose of the corresponding transformation matrix,
Figure GDA0002356386060000076
is the connecting rod skeleton vector in the depth camera coordinate system.
S5, calculating the vector coordinates of each connecting rod of the robot in the parent connecting rod coordinate system according to the angle of each joint of the robot;
initial attitude of k-th link for calculating attitude similarity of robot in parent link coordinate systemThe rod vector coordinates are
Figure GDA0002356386060000077
According to the rotation angle theta of each secondary joint of the primary jointmkiUsing a corresponding rotation matrix R (theta)mki) Obtaining the connecting rod vector coordinates of the current attitude
Figure GDA0002356386060000078
Figure GDA0002356386060000079
Wherein n is the number of secondary joints of the target connecting rod female joint, and the smaller the numerical value of i is, the closer the secondary joint is to the female connecting rod in the connecting rod-joint chain.
And S6, calculating the similarity between the human body and the human body posture according to the bone vector coordinates of the human body bone model connecting rod in the parent connecting rod coordinate system and the vector coordinates of the robot connecting rod coordinate system.
Further, according to the step S6, the correlation coefficient between the k link of the robot and the k link of the human skeleton model
Figure GDA00023563860600000710
Comprises the following steps:
Figure GDA0002356386060000081
in this embodiment, considering 10 connecting rods such as the trunk, the head, the left upper arm, the left lower arm, the left hand, the left thigh, the left calf, the right upper arm, the right lower arm, the right hand, the right thigh, the right calf, etc., the human-shaped robot with the local similarity weighted and the human posture similarity index are:
Figure GDA0002356386060000082
the above description is only for the preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto, and any person skilled in the art can substitute or change the inventive concept or technical solution of the present invention within the protection scope of the present invention.

Claims (9)

1. A humanoid robot simulation similarity evaluation method is characterized in that: the method comprises the following steps:
1) acquiring three-dimensional position information of a human skeleton node through a depth camera;
2) constructing a human skeleton vector according to the three-dimensional position information of the human skeleton node, and establishing a human virtual joint according to the human skeleton vector and a robot joint structure to form a human skeleton model;
3) establishing a connecting rod reference coordinate system based on each connecting rod of the human skeleton model;
4) converting the connecting rod skeleton model vector of the human skeleton model into a parent connecting rod reference coordinate system;
5) calculating the vector coordinates of each connecting rod of the robot in a parent connecting rod coordinate system according to the angle of each joint of the robot;
6) calculating the similarity of the human body posture and the human body posture according to the skeleton vector coordinates of the human body skeleton model connecting rod in the parent connecting rod coordinate system and the vector coordinates of the robot connecting rod in the parent connecting rod coordinate system;
in step 2), the human-shaped robot and the human skeleton model specifically include:
defining the trunk, the head, the left upper arm, the left lower arm, the left hand, the left thigh, the left calf, the left foot, the right upper arm, the right lower arm, the right hand, the right thigh, the right calf and the right foot of the humanoid robot and the human skeleton model as connecting rods; defining a link near the torso as a parent link to a link further from the torso; defining links further from the torso as child links of links closer to the torso; defining the neck, the left shoulder, the left elbow, the left wrist, the left hip, the left knee, the left ankle, the right shoulder, the right elbow, the right wrist, the right hip, the right knee and the right ankle as main joints; defining different degrees of freedom of a main joint of the humanoid robot as a secondary joint of the main joint; defining a primary joint close to the trunk as a primary joint of a primary joint far away from the trunk, and defining a secondary joint close to the trunk as a primary joint of a secondary joint far away from the trunk; defining a main joint far away from the trunk as a sub-joint close to the main joint of the trunk, and defining a sub-joint far away from the trunk as a sub-joint close to a sub-joint of the trunk, wherein two sub-joints which are a main joint and a sub-joint can belong to the same main joint or different main joints; defining a connecting rod which is close to a trunk in two connecting rods connected with a joint as a female connecting rod of the joint, and defining a connecting rod which is far away from the trunk as a child connecting rod of the joint; the main joint close to the trunk of the two main joints connected with one connecting rod is defined as the female joint of the connecting rod, and the main joint far away from the trunk is defined as the child joint of the connecting rod.
2. The humanoid robot simulation similarity evaluation method of claim 1, characterized in that: the human skeleton node refers to a joint point capable of rotating in a skeleton and a skeleton end node.
3. The humanoid robot simulation similarity evaluation method of claim 1, characterized in that: the method comprises the following steps of constructing a human skeleton vector according to three-dimensional position information of human skeleton nodes, establishing a human virtual joint according to the human skeleton vector and a robot joint structure, and forming a human skeleton model, and specifically comprises the following steps:
the method comprises the steps of establishing virtual joints with the same number and types as the joints of the humanoid robot in a human body skeleton model, setting the secondary joint rotating shaft of each virtual joint to be collinear or perpendicular to the skeleton vector of a mother connecting rod of the joint in an initial posture according to the joint characteristics of the humanoid robot, and setting the two secondary joints to be perpendicular to each other if the two secondary joints belong to the same main joint.
4. The humanoid robot simulation similarity evaluation method of claim 3, characterized in that: the virtual joint types include: rolling, pitching and yawing; the initial posture is defined as the vertical posture of two arms which are pendulous, and the corresponding joint angle is defined as the initial joint angle.
5. The humanoid robot simulation similarity evaluation method of claim 1, characterized in that: the distance between the connecting rod or the joint and the trunk is judged to be close to or far away from, and the judgment basis is as follows: the link or joint is measured from the torso in a link-joint chain, rather than in full space.
6. The humanoid robot simulation similarity evaluation method of claim 1, characterized in that: the establishing of the connecting rod reference coordinate system of each connecting rod of the human skeleton model specifically comprises the following steps:
for a connecting rod of which the sub-joint is a main joint containing two sub-joints, two axes of a reference coordinate system of the connecting rod are respectively collinear with the rotating axes of the two sub-joints at the initial joint angle, and the other rotating axis is vertical to a plane formed by the two axes to form a right-hand rectangular coordinate system;
for a connecting rod of which the sub joint is a main joint containing a sub joint, one coordinate axis of a reference coordinate system is collinear with a rotating axis of the sub joint, one coordinate axis is collinear or vertical with a skeleton vector of the connecting rod, and the other rotating axis is vertical to a plane formed by the two axes to form a right-hand rectangular coordinate system;
the coordinate origin of the connecting rod coordinate system is located at the tail end point of the connecting rod, namely the center point of the sub-joint of the connecting rod.
7. The humanoid robot simulation similarity evaluation method of claim 1, characterized in that: the method for converting the connecting rod skeleton vector of the human skeleton model into the parent connecting rod reference coordinate system specifically comprises the following steps:
using conversion momentsArray RmConverting the link skeleton vector in the depth camera coordinate system into the link coordinate system of its parent link, converting the matrix RmComprises the following steps:
Figure FDA0002356386050000031
wherein
Figure FDA0002356386050000032
Respectively are vectors which are in the same direction with the positive directions of the X axis, the Y axis and the Z axis of the depth camera coordinate system in the depth camera coordinate system,
Figure FDA0002356386050000033
respectively are vectors which are in the same direction with the positive directions of the X axis, the Y axis and the Z axis of the connecting rod reference system coordinate of the target connecting rod female connecting rod in the depth camera coordinate system;
then for the k-th manikin link used to compute pose similarity, the link skeleton vector in its parent link's link reference frame
Figure FDA0002356386050000034
Comprises the following steps:
Figure FDA0002356386050000035
wherein R ismkFor the purpose of the corresponding transformation matrix,
Figure FDA0002356386050000036
is the connecting rod skeleton vector in the depth camera coordinate system.
8. The humanoid robot simulation similarity evaluation method of claim 1, characterized in that: the calculating of the vector coordinates of each connecting rod of the robot in the parent connecting rod coordinate system according to the angle of each joint of the robot specifically comprises:
with each link of the robot in its parent link coordinate systemThe initial attitude tie-bar vector coordinates are all (0,0,1)TAccording to the rotation angle theta of each secondary joint of the primary jointmkiUsing a corresponding rotation matrix R (theta)mki) Obtaining the connecting rod vector coordinates of the current attitude
Figure FDA0002356386050000037
Figure FDA0002356386050000038
Wherein n is the number of secondary joints of the target connecting rod female joint, and the smaller the numerical value of i is, the closer the secondary joint is to the female connecting rod in the connecting rod-joint chain.
9. The humanoid robot simulation similarity evaluation method of claim 1, characterized in that: the method for calculating the similarity between the human-shaped robot and the human body posture according to the skeleton vector coordinates of the human body skeleton model connecting rod in the parent connecting rod coordinate system and the vector coordinates of the robot connecting rod in the parent connecting rod coordinate system specifically comprises the following steps:
and according to the connecting rod vectors after the coordinates are converted, calculating the correlation coefficients of the postures of the two connecting rods by utilizing the cosine values of the included angles between the human skeleton model connecting rod skeleton vectors and the corresponding robot connecting rod vectors, and further calculating the arithmetic mean of the correlation coefficients of all groups of connecting rods to obtain the similarity index simulated by the robot action.
CN201711202768.7A 2017-11-27 2017-11-27 Humanoid robot simulation similarity evaluation method Active CN107818318B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711202768.7A CN107818318B (en) 2017-11-27 2017-11-27 Humanoid robot simulation similarity evaluation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711202768.7A CN107818318B (en) 2017-11-27 2017-11-27 Humanoid robot simulation similarity evaluation method

Publications (2)

Publication Number Publication Date
CN107818318A CN107818318A (en) 2018-03-20
CN107818318B true CN107818318B (en) 2020-05-22

Family

ID=61610194

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711202768.7A Active CN107818318B (en) 2017-11-27 2017-11-27 Humanoid robot simulation similarity evaluation method

Country Status (1)

Country Link
CN (1) CN107818318B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108762495B (en) * 2018-05-18 2021-06-29 深圳大学 Virtual reality driving method based on arm motion capture and virtual reality system
CN109064486A (en) * 2018-06-20 2018-12-21 华南理工大学 A kind of anthropomorphic robot and human body attitude method for evaluating similarity
CN108961867A (en) * 2018-08-06 2018-12-07 南京南奕亭文化传媒有限公司 A kind of digital video interactive based on preschool education
CN109432753B (en) * 2018-09-26 2020-12-29 Oppo广东移动通信有限公司 Action correcting method, device, storage medium and electronic equipment
CN110135078B (en) * 2019-05-17 2023-03-14 浙江凌迪数字科技有限公司 Human body parameter automatic generation method based on machine learning
CN112634415B (en) * 2020-12-11 2023-11-10 北方信息控制研究院集团有限公司 Human body skeleton model-based personnel action real-time simulation method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6980889B2 (en) * 2003-10-08 2005-12-27 Sony Corporation Information processing apparatus and method, program storage medium, and program
CN1830635A (en) * 2005-03-10 2006-09-13 北京理工大学 Human imitation robot action similarity evaluation based on human body motion track
CN104816298A (en) * 2015-05-12 2015-08-05 北京理工大学 Similarity elevating method of humanoid robot tumble action
CN106971050A (en) * 2017-04-18 2017-07-21 华南理工大学 A kind of Darwin joint of robot Mapping Resolution methods based on Kinect

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6980889B2 (en) * 2003-10-08 2005-12-27 Sony Corporation Information processing apparatus and method, program storage medium, and program
CN1830635A (en) * 2005-03-10 2006-09-13 北京理工大学 Human imitation robot action similarity evaluation based on human body motion track
CN104816298A (en) * 2015-05-12 2015-08-05 北京理工大学 Similarity elevating method of humanoid robot tumble action
CN106971050A (en) * 2017-04-18 2017-07-21 华南理工大学 A kind of Darwin joint of robot Mapping Resolution methods based on Kinect

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"Joint movement similarities for robust 3D action recognition using skeletal data";Hossein Pazhoumand-Dar，Chiou-Peng Lam, Martin Masek;《J. Vis. Commun. Image R. 30 (2015)》;20151231;第10-21页 *
"Whole-body humanoid robot imitation with pose similarity evaluation";Jie Lei，MingliSong，Ze-NianLi，ChunChen;《Signal Processing108(2015)》;20151231;第134-146页 *

Also Published As

Publication number Publication date
CN107818318A (en) 2018-03-20

Similar Documents

Publication Publication Date Title
CN107818318B (en) Humanoid robot simulation similarity evaluation method
CN107953331B (en) human body posture mapping method applied to humanoid robot action simulation
CN107943283B (en) Mechanical arm pose control system based on gesture recognition
CN106078752A (en) Method is imitated in a kind of anthropomorphic robot human body behavior based on Kinect
CN113967911B (en) Follow control method and system of humanoid mechanical arm based on tail end working space
CN109079794B (en) Robot control and teaching method based on human body posture following
CN107856014B (en) Mechanical arm pose control method based on gesture recognition
CN108858188B (en) Human body turning and displacement mapping method applied to humanoid robot
CN109048897B (en) Method for teleoperation of master-slave robot
CN103085069A (en) Novel robot kinematics modeling method
CN103824326A (en) Dynamic human body three-dimensional modeling method
CN109064486A (en) A kind of anthropomorphic robot and human body attitude method for evaluating similarity
Yang et al. A study of the human-robot synchronous control system based on skeletal tracking technology
Teachasrisaksakul et al. Imitation of dynamic walking with BSN for humanoid robot
CN112650079B (en) Inverse kinematics solution method of seven-degree-of-freedom robot
CN112959330B (en) Robot double-arm motion man-machine corresponding device and method based on master-slave dynamic motion elements
Rosado et al. Reproduction of human arm movements using Kinect-based motion capture data
CN115781666A (en) Control method for robot whole body simulation system
Du et al. Data fusion of multiple kinect sensors for a rehabilitation system
Qian et al. DH Parameter Method-based Wearable Motion Tracking
Zhong et al. Inverse Kinematics Analysis of Humanoid Robot Arm by Fusing Denavit–Hartenberg and Screw Theory to Imitate Human Motion With Kinect
CN112276947A (en) Robot motion simulation method, device, equipment and storage medium
Tsai et al. Two-phase optimized inverse kinematics for motion replication of real human models
CN112634415B (en) Human body skeleton model-based personnel action real-time simulation method
Zhou et al. Development of a synchronized human-robot-virtuality interaction system using cooperative robot and motion capture device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant