WO2019218457A1 - Virtual reality driving method based on arm motion capture, and virtual reality system - Google Patents

Abstract

A virtual reality driving method based on arm motion capture, and a virtual reality system. The method comprises: when a motion capture system is worn on the human body, initializing a pre-set posture to obtain initial position and posture data; capturing real-time posture data of the human body, and determining a first arm posture by means of a transformation matrix between arm connecting rod members according to the real-time posture data and the initial position and posture data; and converting, according to a pre-set built-in model, the first arm posture into a second arm posture of a pre-set virtual character, and driving the pre-set virtual character according to the second arm posture (S30). According to acquired initial position and posture data and real-time posture data, an arm kinematic chain structure is used to determine arm posture data by means of a transformation matrix between connecting rods, thereby improving the accuracy of arm motion recognition. Moreover, the arm posture data is converted based on a built-in model to drive a 3D virtual character to move, thereby ensuring that the spatial positions of the virtual character and the arm remain consistent with the spatial position of a real character.

Description

Virtual reality driving method based on arm motion capture and virtual reality system Technical field

The present invention relates to the field of intelligent terminal technologies, and in particular, to a virtual reality driving method based on arm motion capture and a virtual reality system.

Background technique

Virtual reality (VR) is a new technology that integrates real world information and virtual world information "seamlessly". It combines real and illusory that could not be experienced in the real world through cutting-edge technologies such as computers. After the simulation and superimposition, the unreal characters or objects are superimposed on the real world, and are perceived by the human visual senses, thereby achieving an experience beyond reality. This allows real-world and illusory objects to be superimposed in the same space in real time. The existing virtual implementation is generally based on a motion capture system to identify human motion, and to control the virtual reality character according to the human body motion, in particular, to control the character through the human arm motion. For example, human body arm motion is recognized based on inertial sensors and based on computer vision. However, the effect of capturing the arm in the above manner is not good. For example, the computer vision-based method is easily interfered by the external environment, such as lighting conditions, background, and obstructions. The inertial sensor-based method is used to measure noise and travel. The influence of factors such as errors cannot be accurately tracked for a long time.

Summary of the invention

The present invention aims to provide a virtual reality driving method based on arm motion capture and a virtual reality system.

In order to solve the above technical problem, the technical solution adopted by the present invention is as follows:

A virtual reality driving method based on arm motion capture, comprising:

When the human body wears the motion capture system, initializing the preset posture to obtain initial pose data, wherein the initial pose data includes preset pose data and human body raw data;

Capturing the real-time posture data of the human body, determining the first arm posture according to the real-time attitude data and the initial pose data by using an arm-to-link transformation matrix method, wherein the first arm posture includes a trunk joint and an arm motion chain joint;

Converting the first arm posture into a second arm posture of a preset virtual character according to a preset built-in model, and driving the preset virtual character according to the second arm posture.

The virtual reality driving method based on arm motion capture, wherein the motion capture system includes at least a head display, a left and right handle, a left and right upper arm tracker, and a torso tracker.

The virtual reality driving method based on the arm motion capture, wherein when the human body wears the motion capture system, initializing the preset posture to obtain the initial pose data specifically includes:

When the human body wears the motion capture system, capturing preset posture data of the human body in a preset posture, wherein the preset posture includes a first posture and a second posture;

Correcting a preset skeleton model according to the preset pose data corresponding to the first posture;

Calculating initial body data based on the preset pose data to obtain initial pose data.

The virtual reality driving method based on the arm motion capture, wherein the calculating the initial data of the human body according to the preset pose data to obtain the initial pose data specifically includes:

Calculating a relative positional relationship of each joint of the upper body according to the preset pose data corresponding to the first posture;

Comparing the preset pose data corresponding to the second posture with the preset pose data corresponding to the first posture to calculate human body raw data to obtain initial pose data.

The virtual reality driving method based on the arm motion capture, wherein the capturing the real-time attitude data of the human body, determining the first arm posture by the arm-to-part transformation matrix method according to the real-time attitude data and the initial pose data specifically includes:

Capturing the real-time posture data of the human body, calculating the real-time data of the trunk joint according to the preset torso posture formula, and calculating the real-time position data of the upper arm according to the preset upper arm posture formula;

Determining the shoulder joint position according to the initial pose data, and calculating real-time data of the elbow joint according to the shoulder joint data and the shoulder joint change matrix, wherein the elbow joint data is offset from the upper arm length in the X-axis direction of the coordinate system of the shoulder joint ;

Calculating an angle between the upper arm and the forearm according to the shoulder joint data and the elbow joint data, and calculating forearm pose data according to the included angle to obtain a first arm posture.

The virtual reality driving method based on arm motion capture, wherein the angle between the upper arm and the forearm is calculated according to the shoulder joint data and the elbow joint data, and the forearm pose data is calculated according to the included angle to obtain the first The arm posture specifically includes:

Determining the first unit vector of the elbow joint to the shoulder joint based on the shoulder joint data and the real-time data of the elbow joint, and determining the second unit vector of the elbow joint pointing to the wrist joint according to the real data of the elbow joint and the real-time data of the wrist joint;

Calculating an angle between the upper arm and the forearm by using a cosine theorem according to the first unit vector and the second unit vector, and calculating forearm pose data according to the included angle to obtain a first arm posture.

The virtual reality driving method based on the arm motion capture, wherein when the human body wears the motion capture system, initializing the preset posture to obtain the initial pose data includes:

The preset skeleton model is received and stored, and each joint coordinate system of the preset skeleton model is associated with the preset built-in model to obtain a correspondence between the preset skeleton model and the preset built-in model.

The virtual reality driving method based on arm motion capture, wherein the first arm posture is converted into a second arm posture of a preset virtual character according to a preset built-in model, and the pre-drive is driven according to the second arm posture The virtual role specifically includes:

Redirecting the first arm posture to a coordinate system of each joint point of the preset built-in model;

Converting the first arm posture into each joint point coordinate system of the preset skeleton model according to the correspondence relationship to obtain a second arm posture;

Determining a preset virtual character corresponding to the preset skeleton model according to the second arm posture.

A computer readable storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement an arm based as described above The steps in the virtual reality driving method of motion capture.

A virtual reality system includes: a motion capture system and a virtual reality device, the virtual reality device including a processor, a memory, and a communication bus; and the memory stores a computer readable program executable by the processor;

The communication bus implements connection communication between the processor and the memory;

The step in the virtual reality driving method based on the arm motion capture of any of the above is implemented when the processor executes the computer readable program.

Advantageous Effects: Compared with the prior art, the present invention provides a virtual reality driving method based on arm motion capture and a virtual reality system, the method comprising: initializing a preset posture to obtain an initial position when the human body wears the motion capturing system Attitude data; capturing real-time posture data of the human body, determining a first arm posture by an arm-to-part transformation matrix method according to real-time attitude data and initial pose data, wherein the first arm posture includes a trunk joint and an arm motion chain joint Converting the first arm posture into a second arm posture of a preset virtual character according to a preset built-in model, and driving the preset virtual character according to the second arm posture. According to the initial pose data and the real-time posture data, the present invention uses the arm kinematic chain structure to determine the arm posture data in the form of a transformation matrix between the links, thereby improving the accuracy of the arm motion recognition. At the same time, based on the built-in model, the arm posture data is converted to drive the 3D virtual character movement, which ensures that the virtual character and the arm space position are consistent with the spatial position of the real character.

DRAWINGS

FIG. 1 is a flowchart of an embodiment of a virtual reality driving method based on arm motion capture provided by the present invention.

2 is a schematic diagram of a wear motion capture system in an embodiment of a virtual reality driving method based on arm motion capture provided by the present invention.

FIG. 3 is a schematic diagram of a first posture in an embodiment of a virtual reality driving method based on arm motion capture according to the present invention.

FIG. 4 is a schematic diagram of a second posture in an embodiment of a virtual reality driving method based on arm motion capture according to the present invention.

FIG. 5 is a schematic structural diagram of a virtual reality device according to an embodiment of a virtual reality system according to the present invention.

Detailed ways

The present invention provides a virtual reality driving method and a virtual reality system based on arm motion capture. The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The singular forms "a", "an", "the" It is to be understood that the phrase "comprise" or "an" Integers, steps, operations, components, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element. Further, "connected" or "coupled" as used herein may include either a wireless connection or a wireless coupling. The phrase "and/or" used herein includes all or any one and all combinations of one or more of the associated listed.

Those skilled in the art will appreciate that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs, unless otherwise defined. It should also be understood that terms such as those defined in a general dictionary should be understood to have meaning consistent with the meaning in the context of the prior art, and will not be idealized or excessive unless specifically defined as here. The formal meaning is explained.

The contents of the invention will be further described by the following description of embodiments with reference to the accompanying drawings.

Please refer to FIG. 1. FIG. 1 is a flowchart of a preferred embodiment of a self-starting control method provided by the present invention. The method includes:

S10. When the human body wears the motion capture system, the preset posture is initialized to obtain initial pose data, wherein the initial pose data includes preset pose data and human body raw data.

Specifically, the motion capture system is configured to capture human body motion, and at least includes a head display, a left and right handle, a left and right upper arm tracker, and a torso tracker. As shown in FIG. 2, the head is used for wearing on a human head, the left and right handles are respectively held on the left and right hands of the human body, the left upper arm tracker is used to be worn on the left upper arm position, and the right upper arm tracker is used for wearing. In the right upper arm position, the torso tracker is used to be worn in the chest position. The head is used to collect head posture data, the left and right handles are used to collect wrist joint posture data, and the left and right upper arm trackers are used to adopt shoulder joint posture data.

In addition, the preset posture includes a first posture and a second posture. After the human body is equipped with a motion capture system, the human body respectively positions the first posture and the second posture, and the motion capture system respectively captures the human body. The first posture data of the first posture and the posture data of the second posture, and the initial pose data is acquired according to the first posture data and the second posture time. Correspondingly, when the human body wears the motion capture system, initializing the preset posture to obtain the initial pose data specifically includes:

S11, when the human body wears the motion capture system, capturing preset posture data of the human body in a preset posture, wherein the preset posture includes a first posture and a second posture;

S12. Correct the preset skeleton model according to the preset pose data corresponding to the first posture.

S13. Calculate initial body data according to the preset pose data to obtain initial pose data.

Specifically, the first posture is preferably an "I" type posture, and the second posture is preferably a "T" type posture. As shown in FIG. 3 and FIG. 4, the "T"-shaped posture is that the body stands, and the two arms are stretched out and left; the "I"-shaped posture is that the body stands and the two arms naturally sag. The first posture and the second posture may be performed by the user according to the prompt, and the preset posture is initialized by touching the handle after the placement is completed. In addition, when the human body is in the "I"-type posture, the first head position data of the head is collected by the head display, and the first left and right arm pose data is collected by the left and right arm trackers; when the human body is in the "T" posture, the passage is passed. The second head position data of the head is collected by the head, and the second left and right arm position data is collected by the left and right arm trackers, wherein the pose data includes position data and posture data.

Further, the skeleton model of the virtual character in the virtual space is pre-stored, and is recorded as a preset skeleton model, and the coordinate system corresponding to the head position of the preset skeleton model is set as a root coordinate system, and each joint configuration The corresponding local coordinate system sets a virtual character relative to the root coordinate system. The position of the head can be collected by the head, and the relative postures of the thoracic joint and the left and right clavicle joints remain unchanged during the movement, that is, the position of the thoracic joint and the left and right clavicle joints can be obtained by the torso tracker; The left and right upper arm trackers are obtained; the relative postures of the left and right palms and the handle are always unchanged during the movement, and the posture of the palm can be calculated by the position of the handle. That is to say, when the human body is in the "I" posture, the posture data of the torso tracker and the posture data of the left and right upper arm trackers can be collected, wherein the posture data of the torso tracker is recorded as

The attitude data of the left upper arm tracker is recorded as

The attitude data of the right upper arm tracker is recorded as

And each posture data is represented by a quaternion. After obtaining the "I" type posture data, the preset skeleton model is corrected by using the "I" type posture data.

In addition, after acquiring the first posture data and the second posture data, the initial body data of the human body may be calculated according to the first lion head data and the second posture data. Correspondingly, the calculating the initial data of the human body according to the preset pose data to obtain the initial pose data specifically includes:

S131. Calculate, according to the preset pose data corresponding to the first posture, a relative positional relationship of the joints of the upper body;

S132. Compare the preset pose data corresponding to the second posture with the preset pose data corresponding to the first posture to calculate human body raw data to obtain initial pose data.

Specifically, the obtaining the first head posture data and the first left and right arm position data according to the “I” type posture calculates the distance between the two shoulders (ie, the body width) and the center point of the two shoulders (ie, the chest cavity) The position of the joint), and then calculate the vector of the thoracic joint point to the head (ie, the position of the thoracic joint point to the head position). The arm deployment distance can be calculated by calculating the second head posture data and the second left and right arm position data according to the "T" posture, and calculating the arm length according to the arm deployment distance and the body width, wherein the arm length = ( Arm extension distance - body width) / 2. After obtaining the length of the arm, the length of the upper arm and the forearm can be calculated according to the national standard (GB/T1000-1988) "Chinese adult human body size". Finally, the height is calculated according to the average value of the head-display Z-axis height of the "I"-type attitude and the "T"-type posture, for example, the height is equal to the average value + an offset, wherein the offset is preset. , which can be obtained from a large amount of experimental data. After obtaining the height, the length of the spine, the length of the neck, the length of the leg, the length of the thigh, and the length of the calf can be calculated according to the proportion of "Chinese adult human body size", thereby obtaining the raw data of the human body.

S20. Capture real-time attitude data of the human body, and determine a first arm posture according to the real-time attitude data and the initial pose data by a method of transforming the matrix between the arm and the link, wherein the first arm posture includes a trunk joint and an arm motion chain joint.

Specifically, the motion capture system captures the posture data of the human body in real time, and the posture data can be collected by the head display, the left and right handles, the left and right upper arm trackers, and the torso tracker. In other words, the head data, the left and right handles, the left and right upper arm trackers, and the torso tracker capture the posture data of the human head, the left and right palms, the left and right upper arms, and the torso in real time. After the real-time attitude data is acquired, the position of each joint of the human body can be calculated according to the initial pose data and the real-time posture data, wherein the position is represented by a quaternion form. That is, the coordinates of the human torso and the joints of the arms are updated based on the initial pose data and the real-time posture data. Correspondingly, the capturing the real-time attitude data of the human body, determining the first arm posture by the arm-to-part transformation matrix method according to the real-time attitude data and the initial pose data specifically includes:

S21: capturing real-time posture data of the human body, calculating real-time data of the trunk joint according to the preset torso posture formula, and calculating real-time position data of the upper arm according to the preset upper arm posture formula.

Specifically, the arm motion is described by a rigid body posture (rotation), and a quaternion method is employed. The quaternion is a method used in graphics as a rotation transform operation, which can perform multiplication, inversion, conjugate quaternion, and rotation interpolation. Wherein, the form of the quaternion can be:

(q _x q _y q _z q _w );

It can also be replaced by the following form:

q=p _v +q _w =q _x i+q _y j+q _z k+q _w ;

Where q represents a quaternion, p _v is an imaginary part, represents a vector (q _x q _y q _z ), and q _w is a real part.

Rotating the θ angle around the unit vector (x y z) with the quaternion q can be expressed as:

Further, the rigid body transformation may be considered by integrating the position information and the attitude information of the rigid body, that is, using a transformation matrix. And usually a 4×4 homogeneous transform matrix is used to represent the transform matrix:

among them,

A transformation matrix representing the spatial description of rigid body B in the A coordinate system (for example,

Representing the transformation matrix of the shoulder in the world coordinate system, (p _x p _y p _z ) represents the position information of the rigid body,

Indicates the posture information of the rigid body.

In addition, the transformation matrix can also represent the local coordinate system of the rigid body, such as

In the expression, (p _x p _y p _z ) represents the vector of the rigid body relative to the origin of the world coordinate system.

Each line represents the representation of its orthogonal axis on the parent axis, (u _x u _y u _z ) represents the vector of its x-axis, and (v _x v _y v _z ) represents the vector of its y-axis, (w _x w _y w _z ) represents the vector of its z-axis. If the axes of the two rigid bodies are identical and their relative positions and attitudes remain the same, then when one rigid body moves, the other rigid body performs the same motion in the same coordinate system.

Correspondingly, according to the rigid body transformation process, the preset torso posture formula may be:

Among them, q _BTracker is the real-time attitude data of the torso tracker, and q _Body is the real-time posture of the thoracic joint;

Is the initial posture data of the thoracic joint obtained in the "I" posture.

It is the initial posture data of the torso tracker acquired in the "I" type posture.

The upper arm real-time pose data formula can be:

Wherein the q _Shoulder is the upper arm real-time pose data, and the q _STracker is the real-time posture of the upper arm tracker.

The initial posture data of the upper arm acquired in the "I" posture,

The initial position data of the upper arm obtained in the "I" type posture. In addition, the upper arm tracker is divided into two parts, a left upper arm tracker and a right upper arm tracker. The upper arm real-time attitude data is collected by the left and right trackers, and the left upper arm tracker and the right upper arm tracker can be described by q _STrackerL and q _STrackerR , respectively. Here, q _{STracker is} used to describe it.

Further, after the trunk posture data and the upper arm posture data are acquired, the positions of the trunk and the upper arm may also be offset-adjusted according to the chest position, and the adjustment value of the offset adjustment is a half body width. Wherein, the chest position is equal to the position of the head display + the vector of the head to the chest cavity. The chest position shift adjustment is obtained from the position of the chest cavity + half body width, that is, the position of the left upper arm joint, that is, the position of the right upper arm from the position of the chest cavity - half body width, that is, the position of the right shoulder joint. In addition, according to the pose data of the head directly collected by the head display, the position information of the neck, the chest cavity and the trunk can be obtained by calculating the relative positional relationship with the head, and the posture of the left and right clavicle should be consistent with the trunk.

S22. Determine a shoulder joint position according to the initial pose data, and calculate real-time data of the elbow joint according to the shoulder joint data and the shoulder joint change matrix, wherein the elbow joint data is an X-axis direction offset of a coordinate system of the shoulder joint. The upper arm is long.

Specifically, the initial pose data can read the shoulder joint position (ie, the upper arm starting position) p _shoulder , and the shoulder joint real-time position p _shoulder can be obtained from the chest center position ± half body width, wherein + represents The position of the left shoulder joint, - indicates the position of the right shoulder joint. Correspondingly, the calculation formula of the shoulder joint position may be:

Wherein said

For the transformation matrix, which represents the offset of the half shoulder width is determined by the transformation matrix

In progress, the p _Ribcage is a chest position, and the L _bodyWidth is a body length.

Further, the elbow joint is a child node of the shoulder joint, and the elbow joint position p _elbow is offset from the upper arm length along the x-axis direction of the shoulder joint coordinate system, and is obtained by the following formula

Where p _elbow is the elbow joint position and p _shoulder is the shoulder joint position.

For the transformation matrix, it represents the transformation matrix of the shoulder in the world coordinate system, and L _Upperarm is the upper arm length.

S23. Calculate an angle between the upper arm and the forearm according to the shoulder joint data and the elbow joint data, and calculate forearm pose data according to the included angle to obtain a first arm posture.

Specifically, the elbow joint coordinate system is the same as the shoulder joint coordinate system, and the elbow joint is a swivel joint and the degree of rotational freedom is only one. That is to say, the forearm can only be rotated about the z-axis of the elbow joint to determine the angle θ between the upper arm and the forearm to determine the forearm posture. In this embodiment, the local coordinate system of the elbow joint and the local coordinate system of the shoulder joint are the same, and the angle between the two is equal to α _elbow =180°-θ. Accordingly, the posture calculation formula of the elbow joint can be:

q _elbow =(α _elbow +q _elbow .yaw).toquaternions

among them,

It is the attitude of the elbow joint in the "I" posture. q _elbow .yaw is the angle of rotation around the z-axis under the attitude of the elbow joint in the "I" posture, and toquaternions converts the angular coordinate system to the quaternion method.

The angle θ between the upper arm and the forearm can be calculated according to the shoulder joint position p _shoulder , the elbow joint position p _elbow and the handle position p _hand , and the calculation formula of the angle between the upper arm and the forearm can be:

_{θ = arccos (V e2s V e2h} )

Where V _e2s =p _shoulder -p _elbow represents the unit vector of the elbow joint pointing to the shoulder joint.

V _e2h =p _hand -p _elbow denotes the unit vector of the elbow joint pointing to the wrist joint

Correspondingly, the calculation formula of the α _elbow can be:

El _elbow =180°-arccos(V _e2s , V _e2h )

Therefore, based on the original elbow joint attitude, α _elbow is added to obtain a new elbow joint posture, that is, the elbow joint posture is captured.

Illustratively, the calculating the angle between the upper arm and the forearm according to the shoulder joint data and the elbow joint data, and calculating the forearm posture data according to the included angle to obtain the first arm posture specifically includes:

Determining the first unit vector of the elbow joint to the shoulder joint based on the shoulder joint data and the real-time data of the elbow joint, and determining the second unit vector of the elbow joint pointing to the wrist joint according to the real data of the elbow joint and the real-time data of the wrist joint;

Calculating an angle between the upper arm and the forearm by using a cosine theorem according to the first unit vector and the second unit vector, and calculating forearm pose data according to the included angle to obtain a first arm posture.

S30. Convert the first arm posture into a second arm posture of the preset virtual character according to the preset built-in model, and drive the preset virtual character according to the second arm posture.

Specifically, the preset built-in model is preset and is independent of each joint local coordinate system in the preset bone model, and the preset built-in model forms an orthogonal base by using a forward axis, a horizontal axis, and a vertical axis. Each of the preset skeleton model terminal joints pre-stored by the virtual reality system may be identified by a coordinate axis of the built-in model, so that differences in local coordinate axes of different skeleton models may be ignored. Correspondingly, when the human body wears the motion capture system, before initializing the preset posture to obtain the initial pose data, the method includes:

S030. Receive and store a preset skeleton model, and associate each joint coordinate system of the preset skeleton model with a preset built-in model to obtain a correspondence between the preset skeleton model and the preset built-in model.

Specifically, the built-in model corresponds to each joint of the preset skeleton model, and the joint name of the preset skeleton model that can be imported into the built-in model can be based on the coordinates of the joints of the preset skeleton model and the preset skeleton model respectively. Corresponding relationship is established to establish a correspondence between the preset built-in model and the joints of the preset skeleton model, so that when the captured real-time pose data is respectively imported into the preset built-in model, the preset skeleton model can be automatically imported, thereby Set the virtual character corresponding to the skeleton model to control. In a practical application, a plurality of skeleton models may be preset in the virtual reality system, and the coordinate systems of the joint points of the skeleton models are different, and each of the preset models has partial identical properties at that time. Thereby, the same properties of each preset skeleton model can be acquired, and a built-in skeleton model is generated according to the same property. For example, each imported skeletal model uses the head above the foot, that is, the vector with the preset built-in model facing up; the left hand is on the left side of the right hand, that is, the direction of the model toward the right can be determined; from the top to the right The vector can determine the forward vector by cross-multiplication, which determines the orthonormal basis of the built-in bone model. In this embodiment, the built-in model determines a bone joint of the built-in model according to the correspondence relationship, and the bone joint of the built-in model may be composed of three custom data structures, the data structure including a type of the coordinate axis and one A tag, where the type indicates the type of axis it belongs to, and the tag indicates the directional relationship of the axis to the built-in model. The mark may be 1 or -1, with 1 indicating the same direction and -1 indicating the opposite direction.

Further, the correspondence between the joints of the preset skeleton model and the built-in model can be established in sequence. That is to say, when the spatial information of the joint point of the preset skeleton model is first read, the spatial information of each joint point is associated with the preset built-in model, that is, the joint point coordinates of the different imported skeleton model resources are used by using the built-in model. The axes are marked and marked with the following data structure. For example, for a thoracic joint point, the forward axis of the built-in model corresponds to the y-axis (opposite direction) of the thoracic joint point, the horizontal axis corresponds to the x-axis of the thoracic joint point (in the opposite direction), and the vertical axis corresponds to the thoracic joint point. The axis (consistent direction), so that the preset built-in model is associated with the thoracic joint of the preset skeletal model. Of course, the other joint point axes of the preset skeleton model are respectively compared with the built-in model axes, and the coordinate axes of the joint points of the preset skeleton model and the three axes of the built-in model (forward axis, horizontal axis, and The correspondence of the vertical axis) simultaneously records the direction of the coordinate axis and the built-in model, thereby realizing the establishment of the correspondence between the preset skeleton model and the preset built-in model.

In addition, after the preset built-in model and the preset skeleton model are established, after the arm pose data is acquired, the preset built-in model can be used to redirect to correspond to the corresponding preset skeleton model. In this way, for different preset skeleton models, when the joint point coordinate system is different from the joint point coordinate system of the captured data, the captured data can also be converted into the joint coordinate system of the preset skeleton model. Correspondingly, the converting the first arm posture into the second arm posture of the preset virtual character according to the preset built-in model, and driving the preset virtual character according to the second arm posture comprises:

S31, redirecting the first arm posture to a coordinate system of each joint point of the preset built-in model;

S32. Convert the first arm posture to each joint point coordinate system of the preset bone model according to the correspondence relationship to obtain a second arm posture;

S33. Determine a preset virtual character corresponding to the preset skeleton model according to the second arm posture.

Specifically, the pose data (including the position and posture data) of each joint point included in the first arm posture is acquired, and the x-axis, the y-axis, and the z-axis of the pose data of each joint point are corresponding to the preset built-in. a coordinate axis of the model, and according to a correspondence relationship between the preset built-in model coordinate axis and the coordinate axis of the joint point of the preset skeleton model, the coordinate axis of the captured data corresponding to the built-in model coordinate axis is converted into the coordinate axis of the preset skeleton model, Thereby, the first arm posture is converted into the second arm posture and the virtual character corresponding to the preset skeleton model is driven. In this embodiment, the process of reorienting the first arm posture may be: matching a coordinate axis of the captured data with a coordinate axis of the preset built-in model, and redirecting the coordinate system of the captured data. The specific process may be: capturing three coordinate axes of joint points in the data and three coordinate axes of the built-in model, and correspondingly positioning the forward axis, the horizontal axis and the vertical axis of the built-in model on the coordinate axes of the data model; The axis is labeled as the x-axis, the vector x = the x-axis vector of the captured data; if the forward axis is the y-axis, the vector x = the y-axis vector of the captured data; if the forward axis is the z-axis, the vector x = the captured data The z-axis vector; if the horizontal axis is labeled as the x-axis, the vector y = the x-axis vector of the captured data; if the horizontal axis is the y-axis, the vector y = the y-axis vector of the captured data; if the horizontal axis is the z-axis, Vector y= captures the z-axis vector of the data; if the vertical axis is labeled x-axis, the vector z=x-axis vector of the captured data; if the vertical axis is the y-axis, the vector z=the y-axis vector of the captured data; if vertical The axis is the z-axis, the vector z=the z-axis vector of the captured data; finally, the new coordinate axis is formed according to the vector x, the vector y and the vector z, and the redirection of the captured data has been completed.

Based on the above-described virtual reality driving method based on arm motion capture, the present application further provides a computer readable storage medium, wherein the computer readable storage medium stores one or more programs, the one or more The program may be executed by one or more processors to implement the steps in the virtual motion driving method based on arm motion capture as described in the above embodiments.

Based on the above-described virtual reality driving method based on arm motion capture, the present invention further provides a virtual reality system, including: a motion capture system and a virtual reality device, as shown in FIG. 5, the virtual reality device includes at least one processor (processor) 20; display 21; and memory 22, which may also include a communication interface 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22, and the communication interface 23 can complete communication with each other through the bus 24. The display screen 21 is set to display a user guidance interface preset in the initial setting mode. The communication interface 23 can transmit information. Processor 20 may invoke logic instructions in memory 22 to perform the methods in the above-described embodiments.

In addition, the logic instructions in the memory 22 described above may be implemented in the form of software functional units and sold or used as separate products, and may be stored in a computer readable storage medium.

The memory 22 is a computer readable storage medium, and can be configured to store a software program, a computer executable program, a program instruction or a module corresponding to the method in the embodiment of the present disclosure. The processor 20 performs the functional application and data processing by executing software programs, instructions or modules stored in the memory 22, i.e., implements the methods in the above embodiments.

The memory 22 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to usage of the terminal device, and the like. Further, the memory 22 may include a high speed random access memory, and may also include a nonvolatile memory. For example, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., may also be used to store a program code. State storage medium.

In addition, the above-described storage medium and the specific processes loaded and executed by the plurality of instruction processors in the mobile terminal have been described in detail in the above methods, and will not be further described herein.

It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not limited thereto; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that The technical solutions described in the foregoing embodiments are modified, or the equivalents of the technical features are replaced. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. A virtual reality driving method based on arm motion capture, characterized in that it comprises:

   When the human body wears the motion capture system, initializing the preset posture to obtain initial pose data, wherein the initial pose data includes preset pose data and human body raw data;

   Capturing the real-time posture data of the human body, determining the first arm posture according to the real-time attitude data and the initial pose data by using an arm-to-link transformation matrix method, wherein the first arm posture includes a trunk joint and an arm motion chain joint;

   Converting the first arm posture into a second arm posture of a preset virtual character according to a preset built-in model, and driving the preset virtual character according to the second arm posture.
2. The virtual reality driving method based on arm motion capture according to claim 1, wherein the motion capture system comprises at least a head display, a left and right handle, a left and right upper arm tracker, and a torso tracker.
3. The virtual reality driving method based on the arm motion capture according to claim 1, wherein when the human body wears the motion capture system, initializing the preset posture to obtain the initial pose data specifically includes:

   When the human body wears the motion capture system, capturing preset posture data of the human body in a preset posture, wherein the preset posture includes a first posture and a second posture;

   Correcting a preset skeleton model according to the preset pose data corresponding to the first posture;

   Calculating initial body data based on the preset pose data to obtain initial pose data.
4. The virtual reality driving method based on the arm motion capture according to claim 3, wherein the calculating the initial body data according to the preset pose data to obtain the initial pose data specifically includes:

   Calculating a relative positional relationship of each joint of the upper body according to the preset pose data corresponding to the first posture;

   Comparing the preset pose data corresponding to the second posture with the preset pose data corresponding to the first posture to calculate human body raw data to obtain initial pose data.
5. The virtual reality driving method based on arm motion capture according to claim 1, wherein the capturing real-time attitude data of the human body is determined according to the real-time attitude data and the initial pose data by using an arm-to-link transformation matrix method. The arm posture specifically includes:

   Capturing the real-time posture data of the human body, calculating the real-time data of the trunk joint according to the preset torso posture formula, and calculating the real-time position data of the upper arm according to the preset upper arm posture formula;

   Determining the shoulder joint position according to the initial pose data, and calculating real-time data of the elbow joint according to the shoulder joint data and the shoulder joint change matrix, wherein the elbow joint data is offset from the upper arm length in the X-axis direction of the coordinate system of the shoulder joint ;

   Calculating an angle between the upper arm and the forearm according to the shoulder joint data and the elbow joint data, and calculating forearm pose data according to the included angle to obtain a first arm posture.
6. The virtual reality driving method based on arm motion capture according to claim 5, wherein the angle between the upper arm and the forearm is calculated according to the shoulder joint data and the elbow joint data, and the forearm posture is calculated according to the angle The data to get the first arm pose specifically includes:

   Determining the first unit vector of the elbow joint to the shoulder joint based on the shoulder joint data and the real-time data of the elbow joint, and determining the second unit vector of the elbow joint pointing to the wrist joint according to the real data of the elbow joint and the real-time data of the wrist joint;

   Calculating an angle between the upper arm and the forearm by using a cosine theorem according to the first unit vector and the second unit vector, and calculating forearm pose data according to the included angle to obtain a first arm posture.
7. The virtual reality driving method based on the arm motion capture according to claim 1, wherein when the human body wears the motion capture system, initializing the preset posture to obtain the initial pose data includes:

   The preset skeleton model is received and stored, and each joint coordinate system of the preset skeleton model is associated with the preset built-in model to obtain a correspondence between the preset skeleton model and the preset built-in model.
8. The virtual reality driving method based on arm motion capture according to claim 7, wherein the first arm posture is converted into a second arm posture of a preset virtual character according to a preset built-in model, and according to the The second arm gesture driving the preset virtual character specifically includes:

   Redirecting the first arm posture to a coordinate system of each joint point of the preset built-in model;

   Converting the first arm posture into each joint point coordinate system of the preset skeleton model according to the correspondence relationship to obtain a second arm posture;

   Determining a preset virtual character corresponding to the preset skeleton model according to the second arm posture.
9. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs, the one or more programs being executable by one or more processors to implement claim 1 The steps in the virtual reality driving method based on arm motion capture according to any one of the above.
10. A virtual reality system, comprising: a motion capture system and a virtual reality device, the virtual reality device comprising a processor, a memory and a communication bus; wherein the memory stores a computer executable by the processor Readable program

    The communication bus implements connection communication between the processor and the memory;

    The step in the virtual reality driving method based on the arm motion capture according to any one of claims 1-8 when the processor executes the computer readable program.

Images