CN113268136B

CN113268136B - Thumb and palm degree of freedom resolving method and device and data glove

Info

Publication number: CN113268136B
Application number: CN202010094836.8A
Authority: CN
Inventors: 李红红; 韩久琦; 崔丽华
Original assignee: Jingdong Technology Information Technology Co Ltd
Current assignee: Jingdong Technology Information Technology Co Ltd
Priority date: 2020-02-14
Filing date: 2020-02-14
Publication date: 2024-05-17
Anticipated expiration: 2040-02-14
Also published as: CN113268136A

Abstract

The embodiment of the application relates to a method and a device for resolving freedom degree between thumb and palm and a data glove, wherein the method comprises the following steps: and extracting the first quaternion to the fourth quaternion from quaternions respectively acquired by the first inertial sensor and the second inertial sensor. Acquiring a fifth quaternion acquired by a first inertial sensor at a kth moment and a sixth quaternion acquired by a second inertial sensor; according to the first quaternion to the sixth quaternion, calculating a first reference quaternion and a second reference quaternion which correspond to the fifth quaternion respectively; and calculating the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove at the kth moment according to the fifth quaternion, the first reference quaternion and the second reference quaternion. The technical scheme provided by the application has higher instantaneity and higher stability. And two parameters of the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove are calculated, so that the gesture recognition precision is improved, and complex gesture actions are recognized conveniently.

Description

Thumb and palm degree of freedom resolving method and device and data glove

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to a method and a device for resolving degree of freedom between thumb and palm and a data glove.

Background

The human hand is a main organ for communicating the human body with the outside, and can transmit various gesture information. The human hand is used as an information output organ, and has the characteristics of multiple information types, large information quantity, strong mapping capability and the like. Because of these characteristics, hand wearable devices have been paid attention to, and a large number of smart wearable devices have been designed. In various hand intelligent wearing devices, the data glove has strong development and research potential because the hand gesture information can be fully utilized.

In the prior art, the hand motion is analyzed and identified by using the data glove, and the main technologies can be divided into two types:

(1) Gesture recognition technology based on vision

The gesture recognition technology based on vision is to acquire hand pictures by using a single or a plurality of cameras, process proper image signals, and recognize gesture information of human hands by using a pattern recognition algorithm. The visual gesture recognition technology has obvious advantages, the main hardware component is a camera, the hardware development difficulty is small, the software recognition algorithm is the core of the technology, the system cost is low, and the cost performance is extremely high.

(2) Hardware-based gesture recognition techniques

The gesture recognition technology based on hardware is that a sensor and an external structure are installed at key parts of a human hand so as to realize space positioning of the human hand and provide a touch feedback function. Common sensors used in data gloves are fiber optic sensors, flexible sensors, and the like. The method has the advantages of extremely high space use flexibility, low secondary development cost, high stability and quick response.

Disadvantages of the prior art:

(1) Vision-based gesture recognition techniques: although visual gesture recognition can be completed only by camera support, a complex algorithm is required as support, and complex feature extraction and recognition operation take a lot of time and resources. Moreover, visual gesture recognition relies on cameras, which have limited areas to arrange, are not functional within this range, and may cause personal privacy exposure.

(2) Hardware-based gesture recognition techniques: the data glove based on the optical fiber sensor has high comfort level, sensitive response and extremely high cost, can only roughly identify finger bending, cannot identify the geographic space coordinates of the hand, can generate fatigue damage after being used for many times, and greatly influences the measurement accuracy of angles. The principle of the flexible sensor is basically consistent with that of the optical fiber data glove, and the flexible sensor can only detect the bending condition of fingers and has insufficient gesture recognition precision.

Disclosure of Invention

In view of this, in order to solve the technical problems that in the prior art, the data glove is not enough in precision for complex gesture recognition or the algorithm is too complex, the embodiment of the invention provides a method and a device for resolving the degree of freedom between a thumb and a palm, and the data glove.

In a first aspect, embodiments of the present invention provide a method for resolving degrees of freedom between a thumb and a palm, the method being applied to a data glove; the method comprises the following steps:

Extracting a first quaternion, a second quaternion, a third quaternion and a fourth quaternion from quaternions respectively acquired by the first inertial sensor and the second inertial sensor, wherein the first quaternion and the second quaternion are respectively as follows: when the data glove is determined to be in a first preset initialization posture and in a static state, a first quaternion acquired by a first inertial sensor and a second quaternion acquired by a second inertial sensor; the third quaternion and the fourth quaternion are respectively: when the data glove is determined to be in a second preset initialization posture and in a static state, a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor;

acquiring a fifth quaternion acquired by a first inertial sensor at a kth moment and a sixth quaternion acquired by a second inertial sensor;

According to the first quaternion to the sixth quaternion, calculating a first reference quaternion and a second reference quaternion which correspond to the fifth quaternion respectively;

And calculating the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove at the kth moment according to the fifth quaternion, the first reference quaternion and the second reference quaternion, wherein k is a positive integer greater than 1.

In one possible embodiment, the first inertial sensor and the second inertial sensor are each also used to acquire triaxial angular velocity data; extracting a first quaternion, a second quaternion, a third quaternion and a fourth quaternion from quaternions respectively acquired by a first inertial sensor and a second inertial sensor, wherein the method specifically comprises the following steps of:

according to triaxial angular velocity data acquired by the first inertial sensor and the second inertial sensor respectively, determining a first moment when the data glove is in a first preset initialization posture and in a static state;

According to the triaxial angular velocity data acquired by the first inertial sensor and the second inertial sensor respectively, determining a second moment when the data glove is in a second preset initialization posture and in a static state;

Acquiring a first quaternion acquired by a first inertial sensor and a second quaternion acquired by a second inertial sensor at a first moment; and acquiring a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor at the second moment.

In one possible implementation manner, according to the first quaternion to the sixth quaternion, calculating a first reference quaternion and a second reference quaternion corresponding to the fifth quaternion respectively specifically includes:

According to the sixth quaternion, the first quaternion and the second quaternion, calculating a first reference quaternion corresponding to the fifth quaternion under a first preset initialization posture;

and calculating a second reference quaternion corresponding to the fifth quaternion under a second preset initialization posture according to the fifth quaternion, the third quaternion and the fourth quaternion.

In one possible implementation manner, according to the sixth quaternion, the first quaternion and the second quaternion, calculating a first reference quaternion corresponding to the fifth quaternion in the first preset initialization posture specifically includes:

Calculating a first offset quaternion according to the first quaternion and the second quaternion;

And calculating a first reference quaternion corresponding to the fifth quaternion under a first preset initialization posture according to the sixth quaternion and the first offset quaternion.

In one possible implementation manner, according to the fifth quaternion, the third quaternion and the fourth quaternion, calculating a second reference quaternion corresponding to the fifth quaternion in the second preset initialization posture specifically includes:

calculating a second bias quaternion according to the third quaternion and the fourth quaternion;

and calculating a second reference quaternion corresponding to the fifth quaternion under a second preset initialization posture according to the fifth quaternion and the second offset quaternion.

In one possible implementation manner, according to triaxial angular velocity data acquired by the first inertial sensor and the second inertial sensor respectively, determining a first moment when the data glove is in a first preset initialization posture and in a static state specifically includes:

Acquiring triaxial angular velocity data acquired by a first inertial sensor at a t moment and triaxial angular velocity data acquired by a second inertial sensor at the t moment when the data glove is in a first preset initialization posture;

Calculating thumb combined angular velocity data at the t moment according to the triaxial angular velocity data acquired by the first inertial sensor at the t moment; calculating palm combination angular velocity data at the t moment according to the triaxial angular velocity data acquired by the second inertial sensor at the t moment;

When the thumb and palm combination angular velocity data at the moment t are smaller than a preset threshold value, determining the moment t as a first moment, wherein t is a positive integer, sequentially advancing the values of t, and initially setting the value to be 1.

In one possible implementation manner, according to the triaxial angular velocity data acquired by the first inertial sensor and the second inertial sensor respectively, determining the second moment when the data glove is in the second preset initialization posture and in the static state specifically includes:

Acquiring triaxial angular velocity data acquired at the h moment by the first inertial sensor when the data glove is in a second preset initialization posture, and triaxial angular velocity data acquired at the h moment by the second inertial sensor;

Calculating thumb combined angular velocity data at the h moment according to the three-axis angular velocity data acquired by the first inertial sensor at the h moment, and calculating palm combined angular velocity data at the h moment according to the three-axis angular velocity data acquired by the second inertial sensor at the h moment;

When the thumb and palm combination angular velocity data at the h moment are smaller than a preset threshold value, determining the h moment as a second moment, wherein h is a positive integer, sequentially advancing values of h, and initially setting the value to be 1.

In one possible embodiment, after acquiring the fifth quaternion acquired by the first inertial sensor and the sixth quaternion acquired by the second inertial sensor at the kth time, the method further includes:

And respectively carrying out normalization processing on the first quaternion, the second quaternion, the third quaternion, the fourth quaternion, the fifth quaternion and the sixth quaternion.

In one possible embodiment, the flexion and extension degrees of freedom and the rotation degrees of freedom between the thumb and the palm in the data glove at the kth moment are calculated according to the fifth quaternion, the first reference quaternion and the second reference quaternion, wherein k is a positive integer greater than 1, and specifically includes:

calculating the degree of freedom of flexion and extension between the thumb and the palm in the data glove at the kth moment according to the first reference quaternion and the fifth quaternion;

and calculating the rotation freedom degree between the thumb and the palm in the data glove at the kth moment according to the fifth quaternion, the second reference quaternion and the bending and stretching freedom degree.

In one possible embodiment, the first preset initialization gesture includes: merging initial gestures of the bits; the second preset initialization gesture includes: the initial bit pose is rotated.

In a second aspect, embodiments of the present invention provide a device for resolving degrees of freedom between a thumb and a palm, the device corresponding to a data glove, the device comprising:

The extraction unit is used for extracting a first quaternion, a second quaternion, a third quaternion and a fourth quaternion from quaternions respectively acquired by the first inertial sensor and the second inertial sensor, wherein the first quaternion and the second quaternion are respectively: when the data glove is determined to be in a first preset initialization posture and in a static state, a first quaternion acquired by a first inertial sensor and a second quaternion acquired by a second inertial sensor; the third quaternion and the fourth quaternion are respectively: when the data glove is determined to be in a second preset initialization posture and in a static state, a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor;

The processing unit is used for calculating a first reference quaternion and a second reference quaternion which respectively correspond to the fifth quaternion according to the first quaternion to the sixth quaternion;

In a third aspect, embodiments of the present invention provide a data glove comprising at least: the glove comprises a glove body, a first inertial sensor arranged on the thumb of the glove, a second inertial sensor arranged in the palm of the hand, a processor and a memory;

the first inertial sensor is used for collecting triaxial angular velocity data and quaternions corresponding to the thumb in real time;

the second inertial sensor is used for acquiring triaxial angular velocity data and quaternions corresponding to the palm in real time;

the processor is configured to execute the thumb-palm freedom degree calculation program stored in the memory, so as to implement the thumb-palm freedom degree calculation method as described in any embodiment of the first aspect.

In a fourth aspect, embodiments of the present invention provide a computer storage medium storing one or more programs executable by a data glove as described in the third aspect to implement a thumb-palm degree of freedom resolution method as described in any of the embodiments of the first aspect.

According to the thumb-palm degree-of-freedom resolving method provided by the embodiment of the application, the first quaternion to the fourth quaternion are respectively extracted from quaternions respectively acquired by the first inertial sensor and the second inertial sensor. The first quaternion to the fourth quaternion are quaternions under a preset initialization posture. And acquiring a fifth quaternion acquired by the first inertial sensor and a sixth quaternion acquired by the second inertial sensor at any moment (k moment). And calculating a first reference quaternion and a second reference quaternion corresponding to the fifth quaternion according to the first quaternion to the sixth quaternion. And finally, calculating the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove at any moment according to the fifth quaternion, the first reference quaternion and the second reference quaternion. Through the method, quaternions under different initialization postures are acquired by utilizing the inertial sensors respectively corresponding to the thumb and the palm, and the first reference quaternion and the second reference quaternion corresponding to the thumb at any moment are calculated by combining the quaternions acquired by the inertial sensors respectively corresponding to the thumb and the palm at any moment. Thereby realizing the real-time calibration of the data glove. And further, two parameters of the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove are calculated. Compared with the prior art, the technical scheme provided by the application does not need to support a complex algorithm or extract complex features, avoids the condition that complex feature extraction and recognition take a large amount of time and resources, and has higher instantaneity and higher stability. Moreover, the two parameters of the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove are calculated, so that the gesture recognition precision is improved, and complex gesture actions are recognized conveniently.

Drawings

FIG. 1 is a schematic flow chart of a method for resolving degrees of freedom between thumb and palm according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a graphical principle of calculating rotational degrees of freedom provided by the present invention;

FIG. 3 is a schematic diagram of a device for resolving degrees of freedom between thumb and palm according to an embodiment of the present invention;

fig. 4 is a schematic diagram of a data glove structure according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

For the purpose of facilitating an understanding of the embodiments of the present invention, reference will now be made to the following description of specific embodiments, taken in conjunction with the accompanying drawings, which are not intended to limit the embodiments of the invention.

Fig. 1 is a schematic flow chart of a method for resolving degrees of freedom between a thumb and a palm according to an embodiment of the present application, as shown in fig. 1, the method is applied to a data glove. The data glove comprises at least a glove body, a first inertial sensor mounted on the thumb of the glove, a second inertial sensor mounted in the palm of the hand, a processor and a memory. Before the method is executed, a tested person is required to wear the data glove, adjust gestures according to a preset initialization gesture, and stay for a certain time. For example, 2 seconds. Then, the following method of the present application is performed. The method comprises the following steps:

Step 110, extracting a first quaternion, a second quaternion, a third quaternion and a fourth quaternion from quaternions respectively acquired by the first inertial sensor and the second inertial sensor.

Specifically, the first inertial sensor is mainly used for collecting quaternion and triaxial angular velocity data of the thumb. The second sensor is used for collecting quaternion and triaxial angular velocity data of the palm. Of course, the inertial sensor may also collect three-axis acceleration data, as will be described below.

Wherein, first quaternion and second quaternion are respectively: when the data glove is determined to be in a first preset initialization posture and in a static state, a first quaternion acquired by a first inertial sensor and a second quaternion acquired by a second inertial sensor; the third quaternion and the fourth quaternion are respectively: when the data glove is determined to be in the second preset initialization posture and in the static state, the third quaternion acquired by the first inertial sensor and the fourth quaternion acquired by the second inertial sensor.

Alternatively, in a specific example, the first preset gesture may include a merge bit initial gesture. The second preset initialization gesture includes: the initial bit pose is rotated.

Wherein, the inertial data glove worn by the right hand has five fingers in a straightened and combined state as the initial posture of the combined position of the thumb; the four fingers are combined and straightened, and the thumb is far away from the index finger as far as possible, and is kept parallel to the palm surface, so that the thumb is rotated to be in an initial position.

Optionally, the first quaternion, the second quaternion, the third quaternion and the fourth quaternion are extracted from quaternions respectively acquired from the first inertial sensor and the second inertial sensor, and specifically may be acquired by the following manner:

Further alternatively, when determining the first time and the second time, the following may be obtained:

Similarly, acquiring triaxial angular velocity data acquired at the h moment by the first inertial sensor and triaxial angular velocity data acquired at the h moment by the second inertial sensor when the data glove is in a second preset initialization posture;

In practice, when calculating the thumb-engaging angular velocity data at time t, for example, the thumb-engaging angular velocity data, it can be calculated by using the formula 1:

Wherein, gt _x(T)、gt_y(T)、gt_z (T) is the angular velocity data of the X axis, the Y axis and the Z axis of the thumb at the moment T respectively.

Similarly, when calculating the palm closing angular velocity data at time t, it can be obtained by calculation of formula 2:

where sp_gyro (T) is palm and angular velocity data, and gp _x(T)、gp_y(T)、gp_z (T) is angular velocity data of X-axis, Y-axis, and Z-axis of the palm at time T, respectively.

If the thumb combined angular speed st_gyro (T) and the palm combined angular speed p_gyro (T) are smaller than the preset threshold value, determining that the thumb and the palm are in a static state. When the thumb and the palm are in a static state, the current moment can be determined to be the first moment. Similarly, when the h moment is determined, formula calculation similar to formula 1 and formula 2 is adopted, and if both thumb and palm closing angular velocity data capable of determining the h moment are smaller than a preset threshold value, the h moment can be determined to be the second moment.

Step 120, obtaining a fifth quaternion collected by the first inertial sensor and a sixth quaternion collected by the second inertial sensor at the kth moment.

At the kth time, the testee can control the thumb and the palm of the data glove to form any angle.

In step 130, according to the first quaternion to the sixth quaternion, a first reference quaternion and a second reference quaternion respectively corresponding to the fifth quaternion are calculated.

Specifically, a first reference quaternion corresponding to the fifth quaternion in the first preset initialization posture may be calculated according to the sixth quaternion, the first quaternion and the second quaternion;

Optionally, calculating, according to the sixth quaternion, the first quaternion and the second quaternion, a first reference quaternion corresponding to the fifth quaternion in the first preset initialization posture specifically includes:

According to the fifth quaternion, the third quaternion and the fourth quaternion, calculating a second reference quaternion corresponding to the fifth quaternion under a second preset initialization posture, wherein the calculation method specifically comprises the following steps of:

And 140, calculating the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove at the kth moment according to the fifth quaternion, the first reference quaternion and the second reference quaternion.

Specifically, the purpose of the scheme is to solve the angle data of two degrees of freedom of the thumb relative to the palm. Then, it is necessary to obtain quaternions (first to fourth quaternions) of the thumb merge bit and the rotation bit initial bit in step 110, and then obtain offset quaternions with the palm quaternions according to the calculation. Later, the motion of any position of the glove is calculated mainly by the relative angle of the thumb and the palm, the palm drives the whole hand to move, and the quaternions of the palm and the thumb at different moments are different, but by offsetting the quaternions, the reference quaternion of the merging position of the thumb and the reference quaternion of the rotation position of the thumb of the relative palm can be obtained. Namely a first reference quaternion and a second reference quaternion.

The specific calculation process comprises the following steps:

The first quaternion acquired by the thumb end first inertial sensor when the merging position is stationary is q11= [ q ₀,q₁,q₂,q₃ ], and the second quaternion acquired by the second inertial sensor corresponding to the palm is q0= [ q ₀₀,q₀₁,q₀₂,q₀₃ ]; the third quaternion collected by the thumb end first inertial sensor when the rotation position is static is q12= [ q ₁₀,q₁₁,q₁₂,q₁₃ ], and the fourth quaternion collected by the second inertial sensor corresponding to the palm is q0' = [ q _00',q_01',q_02',q_03' ] for illustration.

After q11, q0, q12 and q0' are acquired, a normalization process may be performed.

Here, the normalization processing of q11 is described as an example, and specifically as follows:

q ₀(T)_n＝q₀ (T)/norm (T) (equation 4)

Q ₁(T)_n＝q₁ (T)/norm (T) (equation 5)

Q ₂(T)_n＝q₂ (T)/norm (T) (equation 6)

Q ₃(T)_n＝q₃ (T)/norm (T) (equation 7)

Wherein norm (T) is the modulus of the quaternion at the time T, q ₀(T)_n、q₁(T)_n、q₂(T)_n、q₃ (T) _n are respectively the gesture quaternions at the time T obtained after normalization, and the thumb quaternion of the merging bit after normalization is nq11= [ q ₀_n,q₁_n,q₂_n,q₃ _n ]; the other three normalized quaternions obtained by the same method are respectively

nq12＝[q₁₀_n,q₁₁_n,q₁₂_n,q₁₃_n]

nq0＝[q₀₀_n,q₀₁_n,q₀₂_n,q₀₃_n]

nq0’＝[q_00'_n,q_01'_n,q_02'_n,q_03'_n]

Similarly, for the quaternion q_finger at the thumb end and the quaternion q_ plam at the palm end at any K moment, nq_finger and nq_ plam are normalized (normalized) respectively, so that subsequent use is facilitated.

According to the first quaternion and the second quaternion, calculating a first offset quaternion, namely calculating an offset quaternion of thumb merging bit quaternion nq11= [ q ₀_n,q₁_n,q₂_n,q₃ _n ] and palm quaternion nq0= [ q ₀₀_n,q₀₁_n,q₀₂_n,q₀₃ _n ], wherein the following formula is specifically referred to:

Q1= (nq 0) ^-1 (nq 11) (formula 8)

Wherein (nq 0) ^-1 is the inverse of nq 0.

According to the third quaternion and the fourth quaternion, a second offset quaternion is calculated, namely, the offset quaternion of the thumb rotation quaternion nq12 and the palm quaternion nq0' is calculated, and the following formula is specifically referred to:

Q2= (nq 0') ^-1 (nq 12) (formula 9)

After the first bias quaternion and the second bias quaternion are calculated, the first reference quaternion and the second reference quaternion need to be calculated. That is, at time k, a first reference quaternion with the thumb in the merge position initial pose and a second reference quaternion with the thumb in the rotate initial pose are calculated.

The specific process comprises the following steps:

nq11= (nq_ plam) ×q1 (formula 10)

Where Nq11 is the first reference quaternion, nq_ plam is the sixth quaternion, and Q1 is the first bias quaternion.

Nq22= (nq_ plam) ×q2 (formula 11)

Where Nq22 is the second reference quaternion, nq_ plam is the sixth quaternion, and Q2 is the second bias quaternion.

Finally, when the degree of freedom of flexion and extension between the thumb and the palm in the data glove is calculated at the kth moment, the degree of freedom is mainly calculated according to the first reference quaternion and the fifth quaternion.

Specifically, the angular difference between quaternions can be calculated by using the dot product of the quaternions, so Qu Shenjiao degrees between the thumb and the palm can be obtained by the included angle between quaternions Nq11 and nq_finger:

stretch _angle＝cos^-1 (θ). Times.2 (equation 13)

Wherein Stretch _angle is the degree of freedom of flexion and extension between thumb and palm in the data glove.

The Pythagorean theorem is also utilized when calculating the rotational degrees of freedom between the thumb and palm in the data glove. Namely, according to the fifth quaternion, the second reference quaternion and the degree of freedom of flexion and extension, the degree of freedom of rotation between the thumb and the palm in the data glove at the kth moment is calculated by using the Pythagorean theorem.

Angle=cos ^-1 (α) ×2 (equation 15)

See in particular fig. 2. angle can be regarded as +.aoa 'in Δaoa', plane AOB is regarded as the wrist plane perpendicular to the palm plane, we want to get the rotation angle of thumb and palm, and we can be regarded as +.aob.

According to the cosine theorem of the triangle, can be calculated to obtain

Since a' B is perpendicular to the plane AOB, the calculation is then:

A' B ²＝OA²×cos²(Stretch_angle) (equation 17)

AB ²＝AA'²-A'B² (equation 18)

OB ²＝OA²×sin²(Stretch_angle) (equation 19)

The cosine law can be used to find:

angle aob=180- & lt BAO- & lt ABO (formula 22)

Exten _angle = AOB (equation 23)

Finally, exten _angle is obtained according to formulas 13 to 23, that is, the degree of freedom of rotation between the thumb and the palm in the data glove at the kth time is obtained. Wherein k is a positive integer greater than 1. Let k be a positive integer greater than 1 because, considering the initial initialization state, at least the data glove is given to collect the data of the initial state and then perform the subsequent calculation. Generally, the value of k should be a value greater than t and h.

Finally, through the calculation of the data glove, the angle values of two degrees of freedom of the thumb and the palm at the moment k can be transmitted through a communication technology (for example, 5G), and the angles of the two degrees of freedom of the thumb of the bionic manipulator are controlled to execute specific application.

According to the thumb and palm degree of freedom resolving method provided by the embodiment of the application, the first quaternion to the fourth quaternion are respectively extracted from quaternions respectively acquired by the first inertial sensor and the second inertial sensor. The first quaternion to the fourth quaternion are quaternions under a preset initialization posture. And acquiring a fifth quaternion acquired by the first inertial sensor and a sixth quaternion acquired by the second inertial sensor at any moment (k moment). And calculating a first reference quaternion and a second reference quaternion corresponding to the fifth quaternion according to the first quaternion to the sixth quaternion. And finally, calculating the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove at any moment according to the fifth quaternion, the first reference quaternion and the second reference quaternion. Through the method, quaternions under different initialization postures are acquired by utilizing the inertial sensors respectively corresponding to the thumb and the palm, and the first reference quaternion and the second reference quaternion corresponding to the thumb at any moment are calculated by combining the quaternions acquired by the inertial sensors respectively corresponding to the thumb and the palm at any moment. Thereby realizing the real-time calibration of the data glove. And further, two parameters of the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove are calculated. Compared with the prior art, the technical scheme provided by the application does not need to support a complex algorithm or extract complex features, avoids the condition that complex feature extraction and recognition take a large amount of time and resources, and has higher instantaneity and higher stability. Moreover, the two parameters of the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove are calculated, so that the gesture recognition precision is improved, and complex gesture actions are recognized conveniently.

Fig. 3 is a device for resolving degrees of freedom between thumb and palm according to an embodiment of the present invention, the device comprising: an extraction unit 301 and a processing unit 302.

The extracting unit 301 is configured to extract a first quaternion, a second quaternion, a third quaternion, and a fourth quaternion from quaternions collected by the first inertial sensor and the second inertial sensor respectively, where the first quaternion and the second quaternion are respectively: when the data glove is determined to be in a first preset initialization posture and in a static state, a first quaternion acquired by a first inertial sensor and a second quaternion acquired by a second inertial sensor; the third quaternion and the fourth quaternion are respectively: when the data glove is determined to be in a second preset initialization posture and in a static state, a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor;

a processing unit 302, configured to calculate a first reference quaternion and a second reference quaternion corresponding to the fifth quaternion respectively according to the first quaternion to the sixth quaternion;

Optionally, the first inertial sensor and the second inertial sensor are each further configured to acquire triaxial angular velocity data;

The processing unit 302 is specifically configured to determine, according to triaxial angular velocity data acquired by the first inertial sensor and the second inertial sensor respectively, a first moment when the data glove is in a first preset initialization posture and in a static state;

and determining a second moment when the data glove is in a second preset initialization posture and in a static state according to the triaxial angular velocity data respectively acquired by the first inertial sensor and the second inertial sensor.

The extracting unit 301 is specifically configured to obtain a first quaternion collected by the first inertial sensor and a second quaternion collected by the second inertial sensor at a first moment; and acquiring a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor at the second moment.

Optionally, the processing unit 302 is specifically configured to calculate, according to the sixth quaternion, the first quaternion, and the second quaternion, a first reference quaternion corresponding to the fifth quaternion in the first preset initialization posture;

Optionally, the processing unit 302 is specifically configured to calculate the first bias quaternion according to the first quaternion and the second quaternion;

Optionally, the processing unit 302 is specifically configured to calculate the second bias quaternion according to the third quaternion and the fourth quaternion;

Optionally, the extracting unit 301 is specifically configured to obtain three-axis angular velocity data collected by the first inertial sensor at the t-th moment and three-axis angular velocity data collected by the second inertial sensor at the t-th moment when the data glove is in the first preset initialization posture;

The processing unit 302 is specifically configured to calculate thumb combined angular velocity data at time t according to the three-axis angular velocity data acquired by the first inertial sensor at time t; calculating palm combination angular velocity data at the t moment according to the triaxial angular velocity data acquired by the second inertial sensor at the t moment;

Optionally, the extracting unit 301 is specifically configured to obtain triaxial angular velocity data acquired at the h moment by the first inertial sensor and triaxial angular velocity data acquired at the h moment by the second inertial sensor when the data glove is in the second preset initialization posture;

the processing unit 302 is specifically configured to calculate thumb combined angular velocity data at the h moment according to the three-axis angular velocity data collected by the first inertial sensor at the h moment, and calculate palm combined angular velocity data at the h moment according to the three-axis angular velocity data collected by the second inertial sensor at the h moment;

Optionally, the processing unit 302 is further configured to normalize the first quaternion, the second quaternion, the third quaternion, the fourth quaternion, the fifth quaternion, and the sixth quaternion respectively.

Optionally, the processing unit 302 is specifically configured to calculate, at the kth time, a degree of freedom of flexion and extension between the thumb and the palm in the data glove according to the first reference quaternion and the fifth quaternion;

Optionally, the first preset initialization gesture includes: merging initial gestures of the bits; the second preset initialization gesture includes: the initial bit pose is rotated.

The functions performed by the functional components in the thumb-palm freedom degree calculating device according to the present embodiment are described in detail in the embodiment corresponding to fig. 1, and thus are not described here again.

According to the device for resolving the degree of freedom between the thumb and the palm, which is provided by the embodiment of the application, the first quaternion to the fourth quaternion are respectively extracted from quaternions respectively acquired by the first inertial sensor and the second inertial sensor. The first quaternion to the fourth quaternion are quaternions under a preset initialization posture. And acquiring a fifth quaternion acquired by the first inertial sensor and a sixth quaternion acquired by the second inertial sensor at any moment (k moment). And calculating a first reference quaternion and a second reference quaternion corresponding to the fifth quaternion according to the first quaternion to the sixth quaternion. And finally, calculating the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove at any moment according to the fifth quaternion, the first reference quaternion and the second reference quaternion. Through the method, quaternions under different initialization postures are acquired by utilizing the inertial sensors respectively corresponding to the thumb and the palm, and the first reference quaternion and the second reference quaternion corresponding to the thumb at any moment are calculated by combining the quaternions acquired by the inertial sensors respectively corresponding to the thumb and the palm at any moment. Thereby realizing the real-time calibration of the data glove. And further, two parameters of the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove are calculated. Compared with the prior art, the technical scheme provided by the application does not need to support a complex algorithm or extract complex features, avoids the condition that complex feature extraction and recognition take a large amount of time and resources, and has higher instantaneity and higher stability. Moreover, the two parameters of the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove are calculated, so that the gesture recognition precision is improved, and complex gesture actions are recognized conveniently.

Fig. 4 is a schematic structural diagram of a data glove according to an embodiment of the present invention, and the data glove 400 shown in fig. 4 at least includes: a glove body (not shown), a first inertial sensor 401 mounted on the glove thumb, a second inertial sensor 402 mounted in the palm of the hand, at least one processor 403, memory 404, at least one network interface 405, and other user interfaces 406. The various components in the thumb and palm degree of freedom solution data glove 400 are coupled together by a bus system 407. It is appreciated that the bus system 407 is used to enable connected communication between these components. The bus system 407 includes a power bus, a control bus, and a status signal bus in addition to a data bus. But for clarity of illustration the various buses are labeled as bus system 407 in fig. 4.

The user interface 406 may include, among other things, a display, a keyboard, or a pointing device (e.g., a mouse, a trackball, a touch pad, or a touch screen, etc.).

It will be appreciated that the memory 404 in embodiments of the invention can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The nonvolatile memory may be a Read-only memory (ROM), a programmable Read-only memory (ProgrammableROM, PROM), an erasable programmable Read-only memory (ErasablePROM, EPROM), an electrically erasable programmable Read-only memory (ElectricallyEPROM, EEPROM), or a flash memory, among others. The volatile memory may be a random access memory (RandomAccessMemory, RAM) that acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as static random access memory (STATICRAM, SRAM), dynamic random access memory (DYNAMICRAM, DRAM), synchronous dynamic random access memory (SynchronousDRAM, SDRAM), double data rate synchronous dynamic random access memory (DoubleDataRate SDRAM, ddr SDRAM), enhanced synchronous dynamic random access memory (ENHANCED SDRAM, ESDRAM), synchronous link dynamic random access memory (SYNCHLINKDRAM, SLDRAM), and direct memory bus random access memory (DirectRambusRAM, DRRAM). The memory 404 described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

In some implementations, the memory 404 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 4041 and application programs 4042.

The operating system 4041 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application 4042 includes various application programs such as a media player (MediaPlayer), a Browser (Browser), and the like for realizing various application services. The program implementing the method of the embodiment of the present invention may be contained in the application 4042.

In the embodiment of the present invention, the processor 403 is configured to execute the method steps provided by the method embodiments by calling a program or an instruction stored in the memory 404, specifically, a program or an instruction stored in the application 4042, for example, including:

Extracting a first quaternion, a second quaternion, a third quaternion and a fourth quaternion from quaternions respectively acquired by the first inertial sensor and the second inertial sensor;

Optionally, determining a first moment when the data glove is in a first preset initialization posture and in a static state according to triaxial angular velocity data respectively acquired by the first inertial sensor and the second inertial sensor;

Optionally, according to the sixth quaternion, the first quaternion and the second quaternion, calculating a first reference quaternion corresponding to the fifth quaternion under a first preset initialization posture;

Optionally, calculating a first bias quaternion from the first quaternion and the second quaternion;

Optionally, calculating a second bias quaternion from the third quaternion and the fourth quaternion;

Optionally, acquiring triaxial angular velocity data acquired by the first inertial sensor at the t-th moment and triaxial angular velocity data acquired by the second inertial sensor at the t-th moment when the data glove is in the first preset initialization posture;

Optionally, acquiring triaxial angular velocity data acquired at the h moment by the first inertial sensor and triaxial angular velocity data acquired at the h moment by the second inertial sensor when the data glove is in the second preset initialization posture;

Optionally, the normalization processing is performed on the first quaternion, the second quaternion, the third quaternion, the fourth quaternion, the fifth quaternion and the sixth quaternion respectively.

Optionally, calculating the degree of freedom of flexion and extension between the thumb and the palm in the data glove at the kth moment according to the first reference quaternion and the fifth quaternion;

The method disclosed in the above embodiment of the present invention may be applied to the processor 403 or implemented by the processor 403. Processor 403 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in the processor 403 or by instructions in the form of software. The processor 403 may be a general purpose processor, a digital signal processor (DigitalSignalProcessor, DSP), an application specific integrated circuit (application specific IntegratedCircuit, ASIC), an off-the-shelf programmable gate array (FieldProgrammableGateArray, FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software elements in a decoding processor. The software elements may be located in a random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 404, and the processor 403 reads the information in the memory 404 and, in combination with its hardware, performs the steps of the method described above.

It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more application specific integrated circuits (ApplicationSpecificIntegratedCircuits, ASIC), digital signal processors (DigitalSignalProcessing, DSP), digital signal processing devices (DSPDEVICE, DSPD), programmable logic devices (ProgrammableLogicDevice, PLD), field programmable gate arrays (Field-ProgrammableGateArray, FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of the application, or a combination thereof.

For a software implementation, the techniques herein may be implemented by means of units that perform the functions herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

It should be further noted that the functions performed by the processor 403 and the memory 404 are described in detail above, and the functions of the first inertial sensor 401 and the second inertial sensor 402 are indirectly described.

Here, it is explicitly described that:

The first inertial sensor 401 is configured to collect three-axis angular velocity data and quaternion corresponding to the thumb in real time;

the second inertial sensor 402 is configured to collect three-axis angular velocity data and quaternions corresponding to the palm in real time.

The data glove provided in this embodiment may be the data glove as shown in fig. 4, and all the steps of the method for resolving the degree of freedom between the thumb and the palm as shown in fig. 1 may be performed, so as to achieve the technical effects of the method for resolving the degree of freedom between the thumb and the palm as shown in fig. 1, and the description thereof will be omitted herein for brevity.

The embodiment of the invention also provides a storage medium (computer readable storage medium). The storage medium here stores one or more programs. Wherein the storage medium may comprise volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid state disk; the memory may also comprise a combination of the above types of memories.

When the one or more programs are executed by the one or more processors, the method for resolving the degree of freedom between the thumb and the palm executed on the data glove side is implemented.

The processor is used for executing a thumb-palm freedom degree resolving program stored in the memory so as to realize the following steps of the thumb-palm freedom degree resolving method executed on the data glove side:

Here, it is explicitly described that:

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of function in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

The foregoing detailed description of the invention has been presented for purposes of illustration and description, and it should be understood that the invention is not limited to the particular embodiments disclosed, but is intended to cover all modifications, equivalents, alternatives, and improvements within the spirit and principles of the invention.

Claims

1. A method of resolving degrees of freedom between a thumb and a palm, the method being applied to a data glove, the method comprising:

extracting a first quaternion, a second quaternion, a third quaternion and a fourth quaternion from quaternions respectively acquired by a first inertial sensor and a second inertial sensor, wherein the first quaternion and the second quaternion are respectively: when the data glove is determined to be in a first preset initialization posture and in a static state, a first quaternion acquired by the first inertial sensor and a second quaternion acquired by the second inertial sensor; the third quaternion and the fourth quaternion are respectively: when the data glove is determined to be in a second preset initialization posture and in a static state, a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor;

acquiring a fifth quaternion acquired by the first inertial sensor and a sixth quaternion acquired by the second inertial sensor at a kth moment;

According to the first quaternion to the sixth quaternion, calculating a first reference quaternion and a second reference quaternion respectively corresponding to the fifth quaternion, including: obtaining a first offset quaternion according to the first quaternion and the second quaternion, and calculating a first reference quaternion under the first preset initialization posture according to the first offset quaternion and the sixth quaternion; obtaining a second offset quaternion according to the third quaternion and the fourth quaternion, and calculating a second reference quaternion under the second preset initialization posture according to the second offset quaternion and the fifth quaternion;

Calculating the degree of freedom of flexion and extension and the degree of freedom of rotation between the thumb and the palm in the data glove at the kth moment according to the fifth quaternion, the first reference quaternion and the second reference quaternion, wherein k is a positive integer greater than 1, and the method comprises the following steps: calculating the degree of freedom of flexion and extension between the thumb and the palm in the data glove at the kth moment according to the first reference quaternion and the fifth quaternion; and calculating the rotation freedom degree between the thumb and the palm in the data glove at the kth moment according to the fifth quaternion, the second reference quaternion and the bending and stretching freedom degree.

2. The method of claim 1, wherein the first inertial sensor and the second inertial sensor are each further for acquiring triaxial angular velocity data; the extracting the first quaternion, the second quaternion, the third quaternion and the fourth quaternion from quaternions respectively collected by the first inertial sensor and the second inertial sensor specifically comprises:

According to the triaxial angular velocity data respectively acquired by the first inertial sensor and the second inertial sensor, determining a second moment when the data glove is in a second preset initialization posture and in a static state;

Acquiring a first quaternion acquired by the first inertial sensor and a second quaternion acquired by the second inertial sensor at the first moment; and acquiring a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor at the second moment.

3. The method according to claim 2, wherein the determining, according to the triaxial angular velocity data acquired by the first inertial sensor and the second inertial sensor respectively, the first moment when the data glove is in the first preset initialization posture and in the stationary state specifically includes:

acquiring triaxial angular velocity data acquired by the first inertial sensor at the t moment and triaxial angular velocity data acquired by the second inertial sensor at the t moment when the data glove is in the first preset initialization posture;

Calculating thumb angular velocity data at the t moment according to the triaxial angular velocity data acquired by the first inertial sensor at the t moment; calculating palm combination angular velocity data at the t moment according to the triaxial angular velocity data acquired by the second inertial sensor at the t moment;

When both the thumb closing angular speed data and the palm closing angular speed data at the t moment are smaller than a preset threshold value, determining the t moment as a first moment, wherein t is a positive integer, sequentially advancing the values of t, and initially setting the value to be 1.

4. The method according to claim 2, wherein the determining, according to the triaxial angular velocity data acquired by the first inertial sensor and the second inertial sensor respectively, the second moment when the data glove is in the second preset initialization posture and in the stationary state specifically includes:

Acquiring triaxial angular velocity data acquired at the h moment by the first inertial sensor and triaxial angular velocity data acquired at the h moment by the second inertial sensor when the data glove is in a second preset initialization posture;

When the thumb closing angular velocity data and the palm closing angular velocity data at the h moment are smaller than a preset threshold, determining the h moment as a second moment, wherein h is a positive integer, sequentially advancing values of h, and initially setting the value to be 1.

5. The method of claim 1, wherein after the acquiring the fifth quaternion acquired by the first inertial sensor and the sixth quaternion acquired by the second inertial sensor at a kth time, the method further comprises:

6. The method of claim 1, wherein the first preset initialization gesture comprises: merging initial gestures of the bits; the second preset initialization gesture includes: the initial bit pose is rotated.

7. A device for resolving degrees of freedom between a thumb and a palm, the device corresponding to a data glove, the device comprising:

The extraction unit is used for extracting a first quaternion, a second quaternion, a third quaternion and a fourth quaternion from quaternions respectively acquired by the first inertial sensor and the second inertial sensor, wherein the first quaternion and the second quaternion are respectively: when the data glove is determined to be in a first preset initialization posture and in a static state, a first quaternion acquired by the first inertial sensor and a second quaternion acquired by the second inertial sensor; the third quaternion and the fourth quaternion are respectively: when the data glove is determined to be in a second preset initialization posture and in a static state, a third quaternion acquired by the first inertial sensor and a fourth quaternion acquired by the second inertial sensor;

The processing unit is configured to calculate, according to the first quaternion to the sixth quaternion, a first reference quaternion and a second reference quaternion that respectively correspond to the fifth quaternion, and includes: obtaining a first offset quaternion according to the first quaternion and the second quaternion, and calculating a first reference quaternion under the first preset initialization posture according to the first offset quaternion and the sixth quaternion; obtaining a second offset quaternion according to the third quaternion and the fourth quaternion, and calculating a second reference quaternion under the second preset initialization posture according to the second offset quaternion and the fifth quaternion;

8. A data glove, the data glove comprising at least: the glove comprises a glove body, a first inertial sensor arranged on the thumb of the glove, a second inertial sensor arranged in the palm of the hand, a processor and a memory;

the second inertial sensor is used for collecting triaxial angular velocity data and quaternions corresponding to the palm in real time;

The processor is configured to execute the thumb-palm degree of freedom calculation program stored in the memory, so as to implement the thumb-palm degree of freedom calculation method according to any one of claims 1 to 6.

9. A computer storage medium storing one or more programs executable by the data glove of claim 8 to implement the thumb-palm degree of freedom resolution method of any one of claims 1 to 6.