US20230195234A1

US20230195234A1 - Gesture acquisition system

Info

Publication number: US20230195234A1
Application number: US17/967,043
Authority: US
Inventors: Chao Fang
Original assignee: Individual
Current assignee: Individual
Priority date: 2017-08-08
Filing date: 2022-10-17
Publication date: 2023-06-22
Also published as: EP3667564A4; US11507192B2; EP3667564A1; WO2019028650A1; US20200192487A1

Abstract

A gesture acquisition system, including: a finger posture acquisition device, a palm following device, and a wristband. The finger posture acquisition device includes a main acquisition module. The palm following device is arranged on a hand or the wrist and is configured to fix the main acquisition module on a palm. The main acquisition module is capable of synchronously swinging with the palm through the palm following device, and is configured to acquire finger postures. The palm following device includes a fixing component and a wrist posture acquisition device. The fixing component is configured to fix the main acquisition module on the palm, and synchronously swings with the palm. The wrist posture acquisition device is connected to the wristband. An end of the wrist posture acquisition device away from the wristband is connected to the fixing component.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of US Patent Application No. 16,785,552, filed on Feb. 7, 2020, which is a continuation of International Patent Application No. PCT/CN2017/096362, filed on Aug. 8, 2017. The content of the aforementioned application, including any intervening amendments thereto, is incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to the technical field of human-machine interaction (HMI), and in particular to a gesture acquisition system.

BACKGROUND

With the development of science and technology, the application of various smart devices is increasingly widespread. Human-machine interaction (HMI) between human beings and smart devices is increasing. HMI includes contact HNI and non-contact HMI. Contact HMI is more mature and complete, while non-contact HMI is still at the starting stage. As an intuitive and natural interaction way, gesture is an important means for human beings to exchange information with each other due to its quick expression and rich meaning. In the prior art, the gesture acquisition sensor is arranged statically in the third person of view or on the helmet or glasses. As a result, the coverage of the image sensor is limited so that the hand may not be captured by occlusion the captured image contains the background where a person stays; the resolution of the hand is low when the hand is far away from the sensor, and the hand may be shielded by other objects; and for a same gesture, image obtained from different angles greatly varies. Therefore, in the gesture recognition process, recognition is started only after focusing, tracking, image segmentation for background deletion, resizing & cropping, and manifold processing. Due to the high cost of power consumption for recognition and illumination and image sensors, there is no real-time, contactless, and precise wearable device for gesture recognition.

SUMMARY

An objective of the present disclosure is to provide a gesture acquisition system that can acquire finger posture information accurately with less recognition time, less computation cost, low power consumption and non-contact and wearable property.
The technical solutions of the present disclosure are described below.
This application provides a gesture acquisition, comprising a finger posture acquisition device, wherein the finger posture acquisition device comprises a main acquisition module which is located below a palm and can synchronously swing with the palm; and the main acquisition module is configured to acquire finger postures.
In some embodiments, the gesture acquisition system further comprises a wrist posture acquisition device configured to acquire adduction-abduction postures of a wrist and flexion-extension postures of the wrist; the wrist posture acquisition device comprises a wrist posture acquisition arm that does an adduction-abduction motion and a flexion-extension motion together with the wrist, a wristband fixed on a forearm, and a sensing module configured to acquire flexion-extension information and adduction-abduction information of the wrist posture acquisition arm; and the wrist posture acquisition arm is connected to the wristband.
In some embodiments, the gesture acquisition system further comprises a wristband tied around the wrist; the wrist posture acquisition arm is connected to the wristband; and the wrist posture acquisition arm does an adduction-abduction motion and a flexion-extension motion together with the wrist by the wristband.
In some embodiments, the wrist posture acquisition arm comprises an adduction-abduction arm that does an adduction-abduction motion together with the wrist and a flexion-extension arm that does a flexion-extension motion together with the wrist; and the sensing module comprises two angle sensors that are, respectively, an adduction-abduction angle sensor configured to acquire an adduction-abduction angle of the adduction-abduction arm and a flexion-extension angle sensor configured to acquire a flexion-extension angle of the flexion-extension arm.
In some embodiments, the wristband and an adduction-abduction axis of the wrist during the adduction-abduction motion have two points of intersection which are corresponding to the adduction-abduction arm; the wristband and an flexion-extension axis of the wrist during flexion-extension motion have two points of intersection which are corresponding to the flexion-extension arm; each of the adduction-abduction arm and the flexion-extension arm has a position fixed end and a free end opposite to the position fixed end; the position fixed end of one of the adduction-abduction arm and the flexion-extension arm is mounted on the wristband and located at one of two corresponding points of intersection, with a corresponding angle sensor being mounted there, the flexion-extension angle sensor being configured to sense rotation angle on the flexion-extension axis and the adduction-abduction angle sensor being configured to sense the rotation angle on the adduction-abduction axis, and the free end thereof is wound, along the wristband, to one of two points of intersection corresponding to the other one of the adduction-abduction arm and the flexion-extension arm and connected to the fixed end of the other one of the adduction-abduction arm and the flexion-extension arm; a join point of the adduction-abduction arm and the flexion-extension arm has an included angle, with the other angle sensor being mounted there; the free end of the other one of the adduction-abduction arm and the flexion-extension arm is arranged around the wristband and extended toward an outer side of the wristband to be fixed on the wrist at a position adjacent to the hand.
In some embodiments, the flexion-extension angle sensor is arranged along the flexion-extension axis and the adduction-abduction angle sensor is arranged along the adduction-abduction axis.
In some embodiments, the free end, located on the wrist at a position adjacent to the hand, of the adduction-abduction arm and the flexion-extension arm is wound to the bottom of the wrist at a position adjacent to the hand, with the main acquisition module being mounted there.
In some embodiments, the wrist posture acquisition arm is a telescopic pole; the wrist posture acquisition arm is connected to the wristband and comprises one end located above or lateral to the wristband and another end fixedly connected to the hand; the wrist posture acquisition arm and the wristband has a point of intersection located between one end and the another end; either or both end is able to swing around the point of intersection; and the sensing module acquires the flexion-extension information and adduction-abduction information of the wrist posture acquisition arm by acquiring swing information of that end.
In some embodiments, fisheye bearings are arranged on the wristband and (or) fixation structure, and the wrist posture acquisition arm is connected to the wristband and(or) fixation structure by running through the fisheye bearing.
In some embodiments, the sensing module is a Hall sensor; a magnet is arranged at a first end of the wrist posture acquisition arm; and the sensing module acquires the flexion-extension information and adduction-abduction information of the wrist posture acquisition arm by acquiring magnetic field information at the first end.
In some embodiments, the sensing module is located in either or both extension directions of the wrist posture acquisition arm and is farther from the one end of the wrist posture acquisition arm than the another end of the wrist posture acquisition arm.
In some embodiments, the gesture acquisition system further comprises a fixation structure that is fixed on the wrist at a position adjacent to the hand, the one end of the wrist posture acquisition arm being mounted on the fixation structure, and the main acquisition module being fixed on the fixation structure or the wrist posture acquisition arm.
In some embodiments, the gesture acquisition system further comprises a forearm posture acquisition device mounted on the wristband; the forearm posture acquisition device is configured to acquire forearm postures; and the forearm posture acquisition device comprises a three-axis gyroscope, a three-axis magnetometer, and a three-axis acceleration sensor.
In some embodiments, the wristband is shaped in a sleeve; a first fixation portion and a second fixation portion are arranged on the wristband at interval; and the first fixation portion and the second fixation portion are matched with parts, protruded on the body skin, of styloid process of radius and styloid process of ulna to fixedly mount the wristband on the wrist, respectively.
In some embodiments, the first fixation portion and the second fixation portion are through-holes or grooves formed on the wristband.
In some embodiments, the gesture acquisition system further comprises an ambient information acquisition device configured to search for, in the ambient environment, electronic devices to be controlled.
In some embodiments, the gesture acquisition system further comprises a processor connected to other electronic elements in the gesture acquisition system.
In some embodiments, the main acquisition module is camera; and the camera can be a depth camera or an image sensor such as an ultrasound.
In some embodiments, the main acquisition module is mounted within 0-5 cm from the wrist joint to fingers and 0-3 cm from the wrist to a person's arm, and 0-3 cm away from the body skin.
With the technical solutions, the present invention has the following beneficial effects.
In the gesture acquisition system of this application, the main acquisition module of the finger posture acquisition device is arranged below a palm and can synchronously swing with the palm such that the angle of view and the position of the main acquisition module are fixed relative to the palm. In this way, for a same gesture, it is recognized by images captured from one fixed direction, reducing the manifold processing time, avoiding the failure in gesture recognition due to occlusion, and saving the computations for tracking, segmentation and size scaling, and thus avoiding errors brought by these steps and reducing the computation time and power consumption. In addition, the defect that it is unable to perform gesture operations for a long period of time is eliminated. Meanwhile, the range of illumination is reduced and the power consumption for illumination is also reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a circuit block diagram of a gesture acquisition system according to an embodiment of the present disclosure;

FIG. 2 is a perspective view of the gesture acquisition system according to an embodiment of the present disclosure;

FIG. 3 is a schematic view of FIG. 2 ;

FIG. 4 is a perspective view of the gesture acquisition system according to an embodiment of the present disclosure;

FIG. 5 is a perspective view of the gesture acquisition system according to an embodiment of the present disclosure;

FIG. 6 is a perspective view of the gesture acquisition system according to an embodiment of the present disclosure;

FIG. 7 is an enlarged view of part VI of FIG. 5 and FIG. 6 ;

FIG. 8 is a perspective view of the gesture acquisition system when an adduction-abduction arm and a flexion-extension arm are dual-arm structured according to an embodiment of the present disclosure;

FIG. 9 is a perspective view of the gesture acquisition system when an adduction-abduction arm and a flexion-extension arm are single-arm structured according to an embodiment of the present disclosure;

FIG. 10 schematically shows a cooperation between a C-shaped ring and a wristband according to an embodiment of the present disclosure; and

FIG. 11 is a structural diagram of the C-shaped ring when the adduction-abduction arm is made of a flexible material.

DETAILED DESCRIPTION OF THE PRESENT INVENTION

In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described below in detail with reference to the accompanying drawings by embodiments. It should be understood that the specific embodiments to be described herein are merely used to explain the present disclosure, rather than limiting the present disclosure.
Referring to FIG. 1 , an embodiment of the present disclosure provides a gesture acquisition system, including a finger posture acquisition device 1. The finger posture acquisition device 1 includes a main acquisition module 11 which is located below a palm and synchronously swings with the palm, and an auxiliary acquisition module (not shown). The main acquisition module 11 may be configured to acquire finger postures by acquiring images of fingers and a part of a palm. The auxiliary acquisition module is also configured to assist in acquiring finger posture information. The auxiliary acquisition module may be located on the hand at any position where finger postures can be acquired, to assist in acquiring finger posture information. The auxiliary acquisition module may be, for example, a camera that acquires finger posture information by acquiring an image of a finger or acquires the position and orientation of the hand in the space with simultaneous localization and mapping (SLAM) algorithm and inverse kinematic or forward kinematic (IK/FK) algorithm. The main and auxiliary acquisition module may include, for another example, microphones that place close to the skin and acquire sound of fingers' action when tapping or snapping. The microphones are placed close to the skin to obtain the sound from the body tissues, so as to effectively avoid other sounds in the air such as speech sound.
It may be understood that, in human-machine interaction (HNI), in addition to finger postures, wrist postures and forearm postures are to be acquired by the gesture acquisition system.
To acquire wrist postures, the gesture acquisition system in the present disclosure further includes, for example, a wrist posture acquisition device 2 configured to acquire adduction-abduction posture information of the wrist and flexion-extension posture information of the wrist. In the present disclosure, by acquiring adduction-abduction posture information of the wrist and flexion-extension posture information of the wrist by the wrist posture acquisition device 2, wrist postures are acquired. It is to be noted that, anatomically, whiling opening your palm, the swing of the wrist around an axis perpendicular to the palm is called the adduction-abduction motion of the wrist, and the swing of the wrist along the axis perpendicular to the palm is called the flexion-extension motion of the wrist. The wrist posture acquisition device 2 in the present disclosure can acquire an adduction-abduction motion and a flexion-extension motion of the wrist so as to acquire the adduction-abduction posture information of the wrist and flexion-extension posture information of the wrist.
Referring to FIGS. 2-5 , in this embodiment, the wrist posture acquisition device 2 includes a wrist posture acquisition arm 22 that does an adduction-abduction motion and a flexion-extension motion together with the wrist and a sensing module 24 configured to acquire the flexion-extension information and the adduction-abduction information of the wrist posture acquisition arm 22, so as to acquire the wrist posture information. To ensure that the wrist posture acquisition arm 22 can do an adduction-abduction motion and a flexion-extension motion together with the wrist and that the sensing module 24 can acquire the flexion-extension information and the adduction-abduction information of the wrist posture acquisition arm 22 accurately, the wrist posture acquisition arm 22 is usually made of a hard material. In this way, the wrist posture acquisition arm 22 is less likely to deform, which can avoid the decreasing of accuracy affected by the easy deformation of the wrist posture acquisition arm to affect the flexion-extension motion and the adduction-abduction of the wrist posture acquisition arm 22.
To acquire forearm posture information, the gesture acquisition system in the present disclosure further includes, for example, a forearm posture acquisition device 3 configured to acquire forearm posture information. In this embodiment, the forearm posture acquisition device 3 includes a three-axis gyroscope, a three-axis magnetometer and a three-axis acceleration sensor, where the three-axis gyroscope is configured to detect the three-axis angular velocity of the forearm, the three-axis acceleration sensor is configured to detect the three-axis acceleration of the forearm, and the three-axis magnetometer is configured to detect the three-axis magnetism of the state of the forearm. With the arrangement of them, the detected forearm posture information is more accurate.
The gesture acquisition system in the present disclosure can also determine electronic devices to be controlled. The gesture acquisition system in the present disclosure further includes, for example, an ambient information acquisition device 4 configured to search for, in the ambient environment, electronic devices (computers, tablets, household appliances, IOTs, on-vehicle devices, etc.) to be controlled in interaction and to locate a person's arm relative to the indoor or outdoor local ambient environment. In this embodiment, the ambient information acquisition device 4 is an ambient information camera. The forearm posture acquisition device 3 further includes a light reflecting point arranged on the wristband. The ambient information acquisition device 4 can acquire information about the position of the light reflecting point.
To ease of human-machine interaction, the gesture acquisition system in the present disclosure further includes a processor 5 and a wireless communication module 6. The finger posture acquisition device 1, the wrist posture acquisition device 2, the forearm posture acquisition device 3, the ambient information acquisition device 4, and the wireless communication module 6 are all connected to the processor 5. The finger posture information acquired by the finger posture acquisition device 1, the wrist posture information acquired by the wrist posture acquisition device 2, the forearm posture information acquired by the forearm posture acquisition device 3, the electronic devices to be controlled which are searched by the ambient information acquisition device 4, and space information of the forearm are all sent to the processor 5. The processor 5 determines the electronic devices to be controlled and calculates spatial coordinates of the forearm according to such information, and outputs corresponding operating instructions according to such information, which operating instructions are sent to the electronic devices to be controlled via the wireless communication module 6. In this way, HMI with the electronic devices to be controlled is realized. That is, electronic devices can be controlled by gestures.
Referring to FIGS. 5-6 , the gesture acquisition system in the present disclosure further includes a wristband 7 tied around the wrist. The main acquisition module 11, the wrist posture acquisition device 2, the forearm posture acquisition device 3, the ambient information acquisition device 4, the wireless communication module 6 and the processor 5 may be, for example, all connected to the wristband 7. As used herein, “connected to the wristband 7” includes “connected to the wristband 7 directly” and “connected to the wristband 7 indirectly, i.e., by an element, for example, a connecting rod or the like”. The arrangement of the wristband 7 is to conveniently realize the synchronous swing of the main acquisition module 11 with the palm. The wrist posture acquisition arm 22 of the wrist posture acquisition device 2 moves together with the wrist rotation. The way of connecting the main acquisition module 11 and the wrist posture acquisition device 2 to the wristband 7 will be described below in details.
In other embodiments, the wristband 7 may be omitted, and the main acquisition module 11 may be fixed below the palm by a sucker or in other ways. The wrist posture acquisition device 2 may be arranged on the wrist by a sucker. The forearm posture acquisition device 3, the ambient information acquisition device 4, the wireless communication module 6 and the processor 5 may be fixed on a person's arm, or connected to a person's arm by structures such as connecting rods and located on an outer side of the arm. The way of mounting the forearm posture acquisition device 3, the ambient information acquisition device 4, the wireless communication module 6 and the processor 5 is not limited thereto, as long as the corresponding signal acquisition, transmission and processing can be realized.
In this embodiment, the wristband 7 is shaped in a sleeve. A first fixation portion 71 and a second fixation portion 72 are arranged on the wristband 7 at interval. The first fixation portion 71 and the second fixation portion 72 are matched with parts, protruded on the body skin, of styloid process of radius and styloid process of ulna to fixedly mount the wristband 7 on the wrist, respectively, in order to prevent the wristband 7 from falling off or sliding at the wrist. Specifically, the first fixation portion 71 and the second fixation portion 72 are through-holes or grooves formed on the wristband 7. In this way, styloid process of radius and styloid process of ulna can be received in the through-holes or grooves to render the wristband 7 to be fixed on the wrist. It is to be noted that, as shown, the description is given by taking a wristband to be worn on the right hand as an example, with the right hand palm facing down, and fingers being perpendicular to and pointing to paper. It may be understood that a wristband to be worn on the left hand is structurally the same as the wristband to be worn on the right hand, except that the first fixation portion 71 and the second fixation portion 72 need to be formed correspondingly to parts, protruded on the body skin, of styloid process of radius on the left hand and styloid process of ulna on the left hand, respectively.
Specifically, referring to FIG. 2 , a stereoscopic structure diagram of a first embodiment of the gesture acquisition system according to the present disclosure is shown. In this embodiment, the wrist posture acquisition arm 22 includes an adduction-abduction arm 220 that does an adduction-abduction motion together with the wrist and a flexion-extension arm 222 that does a flexion-extension motion together with the wrist; and the sensing module 24 includes two angle sensors that are, respectively, an adduction-abduction angle sensor 240 configured to acquire an adduction-abduction angle of the adduction-abduction arm 220 and a flexion-extension angle sensor 242 configured to acquire a flexion-extension angle of the flexion-extension arm 222. In this embodiment, the adduction-abduction angle of the adduction-abduction arm 220 together with the adduction-abduction motion of the wrist is acquired by the adduction-abduction arm 220, and the flexion-extension angle of the flexion-extension arm 222 together with the flexion-extension motion of the wrist is acquired by the flexion-extension angle sensor 242. In this way, the adduction-abduction angle and the flexion-extension angle of the wrist are acquired accurately. Thus, the wrist postures are acquired.
Further, after the wristband 7 is worn on the wrist, the wristband 7 and an adduction-abduction motion axis of the wrist during adduction-abduction motion have two points of intersection which are corresponding to the one end of adduction-abduction arm 220, and the wristband 7 and a flexion-extension axis of the wrist during flexion-extension have two points of intersection which are corresponding to the one end of flexion-extension arm 222. The adduction-abduction arm 220 has a position fixed end and a free end opposite to the position fixed end. The flexion-extension arm 222 also has a position fixed end and a free end opposite to the position fixed end, where the position fixed end of one of the adduction-abduction arm 220 and the flexion-extension arm 222 is mounted on the wristband 7 and located at one of two corresponding points of intersection, with a corresponding angle sensor being mounted there, and the free end thereof is wound, along the wristband 7, to one of two points of intersection corresponding to the other one of the adduction-abduction arm 220 and the flexion-extension arm 222 and connected to the position fixed end of the other one of the adduction-abduction arm 220 and the flexion-extension arm 222. A join point of the adduction-abduction arm 220 and the flexion-extension arm 222 has an included angle, with the other angle sensor being mounted there. The free end of the other one of the adduction-abduction arm 220 and the flexion-extension arm 222 is arranged around the wristband 7 and extended toward an outer side of the wristband 7 to be fixed on the wrist at a position adjacent to the hand. In this embodiment, the free end is fixed on the hand at a position adjacent to the wrist, by clamping the acquisition arm to the hand at a position adjacent to the wrist. In this way, when the wrist does a flexion-extension motion, the flexion-extension arm 222 will do a flexion-extension motion together with the wrist, and the flexion-extension angle sensor 242 can sense the change in the angle of the flexion-extension arm 222 and thus sense the flexion-extension postures of the wrist. When the wrist does an adduction-abduction motion, the adduction-abduction arm 220 will do an adduction-abduction motion together with the wrist, and the adduction-abduction angle sensor 240 can sense the change in the angle of the adduction-abduction arm 220 and thus sense the adduction-abduction postures of the wrist.
In this embodiment, the free end, located on the wrist at a position adjacent to the hand, of the adduction-abduction arm 220 or the flexion-extension arm 222 is wound to the bottom of the wrist at a position adjacent to the hand, with the main acquisition module 11 being mounted there, in order to ensure that the main acquisition module 11 can swing with the palm. In other embodiments, a connecting rod may be extended from the wristband 7 to be located below the palm, and the main acquisition module 11 is mounted on the connecting rod.
To ensure the accuracy of the acquired angle information, the flexion-extension angle sensor 242 is arranged along the flexion-extension axis and the adduction-abduction angle sensor 240 is arranged along the adduction-abduction axis. That is, both the flexion-extension angle sensor 242 and the adduction-abduction angle sensor 240 are perpendicular to the tangent of the corresponding point of intersection.
Specifically, in this embodiment, as shown in FIG. 3 , in the three-dimensional coordinate system, X is the flexion-extension axis, Z is the adduction-abduction axis, and Y is the axial direction of the wristband 7. In the specific arrangement, the wristband 7 and the adduction-abduction axis have two points of intersection A and B, while the wristband 7 and the flexion-extension axis have two points of intersection C and D. One end of the flexion-extension arm 222 is mounted at any one of points of intersection of the wristband 7 and the flexion-extension axis, for example, point of intersection C. The flexion-extension angle sensor 242 is connected to the position fixed end of the flexion-extension arm 222 and arranged on the X-axis. The free end of the flexion-extension arm 222 is arranged around the wristband 7, and wound to the point A; then, the position fixed end of the adduction-abduction arm 220 is connected to the free end of the flexion-extension arm 222; and the free end of the adduction-abduction arm 220 is wound to the outer side of the wristband 7 and fixed on the hand at a position adjacent to the wrist. Specifically, the free end of the adduction-abduction arm 220 is extended to the outer side of the wristband 7 while being wound toward the point of intersection C, and then wound toward the hand at a position adjacent to the wrist to be located on the outer side of the wristband 7 and fixed on the hand at a position adjacent to the wrist. Here, the free end of the adduction-abduction arm 220 is fixed on the hand at a position adjacent to the wrist, by clamping the adduction-abduction arm 220 on the hand at a position adjacent to the wrist. In this way, when the wrist does a flexion-extension motion, the flexion-extension arm 222 swings, surrounding the wristband 7, around the flexion-extension axis of the wrist joint, and the flexion-extension angle sensor 242 rotates together with the flexion-extension arm 222, so that the flexion-extension angle sensor 242 acquires the swing angle of the flexion-extension arm 222 so as to acquire the flexion-extension postures of the wrist. In this case, the adduction-abduction arm 220 will not swing around the flexion-extension axis and the adduction-abduction angle sensor 240 will not acquire the flexion-extension angle of the flexion-extension arm 222. When the wrist does an adduction-abduction motion, the adduction-abduction arm 220 is fixed on the hand adjacent to the wrist and thus swings around the adduction-abduction axis together with the wrist, the angle between the flexion-extension arm 222 and the adduction-abduction arm 220 changes as the adduction-abduction arm 220 swings, and the adduction-abduction angle sensor 240 acquires the swing angle of the adduction-abduction arm 220 so as to acquire the adduction-abduction postures of the wrist. In this case, the flexion-extension arm 222 will not swing around the adduction-abduction axis and the adduction-abduction angle sensor 242 will not acquire the flexion-extension angle of the adduction-abduction arm 220.
To mount the main acquisition module 11 conveniently, the free end of the adduction-abduction arm 220 may be wound to the bottom of the hand at a position adjacent to the wrist and then extended toward the palm. The main acquisition module 11 is mounted at the free end of the adduction-abduction arm 220 to ensure that the main acquisition module 11 synchronously swings together with the palm.
In another embodiment, as shown in FIG. 4 , X is the flexion-extension axis, Z is the adduction-abduction axis, and Y is the axial direction of the wristband 7. In the specific arrangement, in this embodiment, the position fixed end of the adduction-abduction arm 220 is mounted at any one of points of intersection of the wristband 7 and the adduction-abduction axis, for example, point of intersection A. The adduction-abduction angle sensor 240 is fixed at the fixed end of the adduction-abduction arm 220 and arranged along the Z-axis. The free end of the adduction-abduction arm 220 is arranged around the wristband 7, and then wound to, for example, the point C. The fixed end of the flexion-extension arm 222 is connected to the free end of the adduction-abduction arm 220. The flexion-extension angle sensor 242 is mounted at the join point of the flexion-extension arm 222 and the adduction-abduction arm 220. The flexion-extension angle sensor 242 is arranged along the X-axis. The free end of the flexion-extension arm 222 is wound to the outer side of the wristband 7 and fixed on the hand at a position adjacent to the wrist. Specifically, the free end of the flexion-extension arm 222 is wound toward the point of intersection A and then toward the point of intersection D and extended to the outer side of the wristband 7 while being wound, and then wound toward the hand at a position adjacent to the wrist to be located on the outer side of the wristband 7 and fixed on the hand at a position adjacent to the wrist. Here, the free end of the flexion-extension arm 222 is fixed on the hand at a position adjacent to the wrist, by clamping the flexion-extension arm 222 on the hand at a position adjacent to the wrist. In this way, when the wrist does a flexion-extension motion, the flexion-extension arm 222 is fixed on the hand adjacent to the wrist and thus swings around the flexion-extension axis together with the wrist, the angle between the flexion-extension arm 222 and the adduction-abduction arm 220 changes, and the flexion-extension angle sensor 242 thus acquires the swing angle of the flexion-extension arm 222 so as to acquire the flexion-extension postures of the wrist. In this case, the adduction-abduction arm 220 will not swing around the flexion-extension axis and the adduction-abduction angle sensor 240 will not acquire the flexion-extension angle of the flexion-extension arm 222. When the wrist does an adduction-abduction motion, the adduction-abduction arm 220 swings, by the wristband 7, around the adduction-abduction axis together with the wrist, and the adduction-abduction angle sensor 240 swings together with the adduction-abduction axis, so that the adduction-abduction angle sensor 240 acquires the swing angle of the adduction-abduction arm 220 so as to acquire the adduction-abduction postures of the wrist. In this case, the flexion-extension arm 222 will not swing around the adduction-abduction axis and the flexion-extension angle sensor 242 will not acquire the adduction-abduction angle of the adduction-abduction arm 220.
To mount the main acquisition module 11 conveniently, the free end of the flexion-extension arm 222 may be wound to the bottom of the hand at a position adjacent to the wrist and then extended toward the palm. The main acquisition module 11 is mounted at the free end of the flexion-extension arm 222 to ensure that the main acquisition module 11 synchronously swings together with the palm.
Referring to FIGS. 5-6 , a stereoscopic structure diagram of a second embodiment of the gesture acquisition system according to the present disclosure is shown. In this embodiment, the wrist posture acquisition arm 22 is a telescopic pole. The wrist posture acquisition arm 22 is connected to the wristband 7, and includes one end located above, lateral to or below the wristband 7 and another end fixedly connected to the wrist adjacent to the hand. The wrist posture acquisition arm 22 and the wristband 7 has a point of intersection located between the two ends. One end of the wrist posture acquisition arm 22 is able to swing around the point of intersection. The sensing module 24 acquires the flexion-extension information and the adduction-abduction information of the wrist posture acquisition arm 22 by acquiring swing information of the end.
Specifically, in this embodiment, a fisheye bearing 75 is arranged on the wristband 7, and the wrist posture acquisition arm 22 is connected to the wristband 7 by running through the fisheye bearing 75. Both the structure and the operating principle of the fisheye bearing 75 are well-known and will not be repeated here. The wrist posture acquisition arm 22 can swing together with the wrist by the fisheye bearing 75 arranged on the wristband 7 (see FIG. 5 ) or the fixation structure 8 (see FIG. 6 ). Specifically, the part, between the fisheye bearing 75 and the other end of the wrist posture acquisition arm 22, extends or retracts, depending upon actual requirements. The one end of the wrist posture acquisition arm 22 swings together with the wrist by the fisheye bearing 75. The sensing module 24 can acquire the posture of the wrist by acquiring the swing information of the one or two ends. More specifically, when the wrist does an adduction-abduction motion, the one or two ends of the wrist posture acquisition arm 22 swings laterally together with the wrist by the fisheye bearing 75, and the sensing module 24 acquires the adduction-abduction posture information of the wrist by acquiring the lateral swing angle of the one end. When the wrist does a flexion-extension motion, one or two end of the wrist posture acquisition arm 22 swings vertically together with the wrist by the fisheye bearing 75, and the sensing module 24 acquires the flexion-extension posture information of the wrist by acquiring the vertical swing angle of the one or two ends.
In this embodiment, referring to FIG. 7 , the sensing module 24 is a Hall sensor. A magnet 26 is arranged at the one end of the wrist posture acquisition arm 22. The magnet 26 forms a magnetic field in vicinity of this end of the wrist posture acquisition arm 22. The Hall sensor senses the magnetic field formed by the magnet 26. The Hall sensor can sense the change in the magnetic field, which is caused by the swing of the one end of the wrist posture acquisition arm 22. Therefore, the sensing module 24 acquires the up-down swing angle and left-right swing angle of the one or two ends of the wrist posture acquisition arm 22 and thus acquires the flexion-extension angle and the adduction-abduction angle of the wrist, by acquiring magnetic field information at the end of the wrist posture acquisition arm 22.
In this embodiment, the sensing module 24 is located in an extension direction of the wrist posture acquisition arm 22 and is farther from the other end of the wrist posture acquisition arm 22, so that the sensing module 24 can acquire the magnetic field information at the one or two ends of the wrist posture acquisition arm 22 accurately.
In this embodiment, the gesture acquisition system in the present disclosure further includes a fixation structure 8 that is C-shaped. Specifically, the fixation structure 8 may be formed by bending a hard plastic board. The fixation structure 8 is fixed on the wrist at a position adjacent to the hand, for example, fixed by clamping. The fixation structure 8 is closer to the hand than the wristband 7. The other end of the wrist posture acquisition arm 22 is fixed on the fixation structure 8, and the main acquisition module 11 is fixed on the fixation structure 8. To locate the main acquisition module 11 below the palm, a connecting rod is extended from the fixation structure 8 to be located below the palm, for example in a direction to fingers, and the main acquisition module 11 is fixed on the connecting rod.
In the above description, the main acquisition module 11 is mounted within 0-5 cm from the wrist joint to fingers and 0-3 cm from the wrist to a person's arm, and 0-3 cm, preferably 0-2 cm, away from the body skin. The arrangement of the main acquisition module 11 in this region can realize the accurate acquisition of the finger posture information. Compared with the prior art, the gesture acquisition system of the present invention has the following advantages.
1. The cost is reduced.
Traditional gesture recognition devices capture hand images from other perspectives or external environments to recognize gestures. By such methods, each device to be controlled needs to be equipped with a recognition device (keyboard, mouse, touching pad). For example, there are 20 devices to be controlled (pads, computers, TVs, mobile phones, other IOT devices such as smart household appliances) in the home, and each device requires a gesture recognition device. It is very costly. In contrast, in the present disclosure, the electronic devices to be controlled are determined by the ambient information acquisition device 4. Multiple devices are controlled by only one gesture recognition device.
2. The accuracy of gesture recognition is greatly improved.
a) In the prior art, a system of recognizing a gesture will shoot the picture from the camera deployed in the external environments or other perspectives. It is necessary to segment hand actions from the complex and noisy background, in order to reduce the false recognition rate. In the present disclosure, by arranging the main acquisition module below the palm and ensuring that it synchronously swings together with the palm, that is, by ensuring that the angle of view and the position of the main acquisition module are fixed relative to the palm, for a same gesture, this gesture is recognized by images captured from one fixed perspective. It is unnecessary to segment the hand from the background. Meanwhile, the recognition system has no computation cost for background deletion. b) In the prior art, the system of recognizing a gesture from images shot from external environments or other perspectives should process the size scaling, rotation, distance adjustment and occlusion for the segmented gesture images. In the present design, the distance from the finger to the camera is constant, field of view is constant, and size scaling and rotation of the images is not required. Furthermore, there will be no large-area occlusion between the image sensor and hand, but in the prior art there is always occlusion between the camera and hand when the camera was deployed at the other perspective point.
c) For the system of recognizing a gesture from images taken from other perspectives, when the camera is used from other perspectives, it is required to recognize a same finger gesture from different angles and different illumination conditions. To realize this purpose, a large amount of data training is needed (the accuracy of recognition neural network is ensured only by training the recognition system with a large number of images of a same gesture captured from different angles). For a same gesture, for example, holding the fingers straight and then closing them, when it is recognized by images shot from other perspectives, it is necessary to obtain photos from different angles within 360 degrees, so as to ensure the recognition accuracy of this action from each degree. In contrast, by the arrangement of the main acquisition module to the palm at a position adjacent to the wrist, photos are obtained from a fixed angle of view and fixed illumination conditions. For a same gesture, it is unnecessary to obtain photos from different angles within 360*360 degrees to train the recognition system. The amount of data for training can be greatly reduced, and the recognition accuracy can be greatly improved. This manifold processing step is eliminated.
d) Since the camera is arranged at the base of the palm and the maximum distance from the fingers to the camera is thus 15 to 20 cm, the resolution requirement of the image sensor can be reduced and the image data to be processed by the system can also be greatly reduced. In contrast, if the camera is deployed in other perspectives, due to a large distance from the fingers to the camera and the unfixed position, it is required to improve the resolution of the camera to ensure that high-definition images of hand can be obtained at any position after segmentation. In current invention, tracking and auto-focusing is unnecessary, and the time of focusing is reduced. Meanwhile the requirement of FOV angle for the image sensor is also decreased. The calibration step for distortion of lens is also reduced.
e) The maximum distance from the fingers to the camera is 15-20 cm. Therefore, the power consumption for lighting at the minimum intensity required by the camera at 20 cm is greatly reduced. Without the interfering of ambient light, the accuracy for gesture recognition will increase.
3. The recognition is quickened.
The gesture recognition is required to be in real time. It will not be appreciated by users if it takes too much time to recognize a gesture.
It may be found from a) in the above article 2 that it is necessary to segment hand from the images if there is a noisy background, in order to find out the position of the hand. It will always use a neuron network to finish this task. It is time consuming.
It may be found from b) in the above article 2 that scaling the gesture images rotation adjustment also needs to recognize the parameters like scaling ratio and rotation angle. It is more time consuming.
It may be found from c) in the above article 2 that more layers of neurons are required to recognize a finger posture from different angles than from a single angle. These neurons are used for the manifold learning and different illumination conditions. For a same finger posture, if it is recognized within a range of 1 degree, compared with recognition from a single angle, up to 129600 pictures captured from different angles are used for training. If it is recognized from a single angle, only one picture is enough for training. The number of neuron weights and layers can be greatly reduced and the computation is obviously quickened.
It may be found from d) in the above article 2 that the reduction in the resolution of the camera leads to the pixels of the images to be significantly reduced and thus much less pixels to be processed. Compared with the use of a 0.0625 mega-pixel device, the use of a 5 mega-pixel device increases the number of pixels to be processed by almost 80 times. During the processing of convolutional neural networks, one picture is to be traversed by almost 20 kernels in one neural network by a single layer, the performance will be increased by 1600 times. There will be a plurality of layers in the deep neural networks and the performance will be further increased. Large FOV lens are unnecessary, and the calibration computation for lens distortion is also unnecessary. Moreover, autofocus is removed to reduce focus time and motion blur.
4. The power consumption is reduced.
The power consumption of the recognition system is related to the number of floating point operations (multiplication of neural network weights).
It may be found from a) in the above article 2 that the images are segmented by the convolution neural network to filter the noisy background. It will result in high power consumption.
It may be found from b) in the above article 2 that scaling and rotating the gesture images and getting these parameters also increase the number of non-linear weights of the convolutional neural networks, resulting in multifold power consumption.
It may be found from c) in the above article 2 that manifold processing for imagers from a same gesture are reduced. The number ofnon-linear weights ofthe convolutional neural networks and the depth of neural networks are reduced.
It may be found from d) in the above article 2 that pixels of the images are greatly reduced. Compared with the use of a 625 mega-pixel device, the use of a 5 mega-pixel device reduces the number of pixels to be processed in the single-layer convolutional neural network, and the performance will be increased by 1600 times (floating point operations). In the multi-layer deep neural networks, the performance will be multiplied. The power consumption is significantly reduced.
In gesture recognition from images of external environments, for example leapmotion, 80% of operations can be provided by the gesture recognition system itself and the remaining 20% operations are provided by the computer connected thereto via a USB. This method is impossible in wearable devices, because their battery is unable to provide the chips with enough power to obtain the computation results and also because the computation capabilities of the main chip of wearable devices will not be better than that of the computer. Only by the method of the invention, the real-time recognition can be realized in wearable devices.
5. Recognition can be realized anywhere.
When the camera is used in external environments or other perspectives, it is necessary to put the hand within the area where the image sensor is located. Even by the method as employed in hololens, it is also required to raise a person's arm to ensure that the camera on the hamlet can capture the hand. In contrast, by the method of the present invention, the hand may be sensed anywhere.
6. The users are protected against gorilla arms.
Due to the advantage described in the above article 5, it is unnecessary to raise a person's arm for a long period of time and put the hand in the area that can be recognized by the camera. This can avoid soreness of the arm (gorilla arm). This symptom is very common in users of iPad. The users are also protected against mouse hands (carpal tunnel syndrome). Additionally, suffering from the gorilla arms, the users may hold their handheld device at a low position, resulting in cervical spondylosis.
Referring to FIGS. 1, 2, and 8 , a gesture acquisition system is provided, which includes a finger posture acquisition device 1, a palm following device 1000, and a wristband 7. The wristband 7 is arranged on a wrist or a forearm. The palm following device 1000 is arranged on a hand or the wrist and is configured to fix the main acquisition module 11 on a palm of the hand. The palm following device 1000 includes a fixing component and a wrist posture acquisition device 2. The fixing component is configured to fix the main acquisition module 11 on the palm of the hand and to synchronously swing with the palm. The wrist posture acquisition device 2 is connected to the wristband 7. An end of the wrist posture acquisition device 2 away from the wristband is connected to the fixing component. Specifically, the palm following device 1000 has contact on edges of the ulna or radius, and on the palm and the back hand. When the palm does an adduction-abduction motion, the palm following device 1000 is stressed at the edges of the ulna or radius and simultaneously moves with the palm. When the palm makes a flexion-extension motion, the palm following device 1000 is subjected to a force at the palm or at the back of the hand, and simultaneously moves with the palm. The above-mentioned forces in the opposite direction can also be provided by torsion springs, ensuring that the force is applied in just one place. The palm used herein means the palm side of the hand.
In an embodiment, the fixing component is a sticking part, and the main acquisition module 11 is sticked to the palm of the hand by the sticking part. Specifically, the main acquisition module 11 is sticked to the palm of the hand by a double-sided adhesive or other adhesive, thus enabling the main acquisition module 11 to be fixed relative to the palm of the hand.
In an embodiment, referring to FIG. 11 , the fixing component is a C-shaped ring 100. The C-shaped ring 100 is configured to sleeve the palm of the hand. The C-shaped ring 100 is provided with a clamping position 130. The clamping position 130 is arranged adjacent to the palm of the hand and is configured to clamp the main acquisition module 11. In this embodiment, the C-shaped ring 100 is connected to the end of the wrist posture acquisition device 2 away from the wristband 7, and the C-shaped ring 100 is connected to the wristband 7 by the wrist posture acquisition device 2, so as to avoid the C-shaped ring 100 falling from the palm of the hand. In other embodiments, a connection structure may be provided to connect the C-shaped ring 100 to the wristband 7 or to the wrist of the user.
In an embodiment, referring to FIG. 11 , the clamping position 130 is located in the middle of the fixation structure 8.
In an embodiment, referring to FIG. 11 , the C-shaped ring includes a first clamping portion 110 and a second clamping portion 120. The first clamping portion 110 is rotatably connected with the second clamping portion 120. The clamping position 130 is located where the first clamping portion 110 and the second clamping portion 120 are connected. A torsion spring 140 is provided at the clamping position 130. The torsion spring 140 is configured to apply an elastic force opposite to a moving direction of the palm to the first clamping portion 110 and the second clamping portion such that the first clamping portion 110 and the second clamping portion 120 can clamp the palm. Specifically, when the C-shaped ring 100 is sleeved on the palm of the hand, the first clamping portion 110 and the second clamping portion 120 can clamp the palm, thus ensuring that the C-shaped ring 100 can reliably drive the wrist posture acquisition arm 22 making adduction-abduction and flexion-extension motions with the wrist.
In an embodiment, referring to FIG. 8 , the fixing component is a C-shaped ring 100. The C-shaped ring 100 is configured to be sleeved on a side of the hand such that a first end of the C-shaped ring 100 is located at the palm of the hand, and a second end of the C-shaped ring 100 is located at a back of the hand. The first end of the C-shaped ring 100 is configured for arrangement of the main acquisition module 11. In this embodiment, the C-shaped ring 100 is connected to the end of the wrist posture acquisition device 2 away from the wristband 7, and the C-shaped ring 100 is connected to the wristband 7 by the wrist posture acquisition device 2, so as to avoid the C-shaped ring 100 falling from the palm of the hand. In other embodiments, a connection structure may be provided to connect the C-shaped ring 100 to the wristband 7 or to the wrist of the user.
In an embodiment, the wrist posture acquisition device 2 is capable of synchronously swinging with the wrist to collect wrist postures.
In an embodiment, the wrist posture acquisition device 2 includes a wrist posture acquisition arm 22 and a sensing module 24 arranged on the wrist posture acquisition arm 22. The wrist posture acquisition arm 22 is connected to the wristband 7 and is configured to synchronously swing with the wrist. The sensing module 24 is configured to collect posture information of the wrist posture acquisition arm 22, so as to acquire the wrist postures. Specifically, the wristband 7 is placed on the wrist of the hand, and the end of the wrist posture acquisition arm 22 away from the wristband 7 is in contact with the palm of the hand. In this way, the hand will drive the wrist posture acquisition arm 22 to do an adduction-abduction motion or flexion-extension motion. During this motion, the sensing module 24 collects the adduction-abduction information and the flexion-extension information of the wrist posture acquisition arm 22, so as to obtain the adduction-abduction postures and flexion-extension postures of the wrist. In particular, the wrist postures include adduction-abduction postures and flexion-extension postures, and the wrist posture acquisition arm 22 is configured to do an adduction-abduction motion or flexion-extension motion together with the wrist.
In an embodiment, the wrist posture acquisition arm 22 is made of a rigid material, and the sensing module 24 is an angle sensor.
In an embodiment, the wrist posture acquisition arm 22 is made of a flexible material. The sensing module 24 is a resistive flexion angle sensor. The resistive flexion angle sensor is provided along an axis of the wrist posture acquisition arm 22. A length of the resistive flexion angle sensor is the same as a length of the wrist posture acquisition arm 22. The sensing module 24 is configured to collect the posture information of the wrist posture acquisition arm 22. Specifically, the resistive flexion angle sensor can bend and rotate with the wrist posture acquisition arm 22 such that it can collect the adduction-abduction information and the flexion-extension information of each part of the wrist posture acquisition arm 22, and then calculate the adduction-abduction postures and flexion-extension postures of the wrist based on that information. In this embodiment, the length of the resistive flexion angle sensor is the same as the length of the wrist posture acquisition arm 22, which ensures that the resistive flexion angle sensor can detect the adduction-abduction information and the flexion-extension information of each part of the wrist posture acquisition arm 22.
In an embodiment, the adduction-abduction angle sensor 240 is arranged on the adduction-abduction arm 220 and is configured to acquire an adduction-abduction angle of the adduction-abduction arm 220, so as to acquire adduction-abduction information of the adduction-abduction arm 220 and thus obtain adduction-abduction postures of the wrist. The flexion-extension angle sensor 242 is arranged on the flexion-extension arm 222 and is configured to acquire a flexion-extension angle of the flexion-extension arm 222, so as to acquire flexion-extension information of the flexion-extension arm 222 and thus obtain flexion-extension postures of the wrist.
In an embodiment, referring to FIGS. 8-9 , the adduction-abduction arm 220 is hingedly connected to the flexion-extension arm 222, and an end of the adduction-abduction arm 220 or the flexion-extension arm 222 away from the adduction-abduction arm is articulatedly connected with the wristband 7. The adduction-abduction arm 220 is configured to make the adduction-abduction motion with the wrist around an adduction-abduction axis. The flexion-extension arm 222 is configured to make the flexion-extension motion with the wrist around a flexion-extension axis. The adduction-abduction axis and the flexion-extension axis are both perpendicular to an axis of the wristband 7, and the adduction-abduction axis is perpendicular to the flexion-extension axis.
In an embodiment, referring to FIG. 8 , each of the adduction-abduction arm 220 and the flexion-extension arm 222 has a first end and a second end opposite to the first end. The first end of the adduction-abduction arm 220 is articulatedly connected with the wristband 7, and the adduction-abduction angle sensor 240 is arranged where the first end of the adduction-abduction arm 220 is articulatedly connected with the wristband 7. The first end of the flexion-extension arm 222 is articulated with the second end of the adduction-abduction arm 220, and the flexion-extension angle sensor 242 is arranged where the flexion-extension arm 222 is articulatedly connected with the adduction-abduction arm 220. The wristband 7 is arranged on the wrist or the forearm, the second end of the flexion-extension arm 222 is adjacent to the palm of the hand, and the second end of the flexion-extension arm 222 is adjacent to the palm of the hand. The C-shaped ring is arranged on the second end of the flexion-extension arm 222. In this embodiment, the first end of the adduction-abduction arm 220 is articulatedly connected with the wristband 7 through a first articulating shaft. An axis of the first articulating shaft 221 is coaxial with the adduction-abduction axis. The first end of the flexion-extension arm 222 is articulatedly connected with the second end of the adduction-abduction arm 220 through a second articulating shaft 223. An axis of the second articulating shaft 223 is coaxial with the flexion-extension axis. Specifically, during the flexion-extension motion of the wrist, the palm of the hand will drive the flexion-extension arm 222 to rotate around the flexion-extension axis relative to the adduction-abduction arm 220, and during this process, the flexion-extension angle sensor 242 collects the flexion-extension information of the flexion-extension arm 222 to obtain the flexion-extension postures of the wrist. Since the rotary axis between the flexion-extension arm 222 and the adduction-abduction arm 220 is the flexion-extension axis, when the flexion-extension arm 222 rotates around the flexion-extension axis, it does not drive the adduction-abduction arm 220 to rotate. During the adduction-abduction motion of the wrist, the palm of the hand will drive the flexion-extension arm 222 and the adduction-abduction arm 220 to synchronously rotate around the adduction-abduction axis relative to the wristband 7, and during this process, the adduction-abduction angle sensor 240 collects the adduction-abduction information of the adduction-abduction arm 220 to obtain the adduction-abduction postures of the wrist. In this embodiment, the end of the flexion-extension arm 222 away from the adduction-abduction arm 220 is connected to the C-shaped ring 100, so that during the flexion-extension motion of the wrist, the palm of the hand will drive the flexion-extension arm 222 to rotate around the flexion-extension axis relative to the adduction-abduction arm 220 through the C-shaped ring 100. During the adduction-abduction motion of the wrist, the palm of the hand will drive the flexion-extension arm 222 and the adduction-abduction arm 220 to rotate around the adduction-abduction axis relative to the wristband 7 through the C-shaped ring 100. Specifically, the first end refers to the aforementioned position fixed end, and the second end refers to the aforementioned free end.
In an embodiment, referring to FIGS. 8-9 , when the first end of the adduction-abduction arm 220 is articulatedly connected with the wristband 7, the adduction-abduction arm 220 includes a first arm 2201 and a second arm 2202. The flexion-extension arm 222 includes a third arm 2221 and a fourth arm 2222. Each of the first arm 2201, second arm 2202, the third arm 2221, and the fourth arm 2222 has a first end and a second end opposite to each other. The first end of the first arm 2201 is connected to the first end of the second arm 2202. A connection between the first end of the first arm 2201 and the first end of the second arm 2202 is articulatedly connected with the wristband 7, and the adduction-abduction angle sensor 240 is arranged at the connection between the first end of the first arm 2201 and the first end of the second arm 2202. The first end of the third arm 2221 and the second end of the first arm 2201 are articulated at point A. The first end of the fourth arm 2222 and the second end of the second arm 2202 are articulated at point B, and the flexion-extension angle sensor 242 is arranged at point A or point B. The second end of the third arm 2221 is connected to the second end of the fourth arm 2222. In this embodiment, the adduction-abduction arm 220 and the flexion-extension arm 222 are both dual-arm structured, so that the connection between the wristband 7, the adduction-abduction arm 220, and the flexion-extension arm 222 is more stable and reliable. In other embodiments, referring to FIG. 8 , the adduction-abduction arm 220 and the flexion-extension arm 222 are both single-arm structured. Specifically, during the adduction-abduction motion of the wrist, the palm of the hand will drive the adduction-abduction arm 220 to rotate around the adduction-abduction axis relative to the flexion-extension arm 222, and during this process, the adduction-abduction angle sensor 240 collects the adduction-abduction information of the adduction-abduction arm 220 to obtain the adduction-abduction postures of the wrist. Since the rotary axis between the adduction-abduction arm 220 and the flexion-extension arm 222 is the adduction-abduction axis, when the adduction-abduction arm 220 rotates around the adduction-abduction axis, it does not drive the flexion-extension arm 222 to rotate. During the flexion-extension motion of the wrist, the palm of the hand will drive the adduction-abduction arm 220 and the flexion-extension arm 222 to synchronously rotate around the flexion-extension axis relative to the wristband 7, and during this process, the flexion-extension angle sensor 242 collects the flexion-extension information of the flexion-extension arm 222 to obtain the flexion-extension postures of the wrist. In this embodiment, the end of the adduction-abduction arm 220 away from the flexion-extension arm 222 is connected to the C-shaped ring 100, so that during the adduction-abduction motion of the wrist, the palm of the hand will drive the adduction-abduction arm 220 to rotate around the flexion-extension axis relative to the flexion-extension arm 222 through the C-shaped ring 100. During the flexion-extension motion of the wrist, the palm of the hand will drive the adduction-abduction arm 220 and the flexion-extension arm 222 to rotate around the flexion-extension axis relative to the wristband 7 through the C-shaped ring 100.
In an embodiment, referring to FIG. 9 , each of the adduction-abduction arm 220 and the flexion-extension arm 222 has a first end and a second end opposite to each other. The first end of the flexion-extension arm 222 is articulated with the wristband 7, and the flexion-extension angle sensor 242 is arranged at a position where the first end of the flexion-extension arm 222 and the wristband 7 are articulated. The first end of the adduction-abduction arm 220 is articulatedly connected with the second end of the flexion-extension arm 222, and the adduction-abduction angle sensor 240 is arranged at the connection between the adduction-abduction arm 220 and the flexion-extension arm 222. When the wristband 7 is arranged on the wrist or the forearm, the second end of the adduction-abduction arm 220 is adjacent to the palm of the hand, and the C-shaped ring 100 is arranged on the second end of the adduction-abduction arm 220. Specifically, the first end of the flexion-extension arm 222 is articulatedly connected with the wristband 7 through a second articulating shaft 223, and an axis of the second articulating shaft 223 is coaxial with the flexion-extension axis. The first end of the adduction-abduction arm 220 is articulatedly connected with the second end of the flexion-extension arm 222 through a first articulating shaft 221, and an axis of the first articulating shaft 221 is coaxial with the flexion-extension axis. Specifically, the first end refers to the aforementioned position fixed end, and the second end refers to the aforementioned free end.
In an embodiment, referring to FIGS. 8-9 , when the first end of the flexion-extension arm 222 is articulatedly connected with the wristband 7, the adduction-abduction arm 220 includes a first arm 2201 and a second arm 2202. The flexion-extension arm 222 includes a third arm 2221 and a fourth arm 2222. Each of the first arm 2201, second arm 2202, the third arm 2221, and the fourth arm 2222 has a first end and a second end opposite to each other. The first end of the third arm 2221 is connected to the first end of the fourth arm 2222. A connection between the first end of the third arm 2221 and the first end of the fourth arm 2222 is articulatedly connected with the wristband 7, and the flexion-extension angle sensor 242 is arranged at the connection between the first end of the third arm 2221 and the first end of the fourth arm 2222. The first end of the first arm 2201 and the second end of the third arm 2221 are articulated at point A. The first end of the second arm 2202 and the second end of the fourth arm 2222 are articulated at point B, and the adduction-abduction angle sensor 240 is arranged at point A or point B. The second end of the first arm 2201 is connected to the second end of the second arm 2202. Specifically, the adduction-abduction arm 220 and the flexion-extension arm 222 are both dual-arm structured, so that the connection between the wristband 7, the adduction-abduction arm 220, and the flexion-extension arm 222 is more stable and reliable.
Specifically, when the gesture acquisition system is off duty, the C-shaped ring 100 can be pushed in the direction close to the wristband 7 such that the C-shaped ring 100, the adduction-abduction arm 220 and the flexion-extension arm 222 are close to the wristband 7, thus enabling the C-shaped ring 100 to be retracted to reduce the space occupied by the gesture acquisition system. Furthermore, whether the adduction-abduction arm 220 and the flexion-extension arm 222 are dual-arm structured or single-arm structured, the C-ring 100 can be retracted.
In an embodiment, referring to FIG. 10 , when the wrist posture acquisition arm 22 is a telescopic pole, the sensing module 24 is a second angle sensor. The second angle sensor is arranged at a connection between the wrist posture acquisition arm 22 and the wristband 7. The second angle sensor is configured to obtain the posture information of the wrist posture acquisition arm 22 to obtain the wrist postures. Specifically, the second angle sensor is a 2D Hall sensor, and the Hall sensor is provided on the wristband 7. A magnet 26 is arranged at the end of the wrist posture acquisition arm 22 connecting to the wristband 7, and the magnet 26 is accommodated within the Hall sensor to achieve the connection between the wrist posture acquisition arm 22 and the wristband 7. The Hall sensor obtains the flexion-extension information and the adduction-abduction information of the wrist posture acquisition arm 22 by acquiring the magnetic field information of the magnet.
In an embodiment, referring to FIG. 11 , the end of the wrist posture acquisition arm 22 away from the wristband 7 is connected to the C-shaped ring 100, and a second angle sensor is also mounted at where the C-shaped ring 100 and the wrist posture acquisition arm 22 are connected. The two second angle sensors simultaneously obtain the posture information of the wrist posture acquisition arm 22 to obtain the wrist postures. Specifically, the C-shaped ring 100 is also provided with a 2D Hall. The end of the C-shaped ring 100 is also provided with a magnet 26, and the magnet 26 is accommodated within the Hall sensor to achieve the connection between the wrist posture acquisition arm 22 and the C-shaped ring 100. The two Hall sensor obtain the flexion-extension information and the adduction-abduction information of the wrist posture acquisition arm 22 by acquiring the magnetic field information of the magnet.
In an embodiment, the main acquisition module 11 is also configured to acquire ambient images. The wristband 7 is provided with an image sensor 200, and the image sensor 200 is configured to acquire ambient images. Based on the comparison information between the ambient images respectively acquired by the main acquisition module 11 and the image sensor 200, the flexion-extension postures of the wrist are obtained.
In an embodiment, referring to FIGS. 10-11 , the C-shaped ring 100 is provided with a regular triangular marker 300. The wristband 7 is provided with an image sensor 200, and the image sensor 200 is configured to acquire the change in shape of the regular triangular marker 300 during the adduction-abduction motion or the flexion-extension motion of the wrist, so as to calculate and obtain the degree of flexion of the wrist, and thus obtain the wrist postures. Specifically, the regular triangular mark 300 is a regular triangle in the image sensor 200 when the wrist is not bent, and the regular triangular mark 300 is no longer a regular triangle in the image sensor 200 after the wrist is bent. The regular triangular mark 300 appears in different shapes with the movement of the wrist, and the degree of bending of the wrist can be calculated based on the change in shape. Specifically, the wrist posture acquisition arm 22 is made of a rigid material or a flexible material. The C-shaped ring 100 is provided with a regular triangular mark 300, and an image sensor 200 is arranged on the wristband 7.
The foregoing descriptions are merely preferred embodiments of the present invention, and not intended to limit the present invention. Any modifications, equivalent replacements, and improvements without departing from the spirit and principle of the present invention should be within the protection scope of the present invention.

Claims

What is claimed is:

1. A gesture acquisition system, comprising:

a finger posture acquisition device;

a palm following device; and

a wristband;

wherein the wristband is arranged on a wrist or a forearm; the finger posture acquisition device comprises a main acquisition module; the palm following device is arranged on a hand or the wrist; the palm following device is configured to fix the main acquisition module on a palm of the hand; the main acquisition module is capable of synchronously swinging with the palm through the palm following device; the main acquisition module is configured to acquire finger postures; the palm following device comprises a fixing component and a wrist posture acquisition device; the fixing component is configured to fix the main acquisition module on the palm of the hand, and to synchronously swing with the palm; the wrist posture acquisition device is connected to the wristband; and an end of the wrist posture acquisition device away from the wristband is connected to the fixing component.

2. The gesture acquisition system of claim 1, wherein the fixing component is a sticking part or an attachment; and the main acquisition module is sticked to the palm of the hand by the sticking part or attached to the palm of the hand by the attachment.

3. The gesture acquisition system of claim 1, wherein the fixing component is a C-shaped ring; the C-shaped ring is configured to be sleeved on the palm of the hand; the C-shaped ring is provided with a clamping position; the clamping position is arranged adjacent to the palm of the hand; and the clamping position is configured as a position where the main acquisition module is clamped.

4. The gesture acquisition system of claim 3, wherein the C-shaped ring comprises a first clamping portion and a second clamping portion; the first clamping portion is rotatably connected with the second clamping portion; the clamping position is located where the first clamping portion and the second clamping portion are connected; a torsion spring is provided at the clamping position; and the torsion spring is configured to apply an elastic force opposite to a moving direction of the palm to the first clamping portion and the second clamping portion such that the first clamping portion and the second clamping portion clamp the palm.

5. The gesture acquisition system of claim 1, wherein the fixing component is a C-shaped ring; the C-shaped ring is configured to be sleeved on a side of the hand such that a first end of the C-shaped ring is located at the palm of the hand, and a second end of the C-shaped ring is located at a back of the hand; and the first end of the C-shaped ring is configured for arrangement of the main acquisition module.

6. The gesture acquisition system of claim 1, wherein the wrist posture acquisition device is configured to synchronously swing with the wrist to collect wrist postures.

7. The gesture acquisition system of claim 6, wherein the wrist posture acquisition device comprises a wrist posture acquisition arm and a sensing module arranged on the wrist posture acquisition arm and the wristband; the wrist posture acquisition arm is connected to the wristband, and is configured to synchronously swing with the wrist; and the sensing module is configured to collect posture information of the wrist posture acquisition arm with respect to the wristband, so as to acquire the wrist postures.

8. The gesture acquisition system of claim 7, wherein the wrist posture acquisition arm is made of a rigid material; and the sensing module is an angle sensor.

9. The gesture acquisition system of claim 7, wherein the wrist posture acquisition arm is made of a flexible material; the sensing module is a resistive or capacitive flexion angle sensor; the resistive or capacitive flexion angle sensor is provided along an axis of the wrist posture acquisition arm; and a length of the resistive flexion angle sensor is the same as a length of the wrist posture acquisition arm.

10. The gesture acquisition system of claim 8, wherein the wrist posture acquisition arm comprises an adduction-abduction arm that makes an adduction-abduction motion together with the wrist and a flexion-extension arm that makes a flexion-extension motion together with the wrist.

11. The gesture acquisition system of claim 10, wherein the adduction-abduction arm is articulatedly connected to the flexion-extension arm; an end of the adduction-abduction arm away from the flexion-extension arm or an end of the flexion-extension arm away from the adduction-abduction arm is articulatedly connected with the wristband; the adduction-abduction arm is configured to make the adduction-abduction motion with the wrist around an adduction-abduction axis; the flexion-extension arm is configured to make the flexion-extension motion with the wrist around a flexion-extension axis; the adduction-abduction axis and the flexion-extension axis are both perpendicular to an axis of the wristband; and the adduction-abduction axis is perpendicular to the flexion-extension axis.

12. The gesture acquisition system of claim 11, wherein the angle sensor comprises an adduction-abduction angle sensor and a flexion-extension angle sensor; each of the adduction-abduction arm and the flexion-extension arm has a first end and a second end opposite to the first end; the first end of the adduction-abduction arm is articulatedly connected with the wristband, and the adduction-abduction angle sensor is arranged at the adduction-abduction arm or the wristband; the first end of the flexion-extension arm is articulatedly connected with the second end of the adduction-abduction arm, and the flexion-extension angle sensor is arranged at the flexion-extension arm or the adduction-abduction arm; and when the wristband is arranged on the wrist or the forearm, the second end of the flexion-extension arm is adjacent to the palm of the hand, and the fixing component is arranged on the second end of the flexion-extension arm.

13. The gesture acquisition system of claim 12, wherein the first end of the adduction-abduction arm is articulatedly connected with the wristband through a first articulating shaft; an axis of the first articulating shaft is coaxial with the adduction-abduction axis; the first end of the flexion-extension arm is articulatedly connected with the second end of the adduction-abduction arm through a second articulating shaft; and an axis of the second articulating shaft is coaxial with the flexion-extension axis.

14. The gesture acquisition system of claim 11, wherein the angle sensor comprises an adduction-abduction angle sensor and a flexion-extension angle sensor; each of the adduction-abduction arm and the flexion-extension arm has a first end and a second end opposite to the first end; the first end of the flexion-extension arm is articulatedly connected with the wristband, and the flexion-extension angle sensor is arranged at the flexion-extension arm or the wristband; the first end of the adduction-abduction arm is articulatedly connected with the second end of the flexion-extension arm, and the adduction-abduction angle sensor is arranged at the adduction-abduction arm or the flexion-extension arm; and when the wristband is arranged on the wrist or the forearm, the second end of the adduction-abduction arm is adjacent to the palm of the hand, and the fixing component is arranged on the second end of the adduction-abduction arm.

15. The gesture acquisition system of claim 14, wherein the first end of the adduction-abduction arm is articulatedly connected with the second end of the flexion-extension arm through a first articulating shaft; the first end of the flexion-extension arm is articulatedly connected with the wristband through a second articulating shaft; an axis of the second articulating shaft is coaxial with the flexion-extension axis; and an axis of the first articulating shaft is coaxial with the adduction-abduction axis.

16. The gesture acquisition system of claim 12, wherein the adduction-abduction arm comprises a first arm and a second arm; the flexion-extension arm comprises a third arm and a fourth arm; each of the first arm, the second arm, the third arm, and the fourth arm has a first end and a second end opposite to each other; the first end of the first arm is connected to the first end of the second arm; a connection between the first end of the first arm and the first end of the second arm is articulatedly connected with the wristband, and the adduction-abduction angle sensor is arranged at the connection between the first end of the first arm and the first end of the second arm; the first end of the third arm and the second end of the first arm are articulated at a first point; the first end of the fourth arm and the second end of the second arm are articulated at a second point; the flexion-extension angle sensor is arranged at the second point or the first point; and the second end of the third arm is connected to the second end of the fourth arm.

17. The gesture acquisition system of claim 14, wherein the adduction-abduction arm comprises a first arm and a second arm; the flexion-extension arm comprises a third arm and a fourth arm; each of the first arm, the second arm, the third arm, and the fourth arm has a first end and a second end opposite to each other; the first end of the third arm is connected to the first end of the fourth arm; a connection between the first end of the third arm and the first end of the fourth arm is articulatedly connected with the wristband, and the flexion-extension angle sensor is arranged at the connection between the first end of the third arm and the first end of the fourth arm; the first end of the first arm and the second end of the third arm are articulated at a first point; the first end of the second arm and the second end of the fourth arm are articulated at a second point; the adduction-abduction angle sensor is arranged at the second point or the first point; and the second end of the first arm is connected to the second end of the second arm.

18. The gesture acquisition system of claim 7, wherein the wrist posture acquisition arm is a telescopic pole; the wrist posture acquisition arm comprises a first end located above the wristband or at a side of the wristband and a second end fixedly connected to the hand; a connection between the wrist posture acquisition arm and the wristband is located between the first end and the second end; the first end is configured to swing around the connection between the wrist posture acquisition arm and the wristband; and the sensing module is configured to acquire swing information of the first end to acquire the posture information of the wrist posture acquisition arm, so as to obtain the wrist postures.

19. The gesture acquisition system of claim 18, wherein the sensing module comprises a first angle sensor; the first angle sensor is arranged at the connection between the wrist posture acquisition arm and the wristband; and the first angle sensor is configured to acquire the posture information of the wrist posture acquisition arm, so as to acquire the wrist postures.

20. The gesture acquisition system of claim 19, wherein the fixing component is a C-shaped ring; the C-shaped ring is configured to be sleeved on the palm of the hand; the main acquisition module is arranged on the C-shaped ring; an end of the wrist posture acquisition arm away from the wristband is connected to the C-shaped ring; the sensing module further comprises a second angle sensor; the second angle sensor is arranged at a position where the C-shaped ring and the wrist posture acquisition arm are connected; and the first angle sensor and the second angle sensor are configured to simultaneously acquire the posture information of the wrist posture acquisition arm, so as to acquire the wrist postures. A gesture acquisition system, including: a finger posture acquisition device, a palm following device, and a wristband. The finger posture acquisition device includes a main acquisition module. The palm following device is arranged on a hand or the wrist and is configured to fix the main acquisition module on a palm. The main acquisition module is capable of synchronously swinging with the palm through the palm following device, and is configured to acquire finger postures. The palm following device includes a fixing component and a wrist posture acquisition device. The fixing component is configured to fix the main acquisition module on the palm, and synchronously swings with the palm. The wrist posture acquisition device is connected to the wristband. An end of the wrist posture acquisition device away from the wristband is connected to the fixing component.