WO2019028650A1 - 手势采集系统 - Google Patents

手势采集系统 Download PDF

Info

Publication number
WO2019028650A1
WO2019028650A1 PCT/CN2017/096362 CN2017096362W WO2019028650A1 WO 2019028650 A1 WO2019028650 A1 WO 2019028650A1 CN 2017096362 W CN2017096362 W CN 2017096362W WO 2019028650 A1 WO2019028650 A1 WO 2019028650A1
Authority
WO
WIPO (PCT)
Prior art keywords
wrist
arm
flexion
posture
gesture
Prior art date
Application number
PCT/CN2017/096362
Other languages
English (en)
French (fr)
Inventor
方超
Original Assignee
方超
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 方超 filed Critical 方超
Priority to EP17920688.3A priority Critical patent/EP3667564A4/en
Priority to PCT/CN2017/096362 priority patent/WO2019028650A1/zh
Publication of WO2019028650A1 publication Critical patent/WO2019028650A1/zh
Priority to US16/785,552 priority patent/US11507192B2/en
Priority to US17/967,043 priority patent/US20230195234A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
    • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
    • H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
    • H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones

Definitions

  • the present invention relates to the field of human-computer interaction technologies, and in particular, to a gesture collection system.
  • Gesture is an intuitive and natural way of interaction. It is fast and expressive, and it is an important tool for humans to exchange information.
  • gesture recognition is performed according to certain rules, the content of the gesture expression is recognized, which has strong consistency and expandability.
  • the gesture recognition system can directly manipulate software or virtual objects in the application software. However, how to identify gestures more effectively and reach a better state of human-computer interaction is a hot and difficult issue.
  • An object of the present invention is to provide a gesture collection system capable of accurately acquiring finger gesture information and having less recognition time.
  • a gesture collection system including a finger gesture collection device, the finger gesture collection device includes a main collection module located below the palm and swinging synchronously with the palm, the main acquisition The module is used to capture finger gestures.
  • the invention further includes a wrist posture collecting device for collecting a telescopic posture of the wrist and a flexing and extending posture of the wrist, the wrist posture collecting device comprising a wrist posture collecting arm that is extended and flexed and extended with the wrist portion and A sensing module for collecting flexion and extension information and information of the wrist posture acquisition arm.
  • a wristband is further disposed on the wrist, the wrist posture collecting arm is coupled to the wristband, and the wrist posture collecting arm is extended with the wrist through the wristband Flexion and extension.
  • the wrist posture collection arm includes a telescopic arm that is extended with the wrist portion and a flexion and extension arm that flexes and extends with the wrist portion
  • the sensing module includes two angle sensors, and the two angle sensors are respectively A telescopic angle sensor for acquiring a telescopic angle of the telescopic arm and a flexion and extension angle sensor for acquiring a flexion and extension angle of the flexion and extension arm.
  • the wristband and the telescopic shaft of the wrist have two intersection points corresponding to the telescopic arm, and the wristband and the flexion and extension axis of the wrist have two corresponding flexion and extension arms.
  • An intersection point, the telescopic arm and the flexion and extension arm each include a fixed end and a free end opposite to the fixed end, and a fixed end of one of the telescopic arm and the flexion and extension arm is mounted on the wrist Bringing and located at one of the corresponding two intersections and mounting its corresponding angle sensor, the free end being wound along the wristband to the corresponding two of the other of the telescopic arm and the flexion and extension arm
  • the intersection is connected to a fixed end of the other of the telescopic arm and the flexion and extension arm
  • the telescopic arm has a certain angle with the flexion and extension arm connection point and another angle sensor is mounted
  • the free end of the other of the telescopic arm and the flexion and extension arm is disposed about the wristband and extend
  • the flexion and extension angle sensor is disposed along the flexion and extension axis
  • the telescopic angle sensor is disposed along the telescopic axis.
  • a free end of the telescopic arm and the flexion and extension arm at a position of a close hand of the wrist is wound around the bottom of the position of the hand of the wrist and the main acquisition module is mounted.
  • the wrist posture collecting arm is a telescopic rod
  • the wrist posture collecting arm is connected to the wristband, and includes a first end located above or laterally of the wristband and fixed to the hand a second end, the intersection of the wrist posture collecting arm and the wristband is located between the first end and the second end, the first end is capable of swinging around the intersection, the sensing
  • the module acquires the flexion and extension information and the extension information of the wrist posture acquisition arm by acquiring the swing information of the first end.
  • the wristband is provided with a fisheye bearing
  • the wrist posture collecting arm is connected to the wrist posture collecting arm by inserting the fisheye bearing.
  • the sensing module is a Hall sensor
  • the first end of the wrist posture collecting arm is There is a magnet
  • the sensing module acquires the flexion and extension information and the extension information of the wrist posture acquisition arm by acquiring the magnetic field information of the first end.
  • the sensing module is located in an extending direction of the wrist posture collecting arm, and is farther away from the second end of the wrist posture collecting arm than the first end of the wrist posture collecting arm.
  • the method further includes a fixing structure fixed at a position of the wrist near hand, the second end of the wrist posture collecting arm being fixed on the fixing structure, and the main collecting module being fixed at the fixing Structurally.
  • a forearm posture acquisition device mounted on the wristband is further included, and the forearm posture acquisition device is configured to acquire a forearm posture.
  • the forearm attitude acquisition device comprises a three-axis gyroscope, a three-axis magnetometer and a three-axis acceleration sensor.
  • the wristband is sleeve-shaped, and the wristband is spaced apart from the first fixing portion and the second fixing portion, and the first fixing portion and the second fixing portion respectively correspond to the styloid process of the humerus and the ulnar styloid
  • the body surface projection is adapted to secure the wrist strap to the wrist.
  • first fixing portion and the second fixing portion are through holes or grooves formed on the wristband.
  • system further includes an environment information collecting device for searching for an electronic device that needs to be controlled in the surrounding environment.
  • a processor is further included, the processor being coupled to other electronic components in the gesture collection system.
  • the main acquisition module is a camera.
  • the installation area of the main collection module is within 0 to 5 cm of the wrist joint to the finger direction, within 0 to 3 cm of the wrist to the arm, and within 0 to 3 cm of the skin of the body surface.
  • the present invention has the following beneficial effects:
  • the gesture collection system of the present invention sets the main acquisition module of the finger gesture collection device under the palm and swings synchronously with the palm, so that the angle of view and position of the main acquisition module are fixed relative to the palm, so that the same gesture action can only be used from a fixed one.
  • Angle shot images to identify, reduce recognition time, mention High recognition accuracy also omits background deletion calculation consumption.
  • FIG. 1 is a circuit block diagram of a gesture collection system of the present invention
  • FIG. 2 is a schematic perspective structural view of a first embodiment of a gesture collection system according to an embodiment of the present invention
  • Figure 3 is a schematic view of Figure 2;
  • Figure 4 is a schematic view of another embodiment of the present invention.
  • FIG. 5 is a schematic perspective structural diagram of a second embodiment of a gesture collection system according to an embodiment of the present invention.
  • Figure 6 is an enlargement of the portion VI of Figure 5.
  • a gesture collection system includes a finger gesture collection device 1 , which includes a main acquisition module 11 and an auxiliary acquisition module (not shown), and the main acquisition module 11 is disposed in FIG.
  • the main acquisition module 11 can be used to collect finger gesture information, and the main acquisition module 11 can be, for example, a camera.
  • the main acquisition module 11 collects finger gesture information by acquiring a finger image.
  • the auxiliary acquisition module is also used to assist in collecting finger posture information, and the auxiliary acquisition module can be located at any position of the hand that can collect the finger gesture to achieve auxiliary acquisition of the finger posture information.
  • the auxiliary acquisition module can be, for example, a camera, and assists in acquiring finger posture information by acquiring a finger image.
  • the gesture collection system not only needs to collect the finger gesture, but also needs to collect the wrist posture and the forearm posture.
  • the gesture collection system of the present invention further includes, for example, a wrist posture acquisition device 2 for collecting the posture information of the wrist and the flexion and extension posture information of the wrist. .
  • the invention collects the posture information of the wrist and the wrist through the wrist posture collecting device 2
  • the flexion and extension posture information of the department realizes the collection of the posture of the wrist.
  • the wrist posture acquisition device 2 of the present invention can collect the movement and flexion and extension of the wrist to obtain the posture information of the wrist and the flexion and extension posture information of the wrist.
  • the wrist posture acquisition device 2 includes a wrist posture collection arm 22 that is extended and flexed with the wrist, and the sensing module 24 collects the wrist posture collection.
  • the flexion and extension information and the extension information of the arm 22 enable the collection of posture information of the wrist.
  • the sensing module 24 can accurately collect the flexion and extension information and the extension information of the wrist posture acquisition arm 22, and the wrist posture collection arm 22 is usually made of hard
  • the material is made so that the wrist posture collecting arm 22 is not easily deformed, and the bending and extension of the wrist posture collecting arm 22 caused by the easy deformation of the wrist posture collecting arm 22 can be avoided, so as to finally affect the collection accuracy. The problem.
  • the gesture collection system of the present invention further includes, for example, a forearm posture acquisition device 3 for collecting forearm posture information.
  • the forearm attitude acquisition device 3 includes a three-axis gyroscope, a three-axis magnetometer, and a three-axis acceleration sensor, wherein the three-axis gyroscope is used to detect the three-axis angular velocity of the forearm, and the three-axis acceleration sensor is used for The three-axis acceleration of the forearm is detected.
  • the three-axis magnetometer is used to detect the three-axis magnetic force of the forearm, so that the detected forearm posture information is more accurate to obtain accurate forearm posture information.
  • the gesture collection system of the present invention is also capable of determining an electronic device that needs to be controlled.
  • the gesture collection system of the present invention further includes, for example, an environment information collection device 4 for finding an electronic device in the surrounding environment that needs to control interaction (computer , flat panel, household appliances, IOT, in-vehicle equipment, etc.) and achieve positioning of the arm relative to the local peripheral environment of the indoor or outdoor.
  • the environmental information collecting device 4 is an environmental information camera.
  • the forearm attitude acquisition device 3 further includes a reflective spot disposed on the wristband, and the environmental information collection device 4 can collect the position information of the reflective spot.
  • the gesture collection system of the present invention further includes a processor 5 and a wireless communication module 6, a finger posture acquisition device 1, a wrist posture acquisition device 2, a forearm posture acquisition device 3, and a ring.
  • the environment information collecting device 4 and the wireless communication module 6 are both connected to the processor 5, the finger posture information collected by the finger posture acquiring device 1, the wrist posture information of the wrist posture acquiring device 2, and the forearm posture information collected by the forearm posture collecting device 3.
  • the spatial information of the electronic device and the forearm that are to be controlled by the environment information collecting device 4 are sent to the processor 5, and the processor 5 determines the electronic device to be controlled according to the above information and calculates the spatial coordinates of the forearm, and according to the above information.
  • the corresponding operation instruction is output and sent to the electronic device to be controlled through the wireless communication module 6, thereby realizing interaction with the human machine of the electronic device to be controlled, that is, the control of the electronic device can be realized by gesture.
  • the gesture collection system of the present invention further includes a wristband 7 for wearing on the wrist, a main acquisition module 11, a wrist posture acquisition device 2, a forearm posture acquisition device 3, and environmental information collection.
  • the device 4, the wireless communication module 6 and the processor 5 can each be connected, for example, to a wristband 7, where the connection comprises a direct connection and an indirect connection, the direct connection is directly arranged on the wristband 7, and the indirect connection is via a component such as a connection.
  • a rod or the like is connected to the wristband 7.
  • the purpose of the wristband 7 is to facilitate the main acquisition module 11 to swing synchronously with the palm, and the wrist posture acquisition arm 22 of the wrist posture acquisition device 2 moves with the wristband 7.
  • the connection manner between the main collection module 11 and the wrist posture acquisition device 2 and the wristband 7 is specifically described below.
  • the wristband 7 may not be provided, and the main collection module 11 may also be fixed under the palm by a suction cup or the like.
  • the wrist posture acquisition device 2 can also be placed on the wrist by means of a suction cup.
  • the forearm posture acquisition device 3, the environmental information collection device 4, the wireless communication module 6, and the processor 5, etc. may be fixed on the arm or connected to the arm through a structure such as a connecting rod and located outside the arm, the forearm posture acquisition device 3, and the environmental information collection
  • the installation manner of the device 4, the wireless communication module 6, and the processor 5 is not limited thereto, as long as the corresponding signal acquisition, transmission, and processing can be satisfied.
  • the wristband 7 is sleeve-shaped, and the wristband 7 is spaced apart from the first fixing portion 71 and the second fixing portion 72, and the first fixing portion 71 and the second fixing portion 72 are respectively connected to the tibia.
  • the body surface projections of the styloid process and the ulnar styloid are adapted to securely attach the wrist band 7 to the wrist to prevent the wrist band 7 from falling off or sliding from the wrist.
  • the first fixing portion 71 and the second fixing portion 72 are through holes or grooves formed on the wristband 7, so that the sacral styloid process and the ulnar styloid can be received in the through hole or the groove, so that The wrist strap 7 is fixed to the wrist.
  • the wristband worn on the right hand is taken as an example. To describe, the palm of your right hand is facing down, the finger is perpendicular and pointing to the paper. It can be understood that the wristband structure worn on the left hand is the same as the wristband worn on the right hand, except that the opening positions of the first fixing portion 71 and the second fixing portion 72 respectively need to correspond to the body of the tibial styloid process on the left hand. The protruding part of the table and the protruding part of the ulnar styloid process on the left hand.
  • FIG. 2 is a schematic perspective structural view of a first embodiment of the gesture collection system of the present invention.
  • the wrist posture acquisition arm 22 includes a telescopic arm 220 that is extended with the wrist and a flexion arm 222 that flexes and contracts with the wrist.
  • the sensing module 24 includes two angle sensors, and the two angle sensors respectively In order to collect the telescopic angle sensor 240 of the telescopic angle of the telescopic arm 220 and the flexion and extension angle sensor 242 that collects the flexion and extension angle of the flexion and extension arm 222, the present embodiment uses the telescopic angle sensor 240 to collect the telescopic arm 220 when it is extended with the wrist.
  • the angle of the protrusion is obtained by the flexion and extension angle sensor 242 to obtain the flexion and extension angle of the flexion and extension arm 222 as the wrist is extended, so as to accurately collect the angle of the wrist and the angle of flexion and extension, thereby obtaining the posture of the wrist.
  • the wristband 7 and the retracting shaft when the wrist is extended have two intersection points corresponding to the telescopic arm 220, and the wristband 7 and the flexion and extension axis of the wrist flexion and extension have Two intersection points corresponding to the flexion and extension arms 222.
  • the telescopic arm 220 includes a fixed end and a free end opposite to the fixed end.
  • the flexing arm 222 also includes a fixed end and a free end opposite to the fixed end, wherein the telescopic arm 220 and the flexing arm 222 are both a fixed end is mounted on the wristband 7 and located at one of the corresponding two intersections and is fitted with its corresponding angle sensor, the free end is wound along the wristband 7 to the telescopic arm 220 and the flexion and extension
  • the other two intersections of the other of the arms 222 are connected to the fixed ends of the other of the both the arms 204 and the flex arms 222, the arms 220 and the
  • the connecting point of the flexing arm 222 has a certain angle and another angle sensor is mounted, and the free end of the other of the both the arm 220 and the flexing arm 222 is disposed around the wristband 7 and toward the wrist
  • the outer side of the strap 7 extends to a position close to the wrist of the wrist.
  • the position of the proximal wrist portion of the hand is fixed by the manner in which the collecting arm is engaged with the position of the proximal wrist portion of the hand.
  • the flexion and extension arm 222 will flex and extend with the wrist, and the flexion and extension angle sensor 242 can sense the angle change of the flexion and extension arm 222 to sense the flexion and extension posture of the wrist, and the wrist is closed when the wrist is closed.
  • the angle sensor 240 can sense the angle change of the arm 220 to sense the wrist. The attitude of the exhibition.
  • the free end of the telescopic arm 220 and the flexion arm 222 at the position of the close hand of the wrist is wound around the bottom of the position of the hand of the wrist and the main acquisition module 11 is installed to ensure the main The acquisition module 11 can swing with the palm of the hand.
  • the connecting rod can also be extended on the wristband 7 under the palm, and the main collection module 11 is mounted on the connecting rod.
  • the flexion and extension angle sensor 242 is disposed along the flexion and extension axis, and the telescopic angle sensor 240 is disposed along the telescopic axis, that is, the flexion and extension angle sensor 242 and the telescopic angle sensor 240 are both along the edge.
  • the tangent to the intersection is vertical.
  • the X direction represents the flexion and extension axis
  • the Z direction represents the contraction axis
  • the Y direction represents the axial direction of the wristband 7, and in the specific setting, the wrist
  • the belt 7 has two intersections A and B
  • the wristband 7 has two intersections C and D
  • one end of the flexing arm 222 is installed at any intersection of the wristband 7 and the flexion and extension axis, for example, the intersection of C
  • the flexion and extension angle sensor 242 is coupled to the fixed end of the flexion and extension arm 222 and disposed along the X axis.
  • the free end of the flexion and extension arm 222 is disposed around the wristband 7, and then the free end of the flexion and extension arm 222 is wound to the point A, and then the telescopic arm is assembled.
  • the fixed end of the 220 is connected to the free end of the flexion arm 222, and the free end of the telescopic arm 220 is wound around the outer side of the wristband 7 and fixed to the position of the proximal wrist of the hand, specifically the free end of the telescopic arm 220.
  • the winding direction of the C intersection extends to the outside of the wristband 7, and then is wound toward the wrist near the wrist to the outside of the wristband 7 and fixed to the position of the hand near the wrist, where the telescopic arm is used.
  • the position of the 220 card in the position of the hand near the wrist is fixed to the position of the hand near the wrist.
  • the flexion and extension arm 222 is swung around the flexion and extension axis through the wristband 7.
  • the flexion and extension angle sensor 242 is swung with the flexion and extension arm 222, and the flexion and extension angle sensor 242 collects the swing angle of the flexion and extension arm 222 to obtain The flexion and extension posture of the wrist, at this time, the telescopic arm 220 does not swing around the telescopic axis, and the telescopic angle sensor 240 does not collect the telescopic angle of the telescopic arm 220; when the wrist is in the exhibition, the exhibition is closed.
  • the arm 220 is fixed at a position close to the wrist of the hand, and the arm 220 is swung around the wrist about the wrist. The angle between the flexing arm 222 and the arm 220 changes with the swing of the arm 220.
  • the angle sensor 240 collects the swing angle of the telescopic arm 220 to obtain the telescopic posture of the wrist. At this time, the flexion and extension arm 222 does not swing about the flexion and extension axis, and the flexion and extension angle sensor 242 does not collect the flexion and extension arm 222. The angle of the exhibition.
  • the free end of the telescopic arm 220 can be wound around the bottom surface of the hand near the wrist and then extended toward the palm.
  • the main acquisition module 11 is mounted on the free end of the telescopic arm 220 to ensure The main acquisition module 11 swings synchronously with the palm.
  • the X direction represents the flexion and extension axis
  • the Z direction represents the telescopic axis
  • the Y direction is the axial direction of the wristband 7.
  • the telescopic arm The fixed end of the 220 is installed at any intersection of the wristband 7 and the retracting shaft, for example, the intersection of the A
  • the telescopic angle sensor 240 is fixed at the fixed end of the telescopic arm 220
  • the telescopic angle sensor 240 is disposed along the Z axis, and is extended.
  • the free end of the arm 220 is placed around the wristband 7, and then the free end of the telescopic arm 220 is, for example, wound to the point C.
  • the fixed end of the flexion arm 222 is coupled to the free end of the telescopic arm 220.
  • the flexion angle sensor 242 is mounted at the junction of the flexion arm 222 and the telescopic arm 220, and the flexion angle sensor 242 is disposed along the X axis.
  • the free end of the flexion arm 222 is wound around the outer side of the wristband 7 and fixed to the position of the proximal wrist of the hand, specifically the free end of the flexing arm 222 is oriented toward the intersection of A and then wound to the point D, and in the process of winding Extending to the outside of the wristband 7, and then winding toward the wrist near the wrist to the outside of the wristband 7 and fixed to the position of the proximal wrist of the hand, here by the position of the flexor arm 222 and the proximal wrist of the hand
  • the manner of snapping is fixed to the position of the proximal wrist of the hand.
  • the flexion and extension arm 222 When the wrist is flexed and stretched, the flexion and extension arm 222 is fixed at a position close to the wrist of the hand, and the angle between the flexion and extension arm 222 and the telescopic arm 220 is changed, and the flexion and extension angle sensor 242 is collected. The swing angle of the flexion and extension arm 222 is obtained to obtain the flexion and extension posture of the wrist. At this time, the telescopic arm 220 does not swing around the telescopic axis, and the telescopic angle sensor 240 does not collect the telescopic angle of the telescopic arm 220.
  • the arm 220 When the wrist is in the exhibition, the arm 220 is swung by the wristband 7 as the wrist swings around the take-up axis, and the take-up angle sensor 240 swings with the take-up axis, and the take-up angle sensor 240 collects the take-up arm 220.
  • the angle of the swing is to obtain the telescopic posture of the wrist.
  • the flexion and extension arm 222 does not swing about the flexion and extension axis, and the flexion and extension angle sensor 242 does not collect the flexion and extension angle of the flexion and extension arm 222.
  • the free end of the flexion and extension arm 222 can be wound around the bottom surface of the hand near the wrist and then extended toward the palm.
  • the main acquisition module 11 is mounted on the free end of the flexion and extension arm 222 to ensure the main acquisition. Module 11 swings synchronously with the palm of the hand.
  • FIG. 5 is a schematic perspective structural view of a second embodiment of a gesture collection system according to the present invention.
  • the wrist posture collection arm 22 is a telescopic rod
  • the wrist posture collection arm 22 is The wristband 7 is connected, including a first end located above, laterally or below the wristband 7, and a second end fixed to a position of the hand of the wrist, the wrist posture collecting arm 22 and the The intersection of the wristband 7 is located between the first end and the second end, the first end of the wrist posture acquisition arm 22 is swingable around the intersection, and the sensing module 24 obtains the The first end swing information acquires the flexion and extension information and the extension information of the wrist posture acquisition arm 22.
  • the wristband 7 is provided with a fisheye bearing 75
  • the wrist posture collecting arm 22 is connected to the wrist posture collecting arm 22 by means of the fisheye bearing 75, the fisheye
  • the structure and the operating principle of the bearing 75 are all prior art and will not be described herein.
  • the wrist posture acquisition arm 22 can be swung with the wrist through a fisheye bearing 75 mounted on the wristband 7, specifically the wrist attitude acquisition arm 22 between the fisheye bearing 75 and the second end of the wrist attitude acquisition arm 22.
  • the portion is telescoped as needed, and the first end of the wrist posture collecting arm 22 is swung with the fish eye bearing 75, and the sensing module 24 can acquire the swing information of the first end to obtain the wrist posture, more specifically When the wrist is in the exhibition, the first end of the wrist posture collecting arm 22 is swung left and right with the fish eye bearing 75, and the sensing module 24 obtains the angle of the left and right swing of the first end to obtain the extension of the wrist.
  • the first end of the wrist posture collecting arm 22 is swung up and down with the fish eye bearing 75, and the sensing module 24 obtains the angle of the up and down swing of the first end to obtain the flexion and extension of the wrist. Gesture information.
  • the sensing module 24 is a Hall sensor.
  • the first end of the wrist posture acquiring arm 22 is provided with a magnet 26, and the magnet 26 is formed near the first end of the wrist posture collecting arm 22.
  • the magnetic field, the Hall sensor senses the magnetic field formed by the magnet 26, and the Hall sensor can sense the change of the magnetic field.
  • the change of the magnetic field is formed by the first end of the wrist posture collecting arm 22, so that the sensing module 24 obtains the
  • the first end magnetic field information of the wrist posture acquisition arm 22 is used to obtain the vertical swing angle and the left and right swing angle of the first end of the wrist posture acquisition arm 22, thereby acquiring the flexion and extension posture information and the extension posture information of the wrist.
  • the sensing module 24 is located in the extending direction of the wrist posture acquiring arm 22 and is further away from the second end of the wrist posture collecting arm 22 with respect to the first end of the wrist posture collecting arm 22 to facilitate sensing.
  • the module 24 accurately acquires the first end magnetic field information of the wrist posture acquisition arm 22.
  • the gesture collection system of the present invention further includes a fixing structure 8 and the fixing structure 8 is
  • the "C" shape can be specifically bent by a rigid plastic plate, and the fixing structure 8 is fixed at a position of the wrist near the hand, for example, by a snapping manner, and the fixing structure 8 is closer to the hand and the wrist with respect to the wristband 7.
  • the second end of the attitude collecting arm 22 is fixed on the fixed structure 8.
  • the main collecting module 11 is fixed on the fixed structure 8.
  • the fixing structure 8 extends, for example, in the direction of the finger. Below the palm, the main acquisition module 11 is fixed to the connecting rod.
  • the installation area of the main collection module 11 is 0 to 5 cm in the direction of the wrist joint to the finger, 0 to 3 cm in the wrist to the arm, and 0 to 3 cm in the skin of the body surface, preferably from the skin surface of the body surface.
  • the main acquisition module 11 is installed in this area to achieve accurate collection of finger posture information.
  • the gesture collection system of the present invention has the following advantages:
  • the traditional gesture recognition devices use the external environment to capture images and thus identify the gestures. Such methods will require each controlled device to be equipped with an identification device (keyboard mouse pointer). . For example, there are 20 controlled devices (pads, computers, televisions, mobile phones, other intelligent home appliances, etc.) in the home. Each device needs to install a gesture recognition device, which is very costly; and the present invention passes the environment.
  • the information collecting device 4 determines the electronic device that needs to be controlled, and realizes control of a plurality of controlled devices through a set of gesture recognition devices.
  • a system for recognizing a gesture by taking an image in an external environment must first segment the hand motion from a complex and noisy background, thereby reducing the false recognition rate.
  • the invention sets the main collection module on the bottom surface of the palm and swings synchronously with the palm, that is, the angle of view and the position of the main acquisition module are fixed relative to the finger, and the same gesture action is only recognized by the image taken from a fixed angle,
  • the background needs to be separated, and the recognition system can be used without regard to the computational cost required for background deletion.
  • the system for recognizing gestures by taking images in an external environment When the camera is used in an external environment, it is required to recognize the same finger movement posture from different angles, which requires a large amount of data training (a large number of different angles of the same posture) Images to train the recognition system to ensure accuracy).
  • the same posture such as the finger is close together, the posture is captured in the external environment, it is necessary to obtain a photo with a 360-degree imaging angle in different directions, so as to ensure the recognition accuracy of the motion from various angles, and the palm of the hand is used.
  • a fixed position, a fixed angle of view to take photos, for the same posture does not need to obtain photos from 360 * 360 degrees in all directions to train the recognition system. This can greatly reduce the amount of data trained and greatly improve the accuracy of recognition.
  • the finger Since the camera is deployed at the root of the palm, the finger is only 15cm-20cm away from the camera, which can reduce the resolution of the camera and greatly reduce the image data processed by the system. If the camera is deployed in a peripheral environment, the distance from the camera is smaller. Far, and the position is not fixed, so you must increase the resolution of the camera to ensure that you can capture enough clarity in any position.
  • Gesture recognition requires real-time, and if the recognition speed is too slow, the user will not recognize it.
  • the power consumption of the recognition system is related to the number of floating point operations (multiplication of neural network weights).
  • processing the scaling of the gesture image also increases the number of nonlinear weights of the convolutional neural network, and generates multiple power consumption;
  • Using the camera in an external environment requires the hand to be placed in the area where the camera is located, including the way the hololens are used, which requires the arm to be lifted to ensure that the camera in the helmet can reach the hand, and our way to the hand can be perceived at any position. .
  • the present invention does not require lifting the arm for a long time, and placing the hand in the recognition area of the camera can prevent the arm from swelling (the gorilla arm), which is quite common among ipad users. It also prevents the mouse hand (carpal canal syndrome) from happening. In addition, the gorilla arm can cause the user to lower the handheld device, causing cervical problems.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • User Interface Of Digital Computer (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

一种手势采集系统，包括手指姿态采集装置，所述手指姿态采集装置包括位于手掌的下方且与手掌同步摆动的主采集模块，所述主采集模块用于采集手指姿态。手势采集系统，通过将手指姿态采集装置的主采集模块设置于手掌下方并与手掌同步摆动，这样主采集模块的视角和位置相对手掌固定，这样同一个手势动作只需从一个固定角度拍摄的图像来识别，减少识别时间，提高识别准确率，也省略了背景删除计算消耗。

Description

手势采集系统 技术领域
本发明涉及人机交互技术领域，具体涉及一种手势采集系统。
背景技术
随着科技的发展，各种智能设备的应用越来越广泛。人与智能设备之间的人机交互越来越多，人机交互方式有接触式或非接触式两种，接触式的人机交互较为成熟和完善，而非接触式的人机交互操作方式则处于研究起步阶段。近年来，非接触式的人机交互方式一直是人机交互方式中研究的热点和难点问题，研究好非接触式的人机交互方式具有很高的使用价值。
手势是一种直观的、自然的交互方式，表达快捷，表达意义丰富，是人类互相交流信息的重要工具。手势识别时按照某种规则识别出手势表达的内容，具有很强的一致性和扩展性。在应用方面，手势识别系统可以在应用软件中直接操作软件或虚拟对象。然而，如何更有效地对手势进行识别，到达更好的人机交互状态，是研究的热点和难点问题。
发明内容
本发明的目的在于提供一种手势采集系统，能够准确获取手指姿态信息且识别时间少。
为达到上述目的，本发明采用的技术方案是：一种手势采集系统，包括手指姿态采集装置，所述手指姿态采集装置包括位于手掌的下方且与手掌同步摆动的主采集模块，所述主采集模块用于采集手指姿态。
进一步地，还包括用于采集腕部的收展姿态与腕部的屈伸姿态的腕部姿态采集装置，所述腕部姿态采集装置包括随着腕部收展与屈伸的腕部姿态采集臂及用于采集所述腕部姿态采集臂的屈伸信息及收展信息的传感模块。
进一步地，还包括腕带，所述腕带套在手腕上，所述腕部姿态采集臂与所述腕带连接，所述腕部姿态采集臂通过所述腕带随着腕部收展与屈伸。
进一步地，所述腕部姿态采集臂包括随着腕部收展的收展臂及随着腕部屈伸的屈伸臂，所述传感模块包括两个角度传感器，所述两个角度传感器分别为采集所述收展臂的收展角度的收展角度传感器以及采集所述屈伸臂的屈伸角度的屈伸角度传感器。
进一步地，所述腕带与腕部收展时的收展轴有两个对应所述收展臂的交点，所述腕带与腕部屈伸时的屈伸轴有两个对应所述屈伸臂的交点，所述收展臂与所述屈伸臂均包括固定端及与所述固定端相对的自由端，所述收展臂与所述屈伸臂两者其中之一的固定端安装在所述腕带上并位于对应两个交点的其中之一处且安装其对应的角度传感器，自由端沿着所述腕带绕至所述收展臂与所述屈伸臂两者中的另一个对应的两个交点其中之处并与所述收展臂与所述屈伸臂两者中的另一个的固定端连接，所述收展臂与所述屈伸臂连接点具有一定夹角且安装另一个角度传感器，所述收展臂与所述屈伸臂两者中的另一个的自由端绕着所述腕带设置并向所述腕带的外侧延伸至与腕部的近手的位置固定。
进一步地，所述屈伸角度传感器沿着所述屈伸轴设置，所述收展角度传感器沿着所述收展轴设置。
进一步地，所述收展臂与所述屈伸臂两者中的位于腕部的近手的位置的自由端绕至腕部的近手的位置的底部并安装所述主采集模块。
进一步地，所述腕部姿态采集臂为伸缩杆，所述腕部姿态采集臂与所述腕带连接，包括位于所述腕带上方或侧方的第一端以及与所述手部固接的第二端，所述腕部姿态采集臂与所述腕带的交点位于所述第一端与所述第二端之间，所述第一端能够围绕所述交点摆动，所述传感模块通过获取所述第一端的摆动信息获取所述腕部姿态采集臂的屈伸信息与收展信息。
进一步地，所述腕带上设有鱼眼轴承，所述腕部姿态采集臂通过穿设所述鱼眼轴承的方式与所述腕部姿态采集臂连接。
进一步地，所述传感模块为霍尔传感器，所述腕部姿态采集臂的第一端设 有磁铁，所述传感模块通过获取所述第一端的磁场信息的方式，获取所述腕部姿态采集臂的屈伸信息与收展信息。
进一步地，所述传感模块位于所述腕部姿态采集臂的延伸方向上，且相对所述腕部姿态采集臂的第一端更远离所述腕部姿态采集臂的第二端。
进一步地，还包括固定结构，所述固定结构固定在腕部近手的位置，所述腕部姿态采集臂的第二端固定在所述固定结构上，所述主采集模块固定在所述固定结构上。
进一步地，还包括安装在所述腕带上的前臂姿态采集装置，所述前臂姿态采集装置用于采集前臂姿态。
进一步地，所述前臂姿态采集装置包括三轴陀螺仪、三轴磁力计与三轴加速度传感器。
进一步地，所述腕带呈套状，所述腕带间隔地设有第一固定部与第二固定部，所述第一固定部与第二固定部分别与桡骨茎突和尺骨茎突的体表突出部分适配以将所述腕带固定装设在手腕上。
进一步地，所述第一固定部与第二固定部为开设在所述腕带上的通孔或者凹槽。
进一步地，还包括环境信息采集装置，所述环境信息采集装置用于在周围环境寻找需要控制的电子设备。
进一步地，还包括处理器，所述处理器与所述手势采集系统中的其他电子元件连接。
进一步地，所述主采集模块为摄像头。
进一步地，所述主采集模块的安装区域为腕关节向手指方向0～5厘米以内、手腕到手臂0～3厘米以内且距离体表皮肤0～3厘米以内。
由于上述技术方案的应用，本发明具有以下有益效果：
本发明的手势采集系统，通过将手指姿态采集装置的主采集模块设置于手掌下方并与手掌同步摆动，这样主采集模块的视角和位置相对手掌固定，这样同一个手势动作只会用从一个固定角度拍摄的图像来识别，减少识别时间，提 高识别准确率，也省略了背景删除计算消耗。
附图说明
图1为本发明的手势采集系统的电路方框图；
图2为本发明实施例手势采集系统第一实施例的立体结构示意图；
图3为图2的示意图；
图4为本发明另一实施方式的示意图；
图5为本发明实施例手势采集系统第二实施例的立体结构示意图；
图6为图5中VI处的放大。
具体实施方式
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本发明，并不用于限定本发明。
请参阅图1，本发明实施例提供的手势采集系统，包括手指姿态采集装置1，该手指姿态采集装置1包括主采集模块11及辅助采集模块(图未示出)，主采集模块11设在手掌下方并随着手掌同步摆动，主采集模块11可以用于采集手指姿态信息，主采集模块11例如可以为摄像头，主采集模块11通过获取手指图像的方式采集手指姿态信息。辅助采集模块也用于辅助采集手指姿态信息，辅助采集模块可以位于手部的任意一个可以采集到手指姿态的位置，以实现对手指姿态信息的辅助采集。辅助采集模块例如可以为摄像头，通过获取手指图像的方式辅助获取手指姿态信息。
可以理解的是，在人机互动中，手势采集系统不单单要采集手指姿态，还需要采集腕部姿态及前臂姿态。
为了实现对腕部姿态的采集，本发明的手势采集系统例如还包括腕部姿态采集装置2，所述腕部姿态采集装置2用于采集腕部的收展姿态信息与腕部的屈伸姿态信息。本发明通过腕部姿态采集装置2采集腕部的收展姿态信息与腕 部的屈伸姿态信息，实现对腕部姿态的采集。需要说明的是，在解剖学上，将手掌伸开，腕部绕着垂直于手掌的轴摆动即为腕部的收展作动，腕部绕着手腕关节摆动即为腕部的屈伸作动，本发明中的腕部姿态采集装置2可以对腕部的收展作动及屈伸作动进行采集从而获取腕部的收展姿态信息与腕部的屈伸姿态信息。
请结合参阅图2-5，在本实施例中，腕部姿态采集装置2包括随着腕部收展与屈伸的腕部姿态采集臂22，所述传感模块24采集所述腕部姿态采集臂22的屈伸信息及收展信息，从而实现对腕部姿态信息的采集。为了保证腕部姿态采集臂22能够随着腕部收展与屈伸，传感模块24能够精确采集到腕部姿态采集臂22的屈伸信息及收展信息，腕部姿态采集臂22通常由硬质材料制作，这样腕部姿态采集臂22不会轻易形变，可以避免因腕部姿态采集臂22轻易变形所导致的影响腕部姿态采集臂22的屈伸及收展作动，以致最终影响采集准确度的问题。
为了实现对前臂姿态信息的采集，本发明的手势采集系统例如还包括前臂姿态采集装置3，所述前臂姿态采集装置3用于采集前臂姿态信息。在本实施例中，所述前臂姿态采集装置3包括三轴陀螺仪、三轴磁力计与三轴加速度传感器，其中，三轴陀螺仪用于检测前臂的三轴角速度，三轴加速度传感器用于检测前臂的三轴加速度，三轴磁力计用于检测前臂的所处状态的三轴磁力，使检测的前臂姿态信息更加精确，以获取精确的前臂姿态信息。
本发明的手势采集系统还能够确定需要控制的电子设备，本发明的手势采集系统例如还包括环境信息采集装置4，该环境信息采集装置4用于在周围环境寻找需要控制交互的电子设备(电脑，平板，家用电器，IOT，车载设备等)并实现手臂相对于室内或者室外的局部外周环境的定位。在本实施例中，环境信息采集装置4为环境信息摄像头。当然，前臂姿态采集装置3还包括设于腕带上的反光点，环境信息采集装置4可以采集反光点的位置信息。
为便于实现人机互动，本发明的手势采集系统还包括处理器5及无线通信模块6，手指姿态采集装置1、腕部姿态采集装置2、前臂姿态采集装置3、环 境信息采集装置4以及无线通信模块6均与处理器5连接，手指姿态采集装置1采集的手指姿态信息、腕部姿态采集装置2的腕部姿态信息、前臂姿态采集装置3采集的前臂姿态信息及环境信息采集装置4寻找到的需要控制的电子设备与前臂的空间信息均发送至处理器5，处理器5根据上述信息确定需要控制的电子设备以及计算出前臂的空间坐标，并根据上述信息输出相应的操作指令并通过无线通信模块6发送至需要控制的电子设备，从而实现与需要控制的电子设备的人机互动，即通过手势即可实现对电子设备的控制。
请参阅图2-5，本发明的手势采集系统还包括腕带7，腕带7用于戴在手腕上，主采集模块11、腕部姿态采集装置2、前臂姿态采集装置3、环境信息采集装置4、无线通信模块6以及处理器5例如均可以与腕带7连接，这里的连接包括直接连接和间接连接，直接连接就是直接设置在腕带7上，间接连接就是通过某种元件例如连接杆等与腕带7连接。设置腕带7的目的是为了便于实现主采集模块11随着手掌同步摆动，腕部姿态采集装置2的腕部姿态采集臂22随着腕带7运动。主采集模块11及腕部姿态采集装置2与腕带7的连接方式具体见下详述。
当然，在其他实施例中，也可以不设置腕带7，主采集模块11也可以通过吸盘等方式固定在手掌的下方。腕部姿态采集装置2也可以通过吸盘的方式设置在腕部。前臂姿态采集装置3、环境信息采集装置4、无线通信模块6以及处理器5等可以固定在手臂上或者通过连杆等结构与手臂相连并位于手臂的外侧，前臂姿态采集装置3、环境信息采集装置4、无线通信模块6以及处理器5的安装方式不限于此，只要能满足相应的信号采集、传输及处理即可。
在本实施例中，腕带7呈套状，所述腕带7间隔地设有第一固定部71与第二固定部72，所述第一固定部71与第二固定部72分别与桡骨茎突和尺骨茎突的体表突出部分适配以将所述腕带7固定装设在手腕上，以防止腕带7从手腕处脱落或滑动。具体地，所述第一固定部71与第二固定部72为开设在所述腕带7上的通孔或者凹槽，这样桡骨茎突和尺骨茎突能够收容在通孔或凹槽，使得腕带7固定在手腕上。需要说明的是，在图示中，是以戴在右手的腕带为例 进行描述，右手手掌朝下，手指垂直并指向纸面。可以理解的是，戴在左手上的腕带结构和戴在右手上的腕带结构相同，只是第一固定部71与第二固定部72的开设位置需要分别对应左手上的桡骨茎突的体表突出部分和左手上的尺骨茎突的体表突出部分。
具体地，请参阅图2，图2是本发明的手势采集系统第一实施例的立体结构示意图。在该实施例中，腕部姿态采集臂22包括随着腕部收展的收展臂220及随着腕部屈伸的屈伸臂222，传感模块24包括两个角度传感器，两个角度传感器分别为采集收展臂220的收展角度的收展角度传感器240以及采集屈伸臂222的屈伸角度的屈伸角度传感器242，本实施例通过收展角度传感器240采集收展臂220随腕部收展时的收展角度，通过屈伸角度传感器242采集屈伸臂222随腕部收展时的屈伸角度，以实现对腕部收展角度及屈伸角度的精确采集，从而获取腕部姿态。
进一步地，腕带7在戴在手腕上后，腕带7与腕部收展时的收展轴有与收展臂220对应的两个交点，腕带7与腕部屈伸时的屈伸轴有与屈伸臂222对应的两个交点。收展臂220包括固定端及与固定端相对的自由端，屈伸臂222同样包括固定端及与固定端相对的自由端，其中，收展臂220与所述屈伸臂222两者其中之一的固定端安装在所述腕带7上并位于对应两个交点的其中之一处且安装其对应的角度传感器，自由端沿着所述腕带7绕至所述收展臂220与所述屈伸臂222两者中的另一个对应的两个交点其中之处并与所述收展臂220与所述屈伸臂222两者中的另一个的固定端连接，所述收展臂220与所述屈伸臂222连接点具有一定夹角且安装另一个角度传感器，所述收展臂220与所述屈伸臂222两者中的另一个的自由端绕着所述腕带7设置并向所述腕带7的外侧延伸至与腕部的近手的位置固定，本实施例是通过采集臂与手的近手腕部的位置卡接的方式与手的近手腕部的位置固定。这样手腕屈伸动作时，手腕在屈伸时，屈伸臂222将随着手腕屈伸，屈伸角度传感器242即可感应到屈伸臂222的角度变化从而感应到手腕的屈伸姿态，手腕在收展时，收展臂220随着手腕收展，收展角度传感器240即可感应到收展臂220的角度变化从而感应到手腕 的收展姿态。
在本实施例中，收展臂220与屈伸臂222两者中的位于腕部的近手的位置的自由端绕至腕部的近手的位置的底部并安装主采集模块11，以保证主采集模块11可以随着手掌摆动。当然在其他实施例中，也可以在腕带7上延伸出连接杆位于手掌下方，主采集模块11安装在连接杆上。
为了保证采集到的角度信息的准确性，屈伸角度传感器242沿着屈伸轴设置，收展角度传感器240沿着所述收展轴设置，也就是屈伸角度传感器242与收展角度传感器240均与沿对应交点的切线垂直。
具体地，在本实施例中，如图3所示，在三维坐标系中，X方向代表屈伸轴，Z方向代表收展轴，Y方向为腕带7的轴向，在具体设置时，腕带7与收展轴有A、B两个交点，腕带7与屈伸轴有C、D两个交点，屈伸臂222的一端安装在腕带7与屈伸轴的任意一交点例如C交点处，屈伸角度传感器242与屈伸臂222的固定端连接且沿着X轴设置，屈伸臂222的自由端绕着腕带7设置，然后屈伸臂222的自由端绕到A点位置，然后将收展臂220的固定端与屈伸臂222的自由端连接，再将收展臂220的自由端绕向腕带7的外侧且与手的近手腕部的位置固定，具体为收展臂220的自由端朝向C交点所在方向绕线的过程中延伸至腕带7的外侧，然后朝向手靠近手腕的位置绕线至位于腕带7的外侧且与手的近手腕部的位置固定，此处采用收展臂220卡在手的近手腕部的位置的方式与手的近手腕部的位置固定。这样腕部屈伸时，屈伸臂222通过腕带7随着腕部绕着屈伸轴摆动，屈伸角度传感器242随着屈伸臂222摆动，屈伸角度传感器242从而采集到屈伸臂222的摆动角度，以获取腕部的屈伸姿态，此时，收展臂220并不会绕着收展轴摆动，收展角度传感器240不会采集到收展臂220的收展角度；腕部在收展时，收展臂220固定在手的靠近腕部的位置，收展臂220随着腕部绕着收展轴摆动，屈伸臂222与收展臂220之间角度随着收展臂220的摆动发生变化，收展角度传感器240采集到收展臂220的摆动角度，以获取腕部的收展姿态，此时，屈伸臂222并不会绕着屈伸轴摆动，屈伸角度传感器242不会采集到屈伸臂222的收展角度。
为了方便安装主采集模块11，收展臂220的自由端可以绕至手的近手腕部的位置的底面，然后再朝向手掌延伸，主采集模块11安装在收展臂220的自由端，以保证主采集模块11随着手掌同步摆动。
在另一实施例中，如图4所示，X方向代表屈伸轴，Z方向代表收展轴，Y方向为腕带7的轴向，在具体设置时，在该实施例中，收展臂220的固定端安装在腕带7与收展轴的任意一交点例如A交点处，收展角度传感器240固定在收展臂220的固定端且收展角度传感器240沿着Z轴设置，收展臂220的自由端绕着腕带7设置，然后收展臂220的自由端例如绕至C点位置。屈伸臂222的固定端与收展臂220的自由端连接，屈伸角度传感器242安装在屈伸臂222与收展臂220的连接处，屈伸角度传感器242沿着X轴设置。屈伸臂222的自由端绕向腕带7的外侧且与手的近手腕部的位置固定，具体为屈伸臂222的自由端朝向A交点然后再向D点绕线，并在绕线的过程中延伸至腕带7的外侧，然后朝向手靠近手腕的位置绕线至位于腕带7的外侧且与手的近手腕部的位置固定，此处是通过屈伸臂222与手的近手腕部的位置卡接的方式与手的近手腕部的位置固定。这样腕部屈伸时，屈伸臂222固定在手的靠近腕部的位置随着腕部绕着屈伸轴摆动，屈伸臂222与收展臂220之间的夹角发生变换，屈伸角度传感器242从而采集到屈伸臂222的摆动角度，以获取腕部的屈伸姿态，此时，收展臂220并不会绕着收展轴摆动，收展角度传感器240不会采集到收展臂220的收展角度；腕部在收展时，收展臂220通过腕带7随着腕部绕着收展轴摆动，收展角度传感器240随着收展轴摆动，收展角度传感器240采集到收展臂220的摆动角度，以获取腕部的收展姿态，此时，屈伸臂222并不会绕着屈伸轴摆动，屈伸角度传感器242不会采集到屈伸臂222的屈伸角度。
为了方便安装主采集模块11，屈伸臂222的自由端可以绕至手的近手腕部的位置的底面，然后再朝向手掌延伸，主采集模块11安装在屈伸臂222的自由端，以保证主采集模块11随着手掌同步摆动。
请参阅图5，图5是本发明的手势采集系统第二实施例的立体结构示意图，在该实施例中，所述腕部姿态采集臂22为伸缩杆，所述腕部姿态采集臂22与 所述腕带7连接，包括位于所述腕带7上方、侧方或下方的第一端以及与腕部的近手的位置固接的第二端，所述腕部姿态采集臂22与所述腕带7的交点位于所述第一端与所述第二端之间，所述腕部姿态采集臂22的第一端能够围绕所述交点摆动，所述传感模块24通过获取所述第一端的摆动信息获取所述腕部姿态采集臂22的屈伸信息与收展信息。
具体地，在该实施例中，腕带7上设有鱼眼轴承75，腕部姿态采集臂22通过穿设所述鱼眼轴承75的方式与所述腕部姿态采集臂22连接，鱼眼轴承75的结构及作动原理均为现有技术，在此不再赘述。腕部姿态采集臂22可以通过安装在腕带7上的鱼眼轴承75随着腕部摆动，具体为腕部姿态采集臂22在鱼眼轴承75与腕部姿态采集臂22第二端之间的部分根据需要进行伸缩，腕部姿态采集臂22的第一端通过鱼眼轴承75随着腕部摆动，传感模块24可以获取第一端的摆动信息以获取腕部姿态，更具体地是，腕部在收展时，腕部姿态采集臂22的第一端通过鱼眼轴承75随着腕部左右摆动，传感模块24获取第一端的左右摆动的角度以获取腕部的收展姿态信息，腕部在屈伸时，腕部姿态采集臂22的第一端通过鱼眼轴承75随着腕部上下摆动，传感模块24获取第一端的上下摆动的角度以获取腕部的屈伸姿态信息。
在本实施例中，请参阅图6，传感模块24为霍尔传感器，腕部姿态采集臂22的第一端设有磁铁26，磁铁26在腕部姿态采集臂22的第一端附近形成磁场，霍尔传感器感应磁铁26形成的磁场，霍尔传感器可以感应磁场的变化，磁场的变化是由于腕部姿态采集臂22的第一端摆动形成，因此所述传感模块24通过获取所述腕部姿态采集臂22的第一端磁场信息，以获取腕部姿态采集臂22的第一端的上下摆动角度以及左右摆动角度，进而获取腕部的屈伸姿态信息与收展姿态信息。
在本实施例中，传感模块24位于腕部姿态采集臂22的延伸方向上且相对腕部姿态采集臂22的第一端更远离腕部姿态采集臂22的第二端，以便于传感模块24准确获取腕部姿态采集臂22的第一端磁场信息。
在本实施例中，本发明的手势采集系统还包括固定结构8，固定结构8呈 “C”形，具体可以通过硬质塑胶板弯折而成，固定结构8固定在腕部近手的位置例如通过卡接的方式固定，固定结构8相对于腕带7更靠近手，腕部姿态采集臂22的第二端固定在固定结构8上，主采集模块11固定在固定结构8上，为了使主采集模块11位于手掌的下方，固定结构8上例如向手指的方向延伸出连接杆至手掌下方，主采集模块11固定在连接杆上。
在上述内容中，主采集模块11的安装区域为腕关节向手指方向0～5厘米以内，手腕到手臂0～3厘米以内，且距离体表皮肤0～3厘米以内，优选距离体表皮肤0～2厘米以内，将主采集模块11安装在这个区域可以实现对手指姿态信息的精确采集。
与现有技术相比，本发明的手势采集系统具有以下优点：
1.降低成本：传统的手势识别装置都是采用外部环境来拍摄图像从而通过识别手势的方法来进行的，此类方法会导致每一台被控设备都需要配备识别设备(键盘鼠标手标)。例如家里有20个被控设备(pad，电脑，电视机，手机，其他智能家电等IOT设备)，每个设备都需要安装一套手势识别装置，这是非常耗费成本的；而本发明通过环境信息采集装置4确定需要控制的电子设备，实现通过一套手势识别设备就可以对多个被控设备的控制。
2.大幅提高手势识别准确度：
a)现有技术中采用外部环境拍摄图像的方式来识别手势的系统，必须先将手部动作从复杂嘈杂的背景中分离(segment)出来，这样才能降低误识别率。本发明通过设置主采集模块设在手掌底面并与手掌同步摆动，也就是主采集模块的视角和位置相对手指而言固定，同一个手势动作只会用从一个固定角度拍摄的图像来识别，不需要分离背景，同时也可让识别系统不用考虑背景删除所需的计算消耗。
b)现有技术中采用外部环境拍摄图像的方式来识别手势的系统需要对分离出来的手势图像的尺寸缩放，旋转，远近，遮挡进行处理。我们的设计方式，手指的位置距离摄像头是固定的，无需对图像进行尺寸缩放和旋转处理。也不会因为拍摄角度不同导致大面积遮挡的发生。
c)采用外部环境拍摄图像的方式来识别手势的系统在外部环境中使用摄像头时对于同一个手指动作姿势要求从不同角度都能识别，这就要求需要大量数据训练(同一种姿势不同角度的大量图像来训练识别系统才能保证准确度)。同一个姿势比如手指并拢伸直这个姿态在外部环境拍摄图像的方式来识别时，需要360度不同方向的摄像角度获得照片，才能保证从各个角度对这一个动作的识别准确率，而采用在手掌固定的位置，固定的视角拍摄照片，对于同一个姿势不需要从360*360度各个方向上获取照片来训练识别系统。这可以大大减少训练的数据量，也大大提高了识别准确度。
d)由于采用手掌根部部署摄像头，手指距离摄像头最远也只有15cm-20cm，可以降低摄像头分辨率，也大幅降低系统处理的图像数据，而如果采用外周环境部署摄像头的方式，由于人手距离摄像头较远，且位置不固定，所以必须提高摄像头分辨率保证任何位置都能拍摄到足够的清晰度。
3.加快识别速度：
手势识别是需要实时性的，识别速度如果太慢，用户不会认可。
由上面2中的a)可得：如果有嘈杂的背景，就必须对图像进行遍历，定位手所在的位置，很耗费时间；
由上面2中的b)可得：对手势图像的缩放进行处理也要对图像遍历，更耗费时间；
由上面2中的c)可得：对一种手指姿态从不同角度去识别，比单一视角识别需要更多层的神经元，假设以1度的范围对同一个手指姿态进行识别就需要比单一视角识别多达129600不同角度的图片。单一视角只要1张图片训练就可以。可以大幅降低神经元权重的数量，运算加速明显；
由上面2中的d)可得：摄像头分辨率的降低导致图像的像素大幅降低，处理的像素量会小很多。500万像素的比62500像素的设备处理像素的数量降低近80倍。在卷积神经网络处理过程中，1张图在一层神经网络里要用近20个过滤器遍历处理，单层的性能就会提高1600倍，在深度神经网络里会有多层，提高的还要多。
4.降低功耗：
识别系统的功耗和浮点运算次数(神经网络权重的乘法)相关。
由上面2中的a)可得：对卷积核对图像进行遍历，来过滤嘈杂的背景，需要耗费较多功耗；
由上面2中的b)可得：对手势图像的缩放进行处理也要增加卷积神经网络的非线性权重的数量，产生多倍的功耗；
由上面2中的c)可得：减少了同一个姿势的manifolder图像，可以大大降低卷积神经网络的非线性权重的数量和神经网络的深度；
由上面2中的d)可得：图像的像素大幅降低，500万像素的比62500像素的设备处理像素的在单层卷积神经网络处理过程中，会提高1600倍(浮点运算量)，在多层深度神经网络处理过程中又会成倍增加，功耗降低明显。
在leap motion类似的外周环境拍摄方式，自身只能提供80％的运算能力，还有20％的运算靠USB连接的计算机才能运行，这种方式要在穿戴式设备中实现是不可能的，因为穿戴式设备的电池不能提供给芯片足够的功耗来获得运算结果。其次穿戴式设备的主芯片运算能力也不可能比得上计算机。只有采用我们发明的这种方式才能在穿戴式设备中实现实时的识别。
5.可以在任何地方识别
在外部环境中使用摄像头，需要把手放在该摄像头所在的区域，包括hololens所采用的方式都要求手臂举起，保证头盔中的摄像头能照射到手，而我们的方式手放在任何位置都可以感知。
6.防止大猩猩臂
由于5的缘故，本发明不需要长时间抬举手臂，并让手放在摄像头的识别区，可以防止手臂酸胀(大猩猩臂)，这个症状在ipad的使用者当中相当普遍。也可以防止鼠标手(腕管综合症)的发生。此外，大猩猩臂会导致用户将手持设备放低，从而引发颈椎问题。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明 的保护范围之内。

Claims (20)

  1. 一种手势采集系统，其特征在于：包括手指姿态采集装置(1)，所述手指姿态采集装置(1)包括位于手掌的下方且与手掌同步摆动的主采集模块(11)，所述主采集模块(11)用于采集手指姿态。
  2. 根据权利要求1所述的手势采集系统，其特征在于：还包括用于采集腕部的收展姿态与腕部的屈伸姿态的腕部姿态采集装置(2)，所述腕部姿态采集装置(2)包括随着腕部收展与屈伸的腕部姿态采集臂(22)及用于采集所述腕部姿态采集臂(22)的屈伸信息及收展信息的传感模块(24)。
  3. 根据权利要求2所述的手势采集系统，其特征在于：还包括腕带(7)，所述腕带(7)套在手腕上，所述腕部姿态采集臂(22)与所述腕带(7)连接，所述腕部姿态采集臂(22)通过所述腕带(7)随着腕部收展与屈伸。
  4. 根据权利要求3所述的手势采集系统，其特征在于：所述腕部姿态采集臂(22)包括随着腕部收展的收展臂(220)及随着腕部屈伸的屈伸臂(222)，所述传感模块(24)包括两个角度传感器，所述两个角度传感器分别为采集所述收展臂(220)的收展角度的收展角度传感器(240)以及采集所述屈伸臂(222)的屈伸角度的屈伸角度传感器(242)。
  5. 根据权利要求4所述的手势采集系统，其特征在于：所述腕带(7)与腕部收展时的收展轴有两个对应所述收展臂(220)的交点，所述腕带(7)与腕部屈伸时的屈伸轴有两个对应所述屈伸臂(222)的交点，所述收展臂(220)与所述屈伸臂(222)均包括固定端及与所述固定端相对的自由端，所述收展臂(220)与所述屈伸臂(222)两者其中之一的固定端安装在所述腕带(7)上并位于对应两个交点的其中之一处且安装其对应的角度传感器，自由端沿着所述腕带(7)绕至所述收展臂(220)与所述屈伸臂(222)两者中的另一个对应的两个交点其中之处并与所述收展臂(220)与所述屈伸臂(222)两者中的另一个的固定端连接，所述收展臂(220)与所述屈伸臂(222)连接点具有一定夹角且安装另一个角度传感器，所述收展臂(220)与所述屈伸臂(222)两者中的另一个的自由端绕着所述腕带(7)设置并向所述腕带(7)的外侧延伸至与 腕部的近手的位置固定。
  6. 根据权利要求5所述的手势采集系统，其特征在于：所述屈伸角度传感器(242)沿着所述屈伸轴设置，所述收展角度传感器(240)沿着所述收展轴设置。
  7. 根据权利要求5所述的手势采集系统，其特征在于：所述收展臂(220)与所述屈伸臂(222)两者中的位于腕部的近手的位置的自由端绕至腕部的近手的位置的底部并安装所述主采集模块(11)。
  8. 根据权利要求3所述的手势采集系统，其特征在于：所述腕部姿态采集臂(22)为伸缩杆，所述腕部姿态采集臂(22)与所述腕带(7)连接，包括位于所述腕带(7)上方或侧方的第一端以及与所述手部固接的第二端，所述腕部姿态采集臂(22)与所述腕带(7)的交点位于所述第一端与所述第二端之间，所述第一端能够围绕所述交点摆动，所述传感模块(24)通过获取所述第一端的摆动信息获取所述腕部姿态采集臂(22)的屈伸信息与收展信息。
  9. 根据权利要求8所述的手势采集系统，其特征在于：所述腕带(7)上设有鱼眼轴承(75)，所述腕部姿态采集臂(22)通过穿设所述鱼眼轴承(75)的方式与所述腕部姿态采集臂(22)连接。
  10. 根据权利要求9所述的手势采集系统，其特征在于：所述传感模块(24)为霍尔传感器，所述腕部姿态采集臂(22)的第一端设有磁铁(26)，所述传感模块(24)通过获取所述第一端的磁场信息的方式，获取所述腕部姿态采集臂(22)的屈伸信息与收展信息。
  11. 根据权利要求10所述的手势采集系统，其特征在于：所述传感模块(24)位于所述腕部姿态采集臂(22)的延伸方向上，且相对所述腕部姿态采集臂(22)的第一端更远离所述腕部姿态采集臂(22)的第二端。
  12. 根据权利要求8所述的手势采集系统，其特征在于：还包括固定结构(8)，所述固定结构(8)固定在腕部近手的位置，所述腕部姿态采集臂(22)的第二端固定在所述固定结构(8)上，所述主采集模块(11)固定在所述固定结构(8)上。
  13. 根据权利要求3所述的手势采集系统，其特征在于：还包括安装在所述腕带(7)上的前臂姿态采集装置(3)，所述前臂姿态采集装置(3)用于采集前臂姿态。
  14. 根据权利要求13所述的手势采集系统，其特征在于：所述前臂姿态采集装置(3)包括三轴陀螺仪、三轴磁力计与三轴加速度传感器。
  15. 根据权利要求3所述的手势采集系统，其特征在于：所述腕带(7)呈套状，所述腕带(7)间隔地设有第一固定部(71)与第二固定部(72)，所述第一固定部(71)与第二固定部(72)分别与桡骨茎突和尺骨茎突的体表突出部分适配以将所述腕带(7)固定装设在手腕上。
  16. 根据权利要求15所述的手势采集系统，其特征在于：所述第一固定部(71)与第二固定部(72)为开设在所述腕带(7)上的通孔或者凹槽。
  17. 根据权利要求1-16任意一项所述的手势采集系统，其特征在于：还包括环境信息采集装置(4)，所述环境信息采集装置(4)用于在周围环境寻找需要控制的电子设备。
  18. 根据权利要求1-16任意一项所述的手势采集系统，其特征在于：还包括处理器(5)，所述处理器(5)与所述手势采集系统中的其他电子元件连接。
  19. 根据权利要求1-16任意一项所述的手势采集系统，其特征在于：所述主采集模块(11)为摄像头。
  20. 根据权利要求1-16任意一项所述的手势采集系统，其特征在于：所述主采集模块(11)的安装区域为腕关节向手指方向0～5厘米以内、手腕到手臂0～3厘米以内且距离体表皮肤3厘米以内。
PCT/CN2017/096362 2017-08-08 2017-08-08 手势采集系统 WO2019028650A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP17920688.3A EP3667564A4 (en) 2017-08-08 2017-08-08 GESTURE ACQUISITION SYSTEM
PCT/CN2017/096362 WO2019028650A1 (zh) 2017-08-08 2017-08-08 手势采集系统
US16/785,552 US11507192B2 (en) 2017-08-08 2020-02-07 Gesture acquisition system
US17/967,043 US20230195234A1 (en) 2017-08-08 2022-10-17 Gesture acquisition system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/096362 WO2019028650A1 (zh) 2017-08-08 2017-08-08 手势采集系统

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US16/785,552 Continuation-In-Part US11507192B2 (en) 2017-08-08 2020-02-07 Gesture acquisition system
US16/785,552 Continuation US11507192B2 (en) 2017-08-08 2020-02-07 Gesture acquisition system

Publications (1)

Publication Number Publication Date
WO2019028650A1 true WO2019028650A1 (zh) 2019-02-14

Family

ID=65272994

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/096362 WO2019028650A1 (zh) 2017-08-08 2017-08-08 手势采集系统

Country Status (3)

Country Link
US (2) US11507192B2 (zh)
EP (1) EP3667564A4 (zh)
WO (1) WO2019028650A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111158482A (zh) * 2019-12-30 2020-05-15 华中科技大学鄂州工业技术研究院 一种人体动作姿态捕捉方法及系统
CN111369626A (zh) * 2020-03-04 2020-07-03 刘东威 基于深度学习的无标志点上肢运动分析方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114052358B (zh) * 2021-11-05 2023-12-05 杭州中芯微电子有限公司 一种用于aoa多功能人员腕带式标签生产磁力扣安装设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103793057A (zh) * 2014-01-26 2014-05-14 华为终端有限公司 信息处理方法、装置及设备
CN104267807A (zh) * 2014-09-16 2015-01-07 华东理工大学 基于手部动作肌音信号的人机交互方法及交互系统
CN104856707A (zh) * 2015-05-14 2015-08-26 上海大学 基于机器视觉的压力传感数据手套及其抓握过程判断方法

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5313968A (en) * 1990-04-23 1994-05-24 Washington University Joint range of motion analyzer using euler angle
WO1992015247A1 (en) * 1991-03-07 1992-09-17 Exos, Incorporated Exoskeletal measurement system
US6104379A (en) * 1996-12-11 2000-08-15 Virtual Technologies, Inc. Forearm-supported exoskeleton hand-tracking device
US6110130A (en) * 1997-04-21 2000-08-29 Virtual Technologies, Inc. Exoskeleton device for directly measuring fingertip position and inferring finger joint angle
US6651352B2 (en) * 2002-01-04 2003-11-25 Liberty Mutual Wrist motion measurement device
ATE543132T1 (de) * 2006-07-16 2012-02-15 I Cherradi Freifinger-tipptechnologie
US10321873B2 (en) * 2013-09-17 2019-06-18 Medibotics Llc Smart clothing for ambulatory human motion capture
US10234941B2 (en) * 2012-10-04 2019-03-19 Microsoft Technology Licensing, Llc Wearable sensor for tracking articulated body-parts
US9389684B2 (en) * 2013-03-13 2016-07-12 Visual Music Systems, Inc. Platform for finger controls
US9213044B2 (en) * 2013-10-14 2015-12-15 Nokia Technologies Oy Deviational plane wrist input
DE112013007524T5 (de) * 2013-10-24 2016-08-04 Apple Inc. Armbandgerät-Eingabe mittels Handgelenkbewegung
US10488936B2 (en) * 2014-09-30 2019-11-26 Apple Inc. Motion and gesture input from a wearable device
WO2016079774A1 (en) * 2014-11-21 2016-05-26 Johri Abhishek System and method for data and command input
JP2018506773A (ja) * 2014-12-16 2018-03-08 ソマティクス， インコーポレイテッド ジェスチャベースの行動を監視し、それに影響を与える方法およびシステム
US10067564B2 (en) * 2015-08-11 2018-09-04 Disney Enterprises, Inc. Identifying hand gestures based on muscle movement in the arm
US11106273B2 (en) * 2015-10-30 2021-08-31 Ostendo Technologies, Inc. System and methods for on-body gestural interfaces and projection displays
KR102437106B1 (ko) * 2015-12-01 2022-08-26 삼성전자주식회사 마찰음을 이용하는 장치 및 방법
US10894204B2 (en) * 2016-05-04 2021-01-19 Contact Control Interfaces, LLC Exo-tendon motion capture glove device with haptic grip response
US10481699B2 (en) * 2017-07-27 2019-11-19 Facebook Technologies, Llc Armband for tracking hand motion using electrical impedance measurement

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103793057A (zh) * 2014-01-26 2014-05-14 华为终端有限公司 信息处理方法、装置及设备
CN104267807A (zh) * 2014-09-16 2015-01-07 华东理工大学 基于手部动作肌音信号的人机交互方法及交互系统
CN104856707A (zh) * 2015-05-14 2015-08-26 上海大学 基于机器视觉的压力传感数据手套及其抓握过程判断方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3667564A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111158482A (zh) * 2019-12-30 2020-05-15 华中科技大学鄂州工业技术研究院 一种人体动作姿态捕捉方法及系统
CN111158482B (zh) * 2019-12-30 2023-06-27 华中科技大学鄂州工业技术研究院 一种人体动作姿态捕捉方法及系统
CN111369626A (zh) * 2020-03-04 2020-07-03 刘东威 基于深度学习的无标志点上肢运动分析方法及系统

Also Published As

Publication number Publication date
US20230195234A1 (en) 2023-06-22
US11507192B2 (en) 2022-11-22
EP3667564A1 (en) 2020-06-17
US20200192487A1 (en) 2020-06-18
EP3667564A4 (en) 2021-04-07

Similar Documents

Publication Publication Date Title
JP6545258B2 (ja) スマートリング
CN110251080B (zh) 检测正在穿戴可穿戴电子设备的肢体
Loclair et al. PinchWatch: a wearable device for one-handed microinteractions
US10234941B2 (en) Wearable sensor for tracking articulated body-parts
US20230195234A1 (en) Gesture acquisition system
CN202584010U (zh) 一种腕上手势操控系统
US20140085177A1 (en) Method and apparatus for responding to input based upon relative finger position
CN103677265B (zh) 一种智能感应手套及智能感应方法
CN108958615A (zh) 一种显示控制方法、终端及计算机可读存储介质
WO2021004412A1 (zh) 手持输入设备及其指示图标的显示位置控制方法和装置
CN110348275A (zh) 手势识别方法、装置、智能设备及计算机可读存储介质
CN107301415B (zh) 手势采集系统
CN207704451U (zh) 手势采集系统
CN106020442A (zh) 一种智能感应手套的感应方法
Hoshino Hand gesture interface for entertainment games
CN107403178A (zh) 手势采集系统
CN207817740U (zh) 手势采集系统
US20240144727A1 (en) Multiview depth-sensing wearable device
CN103645803A (zh) 微型便携式多功能无线体感捕捉控制系统
JP2015106255A (ja) 表示装置およびプログラム
CN110012325A (zh) 一种电视机及其控制系统
Dai et al. Design of intelligent mobile robot control system based on gesture recognition
CN113268136B (zh) 拇指与手掌间自由度解算方法、装置及数据手套
CN203606782U (zh) 微型便携式多功能无线体感捕捉控制系统
WO2022228056A1 (zh) 一种人机交互方法及设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17920688

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2017920688

Country of ref document: EP

Effective date: 20200309