CN107145236B - Gesture recognition method and system based on wrist tendon pressure related characteristics - Google Patents
Gesture recognition method and system based on wrist tendon pressure related characteristics Download PDFInfo
- Publication number
- CN107145236B CN107145236B CN201710334899.4A CN201710334899A CN107145236B CN 107145236 B CN107145236 B CN 107145236B CN 201710334899 A CN201710334899 A CN 201710334899A CN 107145236 B CN107145236 B CN 107145236B
- Authority
- CN
- China
- Prior art keywords
- controlling
- pressure
- wrist
- identification result
- lighting equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 210000000707 wrist Anatomy 0.000 title claims abstract description 72
- 210000002435 tendon Anatomy 0.000 title claims abstract description 39
- 238000000034 method Methods 0.000 title claims abstract description 26
- 210000003811 finger Anatomy 0.000 claims description 78
- 230000008859 change Effects 0.000 claims description 46
- 210000003813 thumb Anatomy 0.000 claims description 18
- 238000004378 air conditioning Methods 0.000 claims description 12
- 230000006870 function Effects 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 12
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N Silicium dioxide Chemical compound O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 claims description 9
- 239000000741 silica gel Substances 0.000 claims description 9
- 229910002027 silica gel Inorganic materials 0.000 claims description 9
- 230000005540 biological transmission Effects 0.000 claims description 8
- 238000010801 machine learning Methods 0.000 claims description 7
- 210000005224 forefinger Anatomy 0.000 claims description 6
- 210000004932 little finger Anatomy 0.000 claims description 6
- 239000010408 film Substances 0.000 claims 1
- 229920001296 polysiloxane Polymers 0.000 claims 1
- 239000010409 thin film Substances 0.000 claims 1
- 239000000284 extract Substances 0.000 abstract description 5
- 238000002474 experimental method Methods 0.000 abstract description 3
- 210000003205 muscle Anatomy 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 210000000245 forearm Anatomy 0.000 description 3
- 230000036962 time dependent Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000004962 physiological condition Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
- 210000002027 skeletal muscle Anatomy 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B15/00—Systems controlled by a computer
- G05B15/02—Systems controlled by a computer electric
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B19/00—Programme-control systems
- G05B19/02—Programme-control systems electric
- G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
- G05B2219/00—Program-control systems
- G05B2219/20—Pc systems
- G05B2219/26—Pc applications
- G05B2219/2642—Domotique, domestic, home control, automation, smart house
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Theoretical Computer Science (AREA)
- Manufacturing & Machinery (AREA)
- Quality & Reliability (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a gesture recognition method and system based on the pressure-related characteristics of wrist tendons, which do not need complex external equipment, and can realize high-precision and various gesture recognition only by wearing a bracelet provided with a small number of pressure sensors arranged in double rows on the wrist of a user; on one hand, the convenience of gesture recognition is greatly improved, and the method is suitable for various occasions. On the other hand, the scheme performs optimal selection based on human body structure on the position where the pressure sensor is placed, extracts more effective characteristics on the collected pressure information, does not directly perform pressure value matching, and selects a more advanced algorithm, so that the gesture recognition precision is greatly improved. In the current experiment, the recognition precision of twenty-four gestures can reach more than 95% best.
Description
Technical Field
The invention relates to the technical field of human-computer interaction, in particular to a gesture recognition method and system based on wrist tendon pressure related characteristics.
Background
Gesture recognition refers to classifying gesture types by using a certain technology. The existing gesture recognition method comprises video-based gesture recognition, namely, a camera is used for acquiring image information of a hand of a person, and then features of different gestures are extracted by using an image processing method for gesture recognition; the gesture recognition based on the electromyographic signals (EMG) is also used for gesture classification by acquiring the change situation of the electromyographic signals in different gestures in a mode of arranging the electromyographic sensors on the arms of the human body in a large scale; in addition, the gesture recognition based on ultrasonic waves is also provided, the ultrasonic images of the skeletal muscles of the human arms are obtained through ultrasonic sensing, and then the ultrasonic images are classified in an image processing mode, so that the gesture recognition is realized; in addition, there is gesture recognition using a large-scale pressure sensor array, which uses a large number of pressure sensors to collect data and then classifies gestures by means of pressure value set matching or pressure value curve matching.
The existing mainstream method has the following defects:
the gesture recognition scheme based on the video needs to be used in a scene with a camera, so that the use range of the gesture recognition scheme is greatly limited, and the gesture recognition effect is also influenced by conditions such as illumination of a shooting scene; the gesture recognition scheme based on the electromyographic signals needs to arrange a large number of sensor electrodes on the forearm of a human body, which can seriously affect the use experience of a user and greatly limit the application scene; the gesture recognition scheme based on ultrasonic waves requires the use of a special ultrasonic probe and matched processing equipment, is high in cost, needs professional technical support, and is not favorable for large-scale popularization. The gesture recognition scheme based on the large-scale pressure sensor array requires a large number of pressure sensors, so the cost thereof will be very high, and in addition, the method of performing gesture recognition by using direct matching of pressure values or pressure curves will result in low measurement accuracy.
Disclosure of Invention
The invention aims to provide a gesture recognition method and system based on the pressure related characteristics of wrist tendons, which can realize high-precision and various gesture recognition only by wearing a bracelet provided with a small number of pressure sensors arranged in double rows on the wrist of a user, and are low in cost.
The purpose of the invention is realized by the following technical scheme:
a gesture recognition method based on wrist tendon pressure related characteristics comprises the following steps:
acquiring pressure value data acquired by N pressure sensors, and arranging the pressure value data according to a specific mode; each pressure sensor collects pressure data of a specific position of a wrist of a user;
calculating time-based correlation information and space-based correlation information of the pressure change condition of the tendons of the wrist of the user when the gesture changes according to the arrayed N pressure value data, and taking the time-based correlation information and the space-based correlation information as feature information to be recognized;
and analyzing and judging the gesture type most matched with the characteristic information to be recognized by using a machine learning method according to the preset corresponding relation between the characteristic information and the gesture type, thereby finishing gesture recognition.
N pressure sensor is film pressure sensor, distributes and takes elastic insulating bracelet on to use the silica gel pad to support.
The N pressure sensors are arranged in a double-row mode and correspond to the positions of the tendons of the wrist of the user, and the change conditions of the tendons of the wrist of the user are fitted according to the correlation conditions of pressure information among the sensors.
The method further comprises the following steps:
triggering a corresponding control function according to the gesture recognition result, which comprises:
if the identification result is that the thumb is bent, controlling to close the lighting equipment in the living room;
if the recognition result is that the index finger is bent, controlling to close the lighting equipment of the study room;
if the identification result is that the middle finger is bent, controlling to turn off the lighting equipment of the kitchen;
if the identification result is that the ring finger is bent, controlling to turn off the lighting equipment of the master bedroom;
if the identification result is that the little finger is bent, controlling to close the lighting equipment of the secondary bedroom;
if the identification result is that the fist is closed, controlling the lighting equipment to be completely closed;
if the recognition result is that the forefinger is pinched, controlling the electronic lock of the gate to be locked;
if the identification result is that the middle finger is pinched, controlling the electronic lock of the garage gate to be locked;
if the recognition result is that the ring finger is kneaded, controlling the electronic lock of the back door of the house to be closed;
if the recognition result is that the thumb is kneaded, controlling all windows to be closed and locked;
if the identification result is that the thumb is extended, controlling to turn on the lighting equipment of the living room;
if the recognition result is that the index finger extends, controlling to open the lighting equipment of the study room;
if the identification result is that the two fingers are extended, controlling to turn on the lighting equipment of the kitchen;
if the identification result is that the three fingers are extended, controlling to turn on the lighting equipment of the master bedroom;
if the recognition result is that the four fingers are extended, controlling to turn on the lighting equipment of the secondary bedroom;
if the identification result is that the five fingers are extended, controlling to open the music player and play music, wherein the operation mode is the same as that of closing at the last time;
if the identification result is that the index finger is opened, controlling the inner air conditioning system to start working, wherein the operation mode is the same as that of the last closing;
if the recognition result is that the index finger flicks the finger, controlling the inner air-conditioning system to stop working and entering a sleep mode;
if the recognition result is that the palm is right-side-placed, controlling the music player to switch the currently played song to be the next song on the list;
if the recognition result is that the palm is left-placed, controlling the music player to switch the currently played song to be the last song on the list;
if the recognition result is that the palm swings upwards, the air purifier is controlled to be opened and starts to operate;
if the recognition result is that the palm swings downwards, controlling the air purifier to stop running and entering a dormant state;
if the identification result is that the wrist turns left, the control module is controlled to be started, and the system enters an active state;
if the identification result is that the wrist turns right, the control module is controlled to be closed, and the system enters a dormant state.
The time-dependent information of the pressure change condition comprises: dividing the variance of the pressure values of the current pressure sensor when the gesture changes by the mean value of the pressure values, and taking the result as the intensity of the pressure change of the position corresponding to the current pressure sensor under the condition of the gesture change; the mean of the first order differences of the pressure value data characterizes the trend of the pressure change.
The spatial correlation information of the pressure change condition comprises: and a covariance matrix of a time correlation matrix between every two pressure sensors represents the spatial correlation of the pressure of a specific position of the wrist when the gesture changes.
A gesture recognition system based on a wrist tendon pressure-related characteristic, comprising:
the pressure sensing module comprises N pressure sensors and is used for acquiring pressure data of a specific position of the wrist of a user acquired by each pressure sensor;
the signal acquisition and transmission module is used for acquiring pressure value data acquired by the N pressure sensors, and sending the pressure value data to the outside in a wireless transmission mode after the pressure value data are arranged in a specific mode;
the data processing and identifying module is used for calculating time-based correlation information and space-based correlation information of the pressure change condition of the wrist tendon of the user when the gesture changes according to the arrayed N pressure value data, and using the time-based correlation information and the space-based correlation information as feature information to be identified; and analyzing and judging the gesture type most matched with the characteristic information to be recognized by using a machine learning method according to the corresponding relation between the preset characteristic information and the gesture type, thereby finishing gesture recognition.
The pressure sensing module also comprises an insulating bracelet and a silica gel pad;
n pressure sensor is film pressure sensor, distributes and takes elastic insulating bracelet on to use the silica gel pad to support.
The N pressure sensors are arranged in a double-row mode and correspond to the positions of the tendons of the wrist of the user, and the change conditions of the tendons of the wrist of the user are fitted according to the correlation conditions of pressure information among the sensors.
The system further comprises: the control module is used for triggering corresponding control functions according to the gesture recognition result and comprises:
if the identification result is that the thumb is bent, controlling to close the lighting equipment in the living room;
if the recognition result is that the index finger is bent, controlling to close the lighting equipment of the study room;
if the identification result is that the middle finger is bent, controlling to turn off the lighting equipment of the kitchen;
if the identification result is that the ring finger is bent, controlling to turn off the lighting equipment of the master bedroom;
if the identification result is that the little finger is bent, controlling to close the lighting equipment of the secondary bedroom;
if the identification result is that the fist is closed, controlling the lighting equipment to be completely closed;
if the recognition result is that the forefinger is pinched, controlling the electronic lock of the gate to be locked;
if the identification result is that the middle finger is pinched, controlling the electronic lock of the garage gate to be locked;
if the recognition result is that the ring finger is kneaded, controlling the electronic lock of the back door of the house to be closed;
if the recognition result is that the thumb is kneaded, controlling all windows to be closed and locked;
if the identification result is that the thumb is extended, controlling to turn on the lighting equipment of the living room;
if the recognition result is that the index finger extends, controlling to open the lighting equipment of the study room;
if the identification result is that the two fingers are extended, controlling to turn on the lighting equipment of the kitchen;
if the identification result is that the three fingers are extended, controlling to turn on the lighting equipment of the master bedroom;
if the recognition result is that the four fingers are extended, controlling to turn on the lighting equipment of the secondary bedroom;
if the identification result is that the five fingers are extended, controlling to open the music player and play music, wherein the operation mode is the same as that of closing at the last time;
if the identification result is that the index finger is opened, controlling the inner air conditioning system to start working, wherein the operation mode is the same as that of the last closing;
if the recognition result is that the index finger flicks the finger, controlling the inner air-conditioning system to stop working and entering a sleep mode;
if the recognition result is that the palm is right-side-placed, controlling the music player to switch the currently played song to be the next song on the list;
if the recognition result is that the palm is left-placed, controlling the music player to switch the currently played song to be the last song on the list;
if the recognition result is that the palm swings upwards, the air purifier is controlled to be opened and starts to operate;
if the recognition result is that the palm swings downwards, controlling the air purifier to stop running and entering a dormant state;
if the identification result is that the wrist turns left, the control module is started, and the system enters an active state;
if the identification result is that the wrist turns right, the control module is closed, and the system enters a dormant state.
The time-dependent information of the pressure change condition comprises: dividing the variance of the pressure values of the current pressure sensor when the gesture changes by the mean value of the pressure values, and taking the result as the intensity of the pressure change of the position corresponding to the current pressure sensor under the condition of the gesture change; the mean of the first order differences of the pressure value data characterizes the trend of the pressure change.
The spatial correlation information of the pressure change condition comprises: and a covariance matrix of a time correlation matrix between every two pressure sensors represents the spatial correlation of the pressure of a specific position of the wrist when the gesture changes.
According to the technical scheme provided by the invention, complex external equipment is not needed, and high-precision and various gesture recognition can be realized only by wearing a bracelet provided with a small number of pressure sensors arranged in double rows on the wrist of a user; on one hand, the convenience of gesture recognition is greatly improved, and the method is suitable for various occasions. On the other hand, the scheme performs optimal selection based on human body structure on the position where the pressure sensor is placed, extracts more effective characteristics on the collected pressure information, does not directly perform pressure value matching, and selects a more advanced algorithm, so that the gesture recognition precision is greatly improved. In the current experiment, the recognition precision of twenty-four gestures can reach more than 95% best.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a gesture recognition method based on a pressure-related characteristic of a wrist tendon according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of sixteen pressure sensors arranged in a double row according to an embodiment of the present invention;
fig. 3 is an external view of a pressure bracelet composed of sixteen pressure sensors according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a gesture recognition system based on the pressure-related characteristics of the tendons of the wrist according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a refined gesture recognition system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
When the hand exercise is completed, most of people are driven by muscles located on the forearms of the arms, and the muscles of the forearms are connected with the muscles of the hands through tendons at the wrists. When the palm of a hand is in different states, namely, when the hand performs different gestures, the state of tendons at the wrist is different, and the pressure on the wrist is different. Therefore, if pressure values given to the bracelet under different gestures of the wrist tendon of the user can be obtained, relevant change information can be extracted from the pressure values to serve as features, and the type of the gesture is determined through a relevant algorithm. Specifically, the embodiment of the present invention provides a gesture recognition method based on the pressure-related characteristics of the tendons of the wrist, as shown in fig. 1, the method mainly includes the following steps:
In the embodiment of the invention, each sensor can send out the corresponding digital signal according to the magnitude of the current pressure value, and the collected information of the N pressure sensors is transmitted outwards one by one in a polling mode.
In the embodiment of the present invention, the number of the pressure sensors may be 16, the arrangement is as shown in fig. 2, the positions where the pressure sensors 22 and 23 are placed (the pressure sensors 22 and 23 have no structural difference, but the positions are different and respectively represent the positions distributed at one end near the arm and the positions distributed at one end near the palm) need to take into account the influence of the wrist muscle 21, the wrist skeleton 24 and the wrist tendon 25, and meanwhile, when the gesture changes, the pressure change at the wrist tendon of the human hand is a change in a two-dimensional plane rather than a one-dimensional linear change. Therefore, the embodiment of the invention selects to arrange the pressure sensors in double rows, and the pressure sensors respectively correspond to the positions of the selected wrist tendons, and the variation condition of the wrist tendons is fitted according to the correlation condition of the pressure information among the sensors.
All pressure sensors are film pressure sensors, as shown in fig. 3, distributed on an insulating bracelet 32 with elasticity, and the pressure sensor 33 is supported by using a silica gel pad 31, and the pressure sensor 33 is supported by using the silica gel pad 31 to be just attached to the wrist skin of the user.
And step 12, calculating time-based correlation information and space-based correlation information of the pressure change condition of the wrist tendon of the user when the gesture changes according to the arranged N pressure value data, and taking the time-based correlation information and the space-based correlation information as feature information to be recognized.
In an embodiment of the present invention, the time-based correlation information of the pressure change condition includes: dividing the variance of the pressure values of the current pressure sensor when the gesture changes by the mean value of the pressure values, and taking the result as the intensity of the pressure change of the position corresponding to the current pressure sensor under the condition of the gesture change; the mean of the first order differences of the pressure value data characterizes the trend of the pressure change.
The spatial correlation information of the pressure change condition comprises: and a covariance matrix of a time correlation matrix between every two pressure sensors represents the spatial correlation of the pressure of a specific position of the wrist when the gesture changes.
And step 13, analyzing and judging the gesture type most matched with the characteristic information to be recognized by using a machine learning method according to the corresponding relation between the preset characteristic information and the gesture type, thereby completing gesture recognition.
Since the physiological conditions of the user group such as sex, age, weight and the like are different, and the thickness of the wrist and the position of the pressure to be measured are different, in order to obtain higher accuracy of gesture recognition, the method also comprises the process of collecting standard gesture characteristic information (namely preset characteristic information) of the user and the recording step:
1. the user finishes wearing the bracelet according to the self condition.
2. And the user makes gesture actions required by the system according to the system prompt.
3. The system completes the collection of pressure data when the user completes the gesture action.
4. The system extracts characteristic information from the acquired pressure data and stores the corresponding relation between the characteristic information and the gesture type.
For example, the method of machine learning is currently best performed by the K-nearest neighbor method, and the methods used include, but are not limited to, a perceptron, a support vector machine, and the like.
And 14, triggering a corresponding control function according to the gesture recognition result.
The control function is realized through a control module, the control module can be an intelligent home system, and the following control functions can be triggered according to a gesture recognition result:
if the identification result is that the thumb is bent, controlling to close the lighting equipment in the living room;
if the recognition result is that the index finger is bent, controlling to close the lighting equipment of the study room;
if the identification result is that the middle finger is bent, controlling to turn off the lighting equipment of the kitchen;
if the identification result is that the ring finger is bent, controlling to turn off the lighting equipment of the master bedroom;
if the identification result is that the little finger is bent, controlling to close the lighting equipment of the secondary bedroom;
if the identification result is that the fist is closed, controlling the lighting equipment to be completely closed;
if the recognition result is that the forefinger is pinched, controlling the electronic lock of the gate to be locked;
if the identification result is that the middle finger is pinched, controlling the electronic lock of the garage gate to be locked;
if the recognition result is that the ring finger is kneaded, controlling the electronic lock of the back door of the house to be closed;
if the recognition result is that the thumb is kneaded, controlling all windows to be closed and locked;
if the identification result is that the thumb is extended, controlling to turn on the lighting equipment of the living room;
if the recognition result is that the index finger extends, controlling to open the lighting equipment of the study room;
if the identification result is that the two fingers are extended, controlling to turn on the lighting equipment of the kitchen;
if the identification result is that the three fingers are extended, controlling to turn on the lighting equipment of the master bedroom;
if the recognition result is that the four fingers are extended, controlling to turn on the lighting equipment of the secondary bedroom;
if the identification result is that the five fingers are extended, controlling to open the music player and play music, wherein the operation mode is the same as that of closing at the last time;
if the identification result is that the index finger is opened, controlling the inner air conditioning system to start working, wherein the operation mode is the same as that of the last closing;
if the recognition result is that the index finger flicks the finger, controlling the inner air-conditioning system to stop working and entering a sleep mode;
if the recognition result is that the palm is right-side-placed, controlling the music player to switch the currently played song to be the next song on the list;
if the recognition result is that the palm is left-placed, controlling the music player to switch the currently played song to be the last song on the list;
if the recognition result is that the palm swings upwards, the air purifier is controlled to be opened and starts to operate;
if the recognition result is that the palm swings downwards, controlling the air purifier to stop running and entering a dormant state;
if the identification result is that the wrist turns left, the control module is controlled to be started, and the system enters an active state;
if the identification result is that the wrist turns right, the control module is controlled to be closed, and the system enters a dormant state.
The embodiment of the invention provides a scheme different from other schemes using pressure sensors, the scheme only uses a small number of pressure sensors, gives the positions of the pressure sensors, extracts more effective characteristics from pressure value information of the pressure sensors, uses a more efficient algorithm, and realizes that the gesture type identification with higher precision is completed by using a small number of pressure sensors.
According to the recognition of the gesture types of the user and the preset corresponding relation between the gesture types and the operation, the control instruction can be sent to the control module through the wireless module of the bracelet, and the function of remotely controlling the electronic equipment can be achieved. The bracelet easily carries, has good characteristics such as convenience, can be used for intelligent home systems's control.
The correspondence between the types of the user's partial gestures and the commonly used control information is shown in table 1:
TABLE 1 correspondence between partial user gestures and commonly used control information
According to the technical scheme of the embodiment of the invention, complex external equipment is not needed, and high-precision and various gesture recognition can be realized only by wearing a bracelet provided with a small number of pressure sensors arranged in double rows on the wrist of a user; on one hand, the convenience of gesture recognition is greatly improved, and the method is suitable for various occasions. On the other hand, the scheme performs optimal selection based on human body structure on the position where the pressure sensor is placed, extracts more effective characteristics on the collected pressure information, does not directly perform pressure value matching, and selects a more advanced algorithm, so that the gesture recognition precision is greatly improved. In the current experiment, the recognition precision of twenty-four gestures can reach more than 95% best.
Through the above description of the embodiments, it is clear to those skilled in the art that the above embodiments can be implemented by software, and can also be implemented by software plus a necessary general hardware platform. With this understanding, the technical solutions of the embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.), and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods according to the embodiments of the present invention.
Another embodiment of the present invention further provides a gesture recognition system based on the pressure-related characteristics of the tendons of the wrist, which can implement the solutions described in the above embodiments, as shown in fig. 4, the system mainly includes:
the pressure sensing module comprises N pressure sensors and is used for acquiring pressure data of a specific position of the wrist of a user acquired by each pressure sensor;
the signal acquisition and transmission module is used for acquiring pressure value data acquired by the N pressure sensors, and sending the pressure value data to the outside in a wireless transmission mode after the pressure value data are arranged in a specific mode;
the data processing and identifying module is used for calculating time-based correlation information and space-based correlation information of the pressure change condition of the wrist tendon of the user when the gesture changes according to the arrayed N pressure value data, and using the time-based correlation information and the space-based correlation information as feature information to be identified; and analyzing and judging the gesture type most matched with the characteristic information to be recognized by using a machine learning method according to the corresponding relation between the preset characteristic information and the gesture type, thereby finishing gesture recognition.
Furthermore, the pressure sensing module also comprises an insulating bracelet and a silica gel pad;
n pressure sensor is film pressure sensor, distributes and takes elastic insulating bracelet on to use the silica gel pad to support.
Furthermore, the N pressure sensors are arranged in a double-row mode and respectively correspond to the positions of the tendons of the wrist of the user, and the change conditions of the tendons of the wrist of the user are fitted according to the correlation conditions of the pressure information among the sensors.
Further, the system further comprises: the control module is used for triggering corresponding control functions according to the gesture recognition result and comprises:
if the identification result is that the thumb is bent, controlling to close the lighting equipment in the living room;
if the recognition result is that the index finger is bent, controlling to close the lighting equipment of the study room;
if the identification result is that the middle finger is bent, controlling to turn off the lighting equipment of the kitchen;
if the identification result is that the ring finger is bent, controlling to turn off the lighting equipment of the master bedroom;
if the identification result is that the little finger is bent, controlling to close the lighting equipment of the secondary bedroom;
if the identification result is that the fist is closed, controlling the lighting equipment to be completely closed;
if the recognition result is that the forefinger is pinched, controlling the electronic lock of the gate to be locked;
if the identification result is that the middle finger is pinched, controlling the electronic lock of the garage gate to be locked;
if the recognition result is that the ring finger is kneaded, controlling the electronic lock of the back door of the house to be closed;
if the recognition result is that the thumb is kneaded, controlling all windows to be closed and locked;
if the identification result is that the thumb is extended, controlling to turn on the lighting equipment of the living room;
if the recognition result is that the index finger extends, controlling to open the lighting equipment of the study room;
if the identification result is that the two fingers are extended, controlling to turn on the lighting equipment of the kitchen;
if the identification result is that the three fingers are extended, controlling to turn on the lighting equipment of the master bedroom;
if the recognition result is that the four fingers are extended, controlling to turn on the lighting equipment of the secondary bedroom;
if the identification result is that the five fingers are extended, controlling to open the music player and play music, wherein the operation mode is the same as that of closing at the last time;
if the identification result is that the index finger is opened, controlling the inner air conditioning system to start working, wherein the operation mode is the same as that of the last closing;
if the recognition result is that the index finger flicks the finger, controlling the inner air-conditioning system to stop working and entering a sleep mode;
if the recognition result is that the palm is right-side-placed, controlling the music player to switch the currently played song to be the next song on the list;
if the recognition result is that the palm is left-placed, controlling the music player to switch the currently played song to be the last song on the list;
if the recognition result is that the palm swings upwards, the air purifier is controlled to be opened and starts to operate;
if the recognition result is that the palm swings downwards, controlling the air purifier to stop running and entering a dormant state;
if the identification result is that the wrist turns left, the control module is started, and the system enters an active state;
if the identification result is that the wrist turns right, the control module is closed, and the system enters a dormant state.
Further, the time-dependent information of the pressure change condition includes: dividing the variance of the pressure values of the current pressure sensor when the gesture changes by the mean value of the pressure values, and taking the result as the intensity of the pressure change of the position corresponding to the current pressure sensor under the condition of the gesture change; the mean of the first order differences of the pressure value data characterizes the trend of the pressure change.
The spatial correlation information of the pressure change condition comprises: and a covariance matrix of a time correlation matrix between every two pressure sensors represents the spatial correlation of the pressure of a specific position of the wrist when the gesture changes.
The modules in the system according to the embodiment of the present invention may be further specifically subdivided into the structure shown in fig. 5: the double-row pressure sensor, the multiplexer and the A/D module are pressure sensing modules, the single chip microcomputer signal acquisition unit and the wireless transmission module are signal acquisition and transmission modules, and the peripheral equipment control module is a control module; in addition, the corresponding recognition result can also be displayed through the display module.
It should be noted that, specific implementation manners of functions implemented by the functional modules included in the system are described in detail in the foregoing embodiments, and therefore, detailed descriptions thereof are omitted here.
It will be clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the system is divided into different functional modules to perform all or part of the above described functions.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (6)
1. A gesture recognition method based on wrist tendon pressure related characteristics is characterized by comprising the following steps:
acquiring pressure value data acquired by N pressure sensors, and arranging the pressure value data according to a specific mode; each pressure sensor collects pressure data of a specific position of a wrist of a user; the N pressure sensors are arranged in a double-row mode, correspond to the positions of the tendons of the wrist of the user and are matched with the change conditions of the tendons of the wrist of the user according to the correlation conditions of pressure information among the sensors;
calculating time-based correlation information and space-based correlation information of the pressure change condition of the tendons of the wrist of the user when the gesture changes according to the arrayed N pressure value data, and taking the time-based correlation information and the space-based correlation information as feature information to be recognized;
according to the preset corresponding relation between the characteristic information and the gesture type, analyzing and judging the gesture type most matched with the characteristic information to be recognized by using a machine learning method, thereby completing gesture recognition;
wherein the time-wise correlation information of the pressure change condition includes: dividing the variance of the pressure values of the current pressure sensor when the gesture changes by the mean value of the pressure values, and taking the result as the intensity of the pressure change of the position corresponding to the current pressure sensor under the condition of the gesture change; the mean value of the first order difference of the pressure value data represents the trend of pressure change;
the spatial correlation information of the pressure change condition comprises: and a covariance matrix of a time correlation matrix between every two pressure sensors represents the spatial correlation of the pressure of a specific position of the wrist when the gesture changes.
2. The method for recognizing gestures based on the pressure related characteristics of the tendons of the wrist according to claim 1, wherein the N pressure sensors are all thin film pressure sensors, distributed on an insulating bracelet with elasticity, and supported by a silica gel pad.
3. The method of claim 1, wherein the method further comprises:
triggering a corresponding control function according to the gesture recognition result, which comprises:
if the identification result is that the thumb is bent, controlling to close the lighting equipment in the living room;
if the recognition result is that the index finger is bent, controlling to close the lighting equipment of the study room;
if the identification result is that the middle finger is bent, controlling to turn off the lighting equipment of the kitchen;
if the identification result is that the ring finger is bent, controlling to turn off the lighting equipment of the master bedroom;
if the identification result is that the little finger is bent, controlling to close the lighting equipment of the secondary bedroom;
if the identification result is that the fist is closed, controlling the lighting equipment to be completely closed;
if the recognition result is that the forefinger is pinched, controlling the electronic lock of the gate to be locked;
if the identification result is that the middle finger is pinched, controlling the electronic lock of the garage gate to be locked;
if the recognition result is that the ring finger is kneaded, controlling the electronic lock of the back door of the house to be closed;
if the recognition result is that the thumb is kneaded, controlling all windows to be closed and locked;
if the identification result is that the thumb is extended, controlling to turn on the lighting equipment of the living room;
if the recognition result is that the index finger extends, controlling to open the lighting equipment of the study room;
if the identification result is that the two fingers are extended, controlling to turn on the lighting equipment of the kitchen;
if the identification result is that the three fingers are extended, controlling to turn on the lighting equipment of the master bedroom;
if the recognition result is that the four fingers are extended, controlling to turn on the lighting equipment of the secondary bedroom;
if the identification result is that the five fingers are extended, controlling to open the music player and play music, wherein the operation mode is the same as that of closing at the last time;
if the identification result is that the index finger is opened, controlling the inner air conditioning system to start working, wherein the operation mode is the same as that of the last closing;
if the recognition result is that the index finger flicks the finger, controlling the inner air-conditioning system to stop working and entering a sleep mode;
if the recognition result is that the palm is right-side-placed, controlling the music player to switch the currently played song to be the next song on the list;
if the recognition result is that the palm is left-placed, controlling the music player to switch the currently played song to be the last song on the list;
if the recognition result is that the palm swings upwards, the air purifier is controlled to be opened and starts to operate;
if the recognition result is that the palm swings downwards, controlling the air purifier to stop running and entering a dormant state;
if the identification result is that the wrist turns left, the control module is controlled to be started, and the system enters an active state;
if the identification result is that the wrist turns right, the control module is controlled to be closed, and the system enters a dormant state.
4. A system for gesture recognition based on a pressure-related characteristic of a wrist tendon, comprising:
the pressure sensing module comprises N pressure sensors and is used for acquiring pressure data of a specific position of the wrist of a user acquired by each pressure sensor; the N pressure sensors are arranged in a double-row mode, correspond to the positions of the tendons of the wrist of the user and are matched with the change conditions of the tendons of the wrist of the user according to the correlation conditions of pressure information among the sensors;
the signal acquisition and transmission module is used for acquiring pressure value data acquired by the N pressure sensors, and sending the pressure value data to the outside in a wireless transmission mode after the pressure value data are arranged in a specific mode;
the data processing and identifying module is used for calculating time-based correlation information and space-based correlation information of the pressure change condition of the wrist tendon of the user when the gesture changes according to the arrayed N pressure value data, and using the time-based correlation information and the space-based correlation information as feature information to be identified; then, according to the corresponding relation between the preset feature information and the gesture type, analyzing and judging the gesture type which is most matched with the feature information to be recognized by using a machine learning method, thereby completing gesture recognition;
wherein the time-wise correlation information of the pressure change condition includes: dividing the variance of the pressure values of the current pressure sensor when the gesture changes by the mean value of the pressure values, and taking the result as the intensity of the pressure change of the position corresponding to the current pressure sensor under the condition of the gesture change; the mean value of the first order difference of the pressure value data represents the trend of pressure change;
the spatial correlation information of the pressure change condition comprises: and a covariance matrix of a time correlation matrix between every two pressure sensors represents the spatial correlation of the pressure of a specific position of the wrist when the gesture changes.
5. The system for recognizing gestures based on the pressure related characteristics of the tendons of the wrist according to claim 4, wherein the pressure sensing module further comprises an insulating bracelet and a silicone pad;
n pressure sensor is film pressure sensor, distributes and takes elastic insulating bracelet on to use the silica gel pad to support.
6. A gesture recognition system based on wrist tendon pressure-related characteristics according to claim 4,
the system further comprises: the control module is used for triggering corresponding control functions according to the gesture recognition result and comprises:
if the identification result is that the thumb is bent, controlling to close the lighting equipment in the living room;
if the recognition result is that the index finger is bent, controlling to close the lighting equipment of the study room;
if the identification result is that the middle finger is bent, controlling to turn off the lighting equipment of the kitchen;
if the identification result is that the ring finger is bent, controlling to turn off the lighting equipment of the master bedroom;
if the identification result is that the little finger is bent, controlling to close the lighting equipment of the secondary bedroom;
if the identification result is that the fist is closed, controlling the lighting equipment to be completely closed;
if the recognition result is that the forefinger is pinched, controlling the electronic lock of the gate to be locked;
if the identification result is that the middle finger is pinched, controlling the electronic lock of the garage gate to be locked;
if the recognition result is that the ring finger is kneaded, controlling the electronic lock of the back door of the house to be closed;
if the recognition result is that the thumb is kneaded, controlling all windows to be closed and locked;
if the identification result is that the thumb is extended, controlling to turn on the lighting equipment of the living room;
if the recognition result is that the index finger extends, controlling to open the lighting equipment of the study room;
if the identification result is that the two fingers are extended, controlling to turn on the lighting equipment of the kitchen;
if the identification result is that the three fingers are extended, controlling to turn on the lighting equipment of the master bedroom;
if the recognition result is that the four fingers are extended, controlling to turn on the lighting equipment of the secondary bedroom;
if the identification result is that the five fingers are extended, controlling to open the music player and play music, wherein the operation mode is the same as that of closing at the last time;
if the identification result is that the index finger is opened, controlling the inner air conditioning system to start working, wherein the operation mode is the same as that of the last closing;
if the recognition result is that the index finger flicks the finger, controlling the inner air-conditioning system to stop working and entering a sleep mode;
if the recognition result is that the palm is right-side-placed, controlling the music player to switch the currently played song to be the next song on the list;
if the recognition result is that the palm is left-placed, controlling the music player to switch the currently played song to be the last song on the list;
if the recognition result is that the palm swings upwards, the air purifier is controlled to be opened and starts to operate;
if the recognition result is that the palm swings downwards, controlling the air purifier to stop running and entering a dormant state;
if the identification result is that the wrist turns left, the control module is started, and the system enters an active state;
if the identification result is that the wrist turns right, the control module is closed, and the system enters a dormant state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710334899.4A CN107145236B (en) | 2017-05-12 | 2017-05-12 | Gesture recognition method and system based on wrist tendon pressure related characteristics |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710334899.4A CN107145236B (en) | 2017-05-12 | 2017-05-12 | Gesture recognition method and system based on wrist tendon pressure related characteristics |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107145236A CN107145236A (en) | 2017-09-08 |
| CN107145236B true CN107145236B (en) | 2020-02-07 |
Family
ID=59777550
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710334899.4A Active CN107145236B (en) | 2017-05-12 | 2017-05-12 | Gesture recognition method and system based on wrist tendon pressure related characteristics |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107145236B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019074978A1 (en) * | 2017-10-09 | 2019-04-18 | TacSense, Inc. | Body motion and position sensing, recognition and analytics from an array of wearable pressure sensors |
| CN108227924A (en) * | 2018-01-02 | 2018-06-29 | 联想(北京)有限公司 | For the method and electronic equipment of electronic equipment |
| CN110134225B (en) * | 2018-02-09 | 2022-12-06 | 景俊年 | Intelligent bracelet and control method thereof |
| CN108897444A (en) * | 2018-06-21 | 2018-11-27 | 中国科学技术大学 | The method and system of cursor control are realized using wearable wristband type universal serial mouse |
| CN108983622A (en) * | 2018-07-06 | 2018-12-11 | 安徽超清科技股份有限公司 | A kind of intelligent appliance control system based on human body gesture |
| CN109407531A (en) * | 2018-10-30 | 2019-03-01 | 深圳市心流科技有限公司 | Intelligent home furnishing control method, device and computer readable storage medium |
| CN111413874B (en) * | 2019-01-08 | 2021-02-26 | 北京京东尚科信息技术有限公司 | Method, device and system for controlling intelligent equipment |
| CN111401334A (en) * | 2020-04-29 | 2020-07-10 | 北京智宸天驰科技有限公司 | Non-contact mapping type action recognition equipment and method by using sensor |
| CN114488831B (en) * | 2022-01-10 | 2023-09-08 | 锋芒科技南京有限公司 | Internet of things household intelligent control system and method based on man-machine interaction |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1433161A1 (en) * | 2001-09-07 | 2004-06-30 | Harmonic Research, Inc. | Improved wireless control device |
| CN104123007A (en) * | 2014-07-29 | 2014-10-29 | 电子科技大学 | Multidimensional weighted 3D recognition method for dynamic gestures |
| CN104490390A (en) * | 2014-12-30 | 2015-04-08 | 天津大学 | Electrophysiological signal conjoint analysis-based human exercise ability determination method |
| CN105022471A (en) * | 2014-04-23 | 2015-11-04 | 王建勤 | Device and method for carrying out gesture recognition based on pressure sensor array |
-
2017
- 2017-05-12 CN CN201710334899.4A patent/CN107145236B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1433161A1 (en) * | 2001-09-07 | 2004-06-30 | Harmonic Research, Inc. | Improved wireless control device |
| CN105022471A (en) * | 2014-04-23 | 2015-11-04 | 王建勤 | Device and method for carrying out gesture recognition based on pressure sensor array |
| CN104123007A (en) * | 2014-07-29 | 2014-10-29 | 电子科技大学 | Multidimensional weighted 3D recognition method for dynamic gestures |
| CN104490390A (en) * | 2014-12-30 | 2015-04-08 | 天津大学 | Electrophysiological signal conjoint analysis-based human exercise ability determination method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107145236A (en) | 2017-09-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107145236B (en) | Gesture recognition method and system based on wrist tendon pressure related characteristics | |
| Wang et al. | A comparative study on human activity recognition using inertial sensors in a smartphone | |
| EP3411772B1 (en) | Wearable controller for wrist | |
| Lu et al. | A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices | |
| US8447704B2 (en) | Recognizing gestures from forearm EMG signals | |
| Luzhnica et al. | A sliding window approach to natural hand gesture recognition using a custom data glove | |
| WO2020061440A1 (en) | Neuromuscular control of an augmented reality system | |
| Kyritsis et al. | Food intake detection from inertial sensors using LSTM networks | |
| Khodabandelou et al. | Attention-based gated recurrent unit for gesture recognition | |
| Zhang et al. | Recognizing hand gestures with pressure-sensor-based motion sensing | |
| US11281301B2 (en) | Wearable controller for wrist | |
| US10608632B2 (en) | Infrared-based gesture sensing and detection systems, and apparatuses, software, and methods relating to same | |
| Shin et al. | Korean sign language recognition using EMG and IMU sensors based on group-dependent NN models | |
| WO2023184771A1 (en) | Brain-computer interface decoding method and device based on site equivalent enhancement | |
| Li et al. | Hand gesture recognition and real-time game control based on a wearable band with 6-axis sensors | |
| Shen et al. | Toward hand-dominated activity recognition systems with wristband-interaction behavior analysis | |
| Qaroush et al. | Smart, comfortable wearable system for recognizing Arabic Sign Language in real-time using IMUs and features-based fusion | |
| Chen et al. | Dimensionality reduction of data sequences for human activity recognition | |
| CN111012312A (en) | Portable Parkinson's disease bradykinesia monitoring intervention device and method | |
| Fukui et al. | Hand-shape classification with a wrist contour sensor: Analyses of feature types, resemblance between subjects, and data variation with pronation angle | |
| CN110624217A (en) | Rehabilitation glove based on multi-sensor fusion and implementation method thereof | |
| Fan et al. | Eating gestures detection by tracking finger motion | |
| Fukui et al. | Development of wrist contour measuring device for an interface using hand shape recognition | |
| Fukui et al. | Hand Shape Classification with a Wrist Contour Sensor: (Comparison of Feature Types and Observation of Resemblance among Subjects) | |
| CN111657860B (en) | Method and system for identifying sleep stage based on counterstudy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |