CN107704190B - Gesture recognition method and device, terminal and storage medium - Google Patents
Gesture recognition method and device, terminal and storage medium Download PDFInfo
- Publication number
- CN107704190B CN107704190B CN201711076834.0A CN201711076834A CN107704190B CN 107704190 B CN107704190 B CN 107704190B CN 201711076834 A CN201711076834 A CN 201711076834A CN 107704190 B CN107704190 B CN 107704190B
- Authority
- CN
- China
- Prior art keywords
- gesture
- probability
- logistic regression
- regression model
- touch
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 49
- 238000007477 logistic regression Methods 0.000 claims abstract description 64
- 230000001960 triggered effect Effects 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 43
- 230000006870 function Effects 0.000 claims description 25
- 238000013139 quantization Methods 0.000 claims description 16
- 238000011478 gradient descent method Methods 0.000 claims description 10
- 238000006748 scratching Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 10
- 238000004891 communication Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 230000005236 sound signal Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
Abstract
The application discloses a gesture recognition method, a gesture recognition device, a terminal and a storage medium, and belongs to the technical field of terminals. The method comprises the following steps: when the sliding operation triggered by a plurality of touch points is detected in a designated area of a screen, a touch signal is obtained according to a sliding track, a first gesture probability and a second gesture probability are determined through a designated logistic regression model based on the touch signal, the first gesture probability refers to the probability that the current gesture is a lifting gesture, the second gesture probability refers to the probability that the current gesture is a lifting gesture, and the current gesture is recognized based on the first gesture probability and the second gesture probability. That is, according to the application, the touch signals of the touch points in the sliding process are obtained, and based on the touch signals, the division and the recognition of the scratch-up gesture and the lift-off gesture are realized through the appointed logistic regression model, so that the gesture operation of the user can be correctly responded.
Description
Technical Field
The present application relates to the field of terminal technologies, and in particular, to a gesture recognition method, an apparatus, a terminal, and a storage medium.
Background
At present, with the rapid development of terminal technology, the interaction mode between the user and the terminal becomes more and more diversified. For example, a user may operate a terminal using various gestures, which are largely classified into static gestures and dynamic gestures. The static gesture includes, but is not limited to, a tap gesture and the dynamic gesture includes, but is not limited to, a swipe gesture, a flip gesture. The upward-stroke gesture refers to a gesture of sliding from the lower side of the screen to the upper side of the screen by using any part of a hand, and the opening gesture refers to a gesture of sliding along one corner of the screen to the diagonal direction of the screen by using the edge area of the back of the hand.
Disclosure of Invention
The embodiment of the application provides a gesture recognition method, a gesture recognition device, a terminal and a storage medium, which can be used for distinguishing and recognizing a gesture of upwards-sliding and upwards-opening. The technical scheme is as follows:
in a first aspect, a gesture recognition method is provided, and the method includes:
when the sliding operation triggered by a plurality of touch points is detected in the designated area of the screen, acquiring a touch signal according to the sliding track;
determining a first gesture probability and a second gesture probability by specifying a logistic regression model based on the touch signal, wherein the first gesture probability refers to the probability that the current gesture is a lift-off gesture, and the second gesture probability refers to the probability that the current gesture is a stroke-up gesture;
and recognizing the current gesture based on the first gesture probability and the second gesture probability.
Optionally, before determining the first gesture probability and the second gesture probability by specifying a logistic regression model based on the touch signal, the method further includes:
acquiring touch signals of different gestures for multiple times to obtain multiple sample data, wherein the different gestures comprise the upwards-scratching gesture and the upwards-lifting gesture;
and training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model.
Optionally, the preset training model includes a loss function model and an initialized logistic regression model;
training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model, including:
quantizing the plurality of sample data to obtain a sample quantization vector;
inputting the sample quantization vector into the loss function model, and determining an estimation weight by determining a minimum value of the loss function model by a gradient descent method;
and inputting the estimation weight into the initialized logistic regression model to obtain the specified logistic regression model.
Optionally, the recognizing a current gesture based on the first gesture probability and the second gesture probability includes:
determining a maximum gesture probability of the first and second gesture probabilities;
and identifying the current gesture as the gesture corresponding to the maximum gesture probability.
Optionally, the touch signal includes average position information or position difference information, the average position information is an average value of all position information on the sliding track, and the position difference information is a position variation between an initial position and an end position of the sliding track.
In a second aspect, a gesture recognition apparatus is provided, the apparatus comprising:
the acquisition module is used for acquiring a touch signal according to a sliding track when the sliding operation triggered by a plurality of touch points is detected in a designated area of a screen;
a determining module, configured to determine, based on the touch signal, a first gesture probability and a second gesture probability by specifying a logistic regression model, where the first gesture probability refers to a probability that a current gesture is a flick gesture, and the second gesture probability refers to a probability that the current gesture is a flick gesture;
and the recognition module is used for recognizing the current gesture based on the first gesture probability and the second gesture probability.
Optionally, the apparatus further comprises:
the acquisition module is used for acquiring touch signals of different gestures for multiple times to obtain multiple sample data, wherein the different gestures comprise the upwards-scratching gesture and the upwards-lifting gesture;
and the training module is used for training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model.
Optionally, the preset training model includes a loss function model and an initialized logistic regression model; the training module is configured to:
quantizing the plurality of sample data to obtain a sample quantization vector;
inputting the sample quantization vector into the loss function model, and determining an estimation weight by determining a minimum value of the loss function model by a gradient descent method;
and inputting the estimation weight into the initialized logistic regression model to obtain the specified logistic regression model.
In a third aspect, a terminal comprises a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor is configured to perform the steps of the gesture recognition method according to any one of the above first aspects.
In a fourth aspect, a computer-readable storage medium is provided, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the gesture recognition method of any one of the above first aspects.
The technical scheme provided by the embodiment of the application has the following beneficial effects: when a sliding operation triggered by a plurality of touch points is detected on a designated area of a screen, a sliding operation triggered by a user using an edge area of the back of a hand is illustrated, and at this time, a touch signal is acquired according to a sliding track. Based on the touch signal, the probability that the current gesture is a lift-off gesture and the probability that the current gesture is a stroke-up gesture are determined through a designated logistic regression model, namely a first gesture probability and a second gesture probability are determined, and then the current gesture is recognized based on the first gesture probability and the second gesture probability. That is, according to the application, the touch signals of the touch points in the sliding process are obtained, and based on the touch signals, the division and the recognition of the scratch-up gesture and the lift-off gesture are realized through the appointed logistic regression model, so that the gesture operation of the user can be correctly responded.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a flow diagram illustrating a method of gesture recognition in accordance with an exemplary embodiment;
FIG. 2A is a flow diagram illustrating a method of gesture recognition in accordance with another exemplary embodiment;
FIG. 2B is a schematic illustration of a designated area according to the embodiment of FIG. 2A;
FIG. 2C is a schematic view of a plurality of touch points according to the embodiment of FIG. 2A;
FIG. 2D is a diagram illustrating a flick gesture according to the embodiment of FIG. 2A;
FIG. 3A is a block diagram illustrating a configuration of a gesture recognition apparatus according to an exemplary embodiment;
FIG. 3B is a block diagram illustrating a configuration of a gesture recognition apparatus according to another exemplary embodiment;
FIG. 4 is a block diagram illustrating a gesture recognition apparatus 400 according to an example embodiment.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Before describing the gesture recognition method provided by the embodiment of the present application in detail, application scenarios and implementation environments related to the embodiment of the present application are briefly described.
First, a brief description is given of an application scenario related to an embodiment of the present application.
Currently, a terminal can support operations of various gestures. For example, in a practical application scenario, when a user wants to slide upwards to view displayed content, a swipe gesture may be used, and for example, when the user wants to view some confidential information in some public places, a flip gesture may be used, so that the terminal displays the content to be displayed on a certain corner of the screen based on the flip gesture. However, since the swipe gesture and the open gesture are relatively close, in order to correctly respond to the actual operation requirement of the user, the two gestures need to be distinguished and recognized. To this end, the present application provides a gesture recognition method, which can be used to distinguish and recognize two gestures, i.e., a swipe gesture and a flip gesture, and is implemented as the embodiments shown in fig. 1 and fig. 2A below.
Next, a brief description is given of an implementation environment related to the embodiments of the present application.
The gesture recognition method can be applied to a terminal, the terminal can be configured with a touch screen, and the terminal can support operation of various gestures, wherein the various gestures include a scratch gesture and a lift gesture. In an actual application scenario, the terminal may be specifically a device such as a mobile phone, a computer, and a computer, which is not limited in this application embodiment.
Referring to fig. 1, fig. 1 is a flowchart illustrating a gesture recognition method according to an exemplary embodiment, where the gesture recognition method may be applied to a terminal, and the method may include the following implementation steps:
step 101: when the sliding operation triggered by a plurality of touch points is detected on the designated area of the screen, a touch signal is acquired according to the sliding track.
Step 102: based on the touch signal, a first gesture probability and a second gesture probability are determined through a designated logistic regression model, wherein the first gesture probability refers to the probability that the current gesture is a lift-off gesture, and the second gesture probability refers to the probability that the current gesture is a lift-up gesture.
Step 103: and recognizing the current gesture based on the first gesture probability and the second gesture probability.
In the embodiment of the application, when the sliding operation triggered by a plurality of touch points is detected on the designated area of the screen, the sliding operation triggered by the edge area of the back of the hand of the user is explained, and at this time, the touch signal is acquired according to the sliding track. Based on the touch signal, the probability that the current gesture is a lift-off gesture and the probability that the current gesture is a stroke-up gesture are determined through a designated logistic regression model, namely a first gesture probability and a second gesture probability are determined, and then the current gesture is recognized based on the first gesture probability and the second gesture probability. That is, according to the application, the touch signals of the touch points in the sliding process are obtained, and based on the touch signals, the division and the recognition of the scratch-up gesture and the lift-off gesture are realized through the appointed logistic regression model, so that the gesture operation of the user can be correctly responded.
Optionally, before determining the first gesture probability and the second gesture probability by specifying a logistic regression model based on the touch signal, the method further includes:
acquiring touch signals of different gestures for multiple times to obtain multiple sample data, wherein the different gestures comprise the upwards-sliding gesture and the upwards-opening gesture;
and training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model.
Optionally, the preset training model includes a loss function model and an initialized logistic regression model;
training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model, including:
quantizing the plurality of sample data to obtain a sample quantization vector;
inputting the sample quantization vector into the loss function model, and determining an estimation weight by determining a minimum value of the loss function model by a gradient descent method;
and inputting the estimation weight into the initialized logistic regression model to obtain the specified logistic regression model.
Optionally, recognizing the current gesture based on the first gesture probability and the second gesture probability includes:
determining a maximum gesture probability of the first gesture probability and the second gesture probability;
and recognizing the current gesture as the gesture corresponding to the maximum gesture probability.
Optionally, the touch signal includes average position information or position difference information, the average position information refers to an average value of all position information on the sliding track, and the position difference information refers to a position variation between an initial position and an end position of the sliding track.
All the above optional technical solutions can be combined arbitrarily to form an optional embodiment of the present application, and the present application embodiment is not described in detail again.
Referring to fig. 2A, fig. 2A is a flowchart illustrating a gesture recognition method according to another exemplary embodiment, where the gesture recognition method may be applied to a terminal, and the method may include the following implementation steps:
step 201: and acquiring touch signals of different gestures for multiple times to obtain multiple sample data, wherein the different gestures comprise the upwards-sliding gesture and the upwards-opening gesture.
In order to distinguish and recognize the two gestures, model training needs to be performed in advance based on the touch signals triggered by the two gestures respectively, so as to obtain a specified logistic regression model which can subsequently recognize the corresponding gesture based on the touch signals triggered by the two gestures respectively. The process of model training is as described in step 201 to step 202.
That is, the touch signals of the two different gestures need to be acquired for multiple times to obtain multiple sample data, so that model training can be performed based on the multiple sample data subsequently.
The touch signal includes average position information or position difference information, the average position information is an average value of all position information on the sliding track, and the position difference information is a position variation between an initial position and an end position of the sliding track.
For example, in one possible implementation, the touch signal may include average position information. When the terminal acquires the touch signal based on different gestures, the position information on the sliding track can be acquired based on the sliding track of the different gestures, and further, each position information can be a touch coordinate corresponding to a touch point at different moments when the gesture contacts the screen. That is, the terminal may acquire the touch coordinates on a time basis, for example, the touch coordinates of the touch point are acquired at preset time intervals. Then, the terminal determines an average value of the acquired touch coordinates to obtain average position information, and determines the average position information as sample data.
The preset time interval can be set by a user according to actual needs in a self-defined manner, and can also be set by the default of the terminal, which is not limited in the embodiment of the application. For example, the preset time interval is 1 millisecond.
It should be noted that, the coordinate system may be established by using the center of the screen of the terminal as an origin, or may also be established by using a certain vertex angle of the screen of the terminal as an origin, which is not limited in this embodiment of the present application.
For another example, in another possible implementation manner, the touch signal may further include position difference information, that is, for each operation performed by a different gesture, the terminal may acquire, based on the touch signal triggered by the gesture, position information corresponding to an initial position and position information corresponding to an end position of the sliding track, then determine a position variation between the position information corresponding to the initial position and the position information corresponding to the end position, and determine the determined position variation as sample data.
Of course, it should be noted that, here, it is only described that the touch signal includes average position information or position difference information as an example, in an actual application scenario, the touch signal may further include information such as touch strength, and at this time, the touch strength information may be obtained through a pressure sensor, which is not limited in this application.
Step 202: and training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model.
After obtaining the plurality of sample data, the model training can be performed by using the plurality of sample data. In a specific implementation, the preset training model may include a loss function model and an initialized logistic regression model, in which case, training is performed through the preset training model based on the plurality of sample data, and obtaining a specific implementation of the specified logistic regression model may include the following steps (1) to (3):
(1) and quantizing the plurality of sample data to obtain a sample quantization vector.
In one embodiment, the multiple sample data may be quantized, for example, to obtain a sample quantization vector of X ═ XiN, where n is a preset positive integer, which may be preset by a user,for example, the n may be set to 10. Namely, after the plurality of sample data are quantized, an n-dimensional sample quantization vector is obtained.
(2) The sample quantization vector is input into the loss function model, and an estimation weight is determined by determining a minimum value of the loss function model using a gradient descent method.
Wherein the loss function model can be shown as formula (1):
wherein, yj∈{0,1}，j＝1,2，yjRepresenting two-class label information, e.g. when y isjWhen the value is 1, the classification information is a flick gesture, and when y is equal tojWhen the value is 0, the representative classification information is a gesture of drawing upward. w is an unknown evaluation parameter.
Then, L (w) is minimum by the gradient descent method, and when L (w) minimum is found, the estimated weight w can be obtained, wherein in the process of finding L (w) minimum by the gradient descent method, training is continuously performed based on the sample data to minimize L (w), that is, the process of finding L (w) minimum by the gradient descent method is a process of training continuously in practice.
(3) And inputting the estimation weight into the initialized logistic regression model to obtain the specified logistic regression model.
In a specific implementation, the initialized logistic regression model may be as shown in formulas (2) and (3), and after the terminal brings the estimation weight w into the initialized logistic regression models (2) and (3), the specified logistic regression model may be obtained as shown in formulas (4) and (5):
wherein, P (Y ═ 1| x) and P (Y ═ 0| x) represent gesture probabilities corresponding to the two gestures of opening and drawing up, respectively.
Further, the terminal may store the trained designated logistic regression model in advance, so that different gestures may be recognized subsequently through the designated logistic regression model based on touch signals triggered by the different gestures.
It should be noted that, here, only the terminal performs model training based on different gestures, in an actual application scenario, model training may also be performed by other terminals having the same function, and then the trained designated logistic regression model is moved to the terminal.
After the terminal stores the specified logistic regression model, the terminal may perform gesture recognition subsequently, which may specifically include the following steps 203 to 205.
Step 203: when the sliding operation triggered by a plurality of touch points is detected on the designated area of the screen, a touch signal is acquired according to the sliding track.
In an actual implementation, the designated area may be set by a user according to an actual requirement, for example, the designated area may be a lower left area, a lower right area, an upper left area, an upper right area, and the like of the screen, which is not limited in this embodiment of the application.
Referring to fig. 2B, the designated area is taken as the lower left area 21 of the screen for illustration. That is, when the terminal detects a sliding operation triggered by a plurality of touch points on the lower left area 21 of the screen, it indicates that the user can slide upwards through the edge area of the back of the hand, and in order to be able to recognize whether the gesture is a lifting gesture or a swiping gesture, the terminal acquires a touch signal according to the sliding track.
The plurality of touch points may be touch points formed by touching the screen by a plurality of joints where the edge area of the back of the hand of the user is located. For example, as shown at 22 in fig. 2C.
Step 204: based on the touch signal, a first gesture probability and a second gesture probability are determined through a designated logistic regression model, wherein the first gesture probability refers to the probability that the current gesture is a lift-off gesture, and the second gesture probability refers to the probability that the current gesture is a lift-up gesture.
In a specific implementation, the touch signal may be quantized to obtain a quantized vector, and then the terminal may input the obtained quantized vector into a specified logistic regression model shown in the above equations (4) and (5) to determine gesture probabilities corresponding to two gestures, which are a first gesture probability P (Y ═ 1| x) and a second gesture probability P (Y ═ 0| x), respectively.
Further, in an actual implementation, since both the opening gesture and the swiping-up gesture have the feature of sliding upwards, before determining the first gesture probability and the second gesture probability by specifying a logistic regression model based on the touch signal, it may be further determined whether the current gesture is one of the opening gesture and the swiping-up gesture, that is, whether the gesture is the swiping-up gesture.
In a specific implementation, whether the gesture is a sliding-up gesture may be determined based on the sliding track, for example, if the initial position of the sliding track is located in the lower area of the screen and the end position is located in the upper area of the screen, the gesture may be determined to be a sliding-up gesture.
Of course, it should be noted that, here, the description is only given by taking the example of determining whether the gesture is the upward sliding gesture based on the sliding track, in another embodiment, it may also be determined by adopting another manner, for example, it may also be determined based on the position change amount, and the embodiment of the present application does not limit this.
Step 205: and recognizing the current gesture based on the first gesture probability and the second gesture probability.
In a specific implementation, based on the first gesture probability and the second gesture probability, a specific implementation of recognizing the current gesture may include: and determining the maximum gesture probability in the first gesture probability and the second gesture probability, and identifying the current gesture as the gesture corresponding to the maximum gesture probability.
It will be appreciated that the greater the probability of a gesture, the greater the likelihood of stating that the current gesture is the gesture to which the greatest probability of a gesture corresponds. For example, if the first gesture probability is greater than the second gesture probability, the probability that the current gesture is determined to be a lift-off gesture after the recognition is greater than the probability that the current gesture is a swipe-up gesture. On the contrary, if the first gesture probability is smaller than the second gesture probability, it is determined that the probability that the current gesture is the lift-off gesture is smaller than the probability that the current gesture is the pull-up gesture after the recognition, and therefore, the current gesture can be recognized as the pull-up gesture.
For example, when the first gesture probability is 70% and the second gesture probability is 30%, it can be seen that the probability that the current gesture is a lift-off gesture is greater than the probability that the current gesture is a stroke-up gesture, and therefore, the current gesture can be recognized as a lift-off gesture, for example, as shown in fig. 2D.
In the embodiment of the application, when the sliding operation triggered by a plurality of touch points is detected on the designated area of the screen, the sliding operation triggered by the edge area of the back of the hand of the user is explained, and at this time, the touch signal is acquired according to the sliding track. Based on the touch signal, the probability that the current gesture is a lift-off gesture and the probability that the current gesture is a stroke-up gesture are determined through a designated logistic regression model, namely a first gesture probability and a second gesture probability are determined, and then the current gesture is recognized based on the first gesture probability and the second gesture probability. That is, according to the application, the touch signals of the touch points in the sliding process are obtained, and based on the touch signals, the division and the recognition of the scratch-up gesture and the lift-off gesture are realized through the appointed logistic regression model, so that the gesture operation of the user can be correctly responded.
Referring to fig. 3A, fig. 3A is a block diagram illustrating a gesture recognition apparatus according to an exemplary embodiment, which may be implemented by software, hardware or a combination of the two, and may include:
an obtaining module 301, configured to obtain a touch signal according to a sliding track when a sliding operation triggered by multiple touch points is detected on a designated area of a screen;
a determining module 302, configured to determine, based on the touch signal, a first gesture probability and a second gesture probability by specifying a logistic regression model, where the first gesture probability refers to a probability that a current gesture is a lift-off gesture, and the second gesture probability refers to a probability that the current gesture is a drag-up gesture;
a recognition module 303, configured to recognize a current gesture based on the first gesture probability and the second gesture probability.
Optionally, referring to fig. 3B, the apparatus further includes:
the acquisition module 304 is configured to acquire touch signals of different gestures for multiple times to obtain multiple sample data, where the different gestures include the swipe gesture and the lift gesture;
a training module 305, configured to perform training through a preset training model based on the multiple sample data to obtain the specified logistic regression model.
Optionally, the preset training model includes a loss function model and an initialized logistic regression model; the training module 305 is configured to:
quantizing the plurality of sample data to obtain a sample quantization vector;
inputting the sample quantization vector into the loss function model, and determining an estimation weight by determining a minimum value of the loss function model by a gradient descent method;
and inputting the estimation weight into the initialized logistic regression model to obtain the specified logistic regression model.
In the embodiment of the application, when the sliding operation triggered by a plurality of touch points is detected on the designated area of the screen, the sliding operation triggered by the edge area of the back of the hand of the user is explained, and at this time, the touch signal is acquired according to the sliding track. Based on the touch signal, the probability that the current gesture is a lift-off gesture and the probability that the current gesture is a stroke-up gesture are determined through a designated logistic regression model, namely a first gesture probability and a second gesture probability are determined, and then the current gesture is recognized based on the first gesture probability and the second gesture probability. That is, according to the application, the touch signals of the touch points in the sliding process are obtained, and based on the touch signals, the division and the recognition of the scratch-up gesture and the lift-off gesture are realized through the appointed logistic regression model, so that the gesture operation of the user can be correctly responded.
FIG. 4 is a block diagram illustrating a gesture recognition apparatus 400 according to an example embodiment. For example, the apparatus 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, an exercise device, a personal digital assistant, and the like.
Referring to fig. 4, the apparatus 400 may include one or more of the following components: processing components 402, memory 404, power components 406, multimedia components 408, audio components 410, input/output (I/O) interfaces 412, sensor components 414, and communication components 416.
The processing component 402 generally controls overall operation of the apparatus 400, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to perform all or a portion of the steps of the methods described above. Further, the processing component 402 can include one or more modules that facilitate interaction between the processing component 402 and other components. For example, the processing component 402 can include a multimedia module to facilitate interaction between the multimedia component 408 and the processing component 402.
The memory 404 is configured to store various types of data to support operations at the apparatus 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 404 may be implemented by any type or combination of volatile or non-volatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.
The multimedia component 408 includes a screen that provides an output interface between the device 400 and a user, in some embodiments, the screen may include a liquid crystal display (L CD) and a Touch Panel (TP). if the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
The audio component 410 is configured to output and/or input audio signals. For example, audio component 410 includes a Microphone (MIC) configured to receive external audio signals when apparatus 400 is in an operational mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may further be stored in the memory 404 or transmitted via the communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.
The I/O interface 412 provides an interface between the processing component 402 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
The sensor component 414 includes one or more sensors for providing various aspects of status assessment for the apparatus 400. For example, the sensor assembly 414 may detect an open/closed state of the apparatus 400, the relative positioning of the components, such as a display and keypad of the apparatus 400, the sensor assembly 414 may also detect a change in the position of the apparatus 400 or a component of the apparatus 400, the presence or absence of user contact with the apparatus 400, orientation or acceleration/deceleration of the apparatus 400, and a change in the temperature of the apparatus 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
The communication component 416 is configured to facilitate wired or wireless communication between the apparatus 400 and other devices. The apparatus 400 may access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 further includes a Near Field Communication (NFC) module to facilitate short-range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
In an exemplary embodiment, the apparatus 400 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), programmable logic devices (P L D), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the gesture recognition methods provided by the embodiments illustrated in fig. 1 or fig. 2A and described above.
In an exemplary embodiment, a non-transitory computer-readable storage medium comprising instructions, such as the memory 404 comprising instructions, executable by the processor 420 of the apparatus 400 to perform the above-described method is also provided. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
A non-transitory computer readable storage medium, wherein instructions of the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform the gesture recognition method provided in the embodiment of fig. 1 or fig. 2A described above.
A computer program product containing instructions which, when run on a computer, cause the computer to perform the gesture recognition method provided in the embodiments of fig. 1 or fig. 2A described above.
It should be noted that: in the gesture recognition apparatus provided in the above embodiment, when the gesture recognition method is implemented, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the gesture recognition apparatus and the gesture recognition method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are described in the method embodiments in detail and are not described herein again.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (9)
1. A method of gesture recognition, the method comprising:
acquiring a touch signal according to a sliding track when a sliding operation triggered by a plurality of touch points is detected on a designated area of a screen, wherein the designated area comprises at least one of a lower left area, a lower right area, an upper left area and an upper right area of the screen;
if the gesture indicated by the sliding track is a sliding gesture, determining a first gesture probability and a second gesture probability through a specified logistic regression model based on the touch signal, the first gesture probability refers to the probability that the current gesture is a lift-off gesture, the second gesture probability refers to the probability that the current gesture is a stroke-up gesture, the opening gesture refers to a gesture of sliding along one corner of the screen to a diagonal direction by using an edge area of the back of the hand, for triggering display of content at a corner of a screen, the specified logistic regression model being for gesture recognition based on the touch signal, the touch signal includes average position information or position difference information, the average position information is an average value of all position information on the sliding track, the position difference information is a position variation between an initial position and an end position of the sliding track;
and recognizing the current gesture based on the first gesture probability and the second gesture probability.
2. The method of claim 1, wherein prior to determining the first gesture probability and the second gesture probability by assigning a logistic regression model based on the touch signal, further comprising:
acquiring touch signals of different gestures for multiple times to obtain multiple sample data, wherein the different gestures comprise the upwards-scratching gesture and the upwards-lifting gesture;
and training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model.
3. The method of claim 2, wherein the pre-set training model comprises a loss function model and an initialized logistic regression model;
training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model, including:
quantizing the plurality of sample data to obtain a sample quantization vector;
inputting the sample quantization vector into the loss function model, and determining an estimation weight by determining a minimum value of the loss function model by a gradient descent method;
and inputting the estimation weight into the initialized logistic regression model to obtain the specified logistic regression model.
4. The method of claim 1, wherein the recognizing a current gesture based on the first gesture probability and the second gesture probability comprises:
determining a maximum gesture probability of the first and second gesture probabilities;
and identifying the current gesture as the gesture corresponding to the maximum gesture probability.
5. A gesture recognition apparatus, the apparatus comprising:
the touch control device comprises an acquisition module, a display module and a control module, wherein the acquisition module is used for acquiring a touch signal according to a sliding track when a sliding operation triggered by a plurality of touch points is detected on a designated area of a screen, and the designated area comprises at least one of a lower left area, a lower right area, an upper left area and an upper right area of the screen;
a determining module, configured to, if the gesture indicated by the sliding track is a slide-up gesture, based on the touch signal, determining a first gesture probability and a second gesture probability by specifying a logistic regression model, wherein the first gesture probability refers to the probability that the current gesture is a lifting gesture, the second gesture probability refers to the probability that the current gesture is a stroke gesture, the lifting gesture refers to a gesture which slides along one corner of the screen to the diagonal direction by utilizing the edge area of the back of the hand, for triggering display of content at a corner of a screen, the specified logistic regression model being for gesture recognition based on the touch signal, the touch signal includes average position information or position difference information, the average position information is an average value of all position information on the sliding track, the position difference information is a position variation between an initial position and an end position of the sliding track;
and the recognition module is used for recognizing the current gesture based on the first gesture probability and the second gesture probability.
6. The apparatus of claim 5, wherein the apparatus further comprises:
the acquisition module is used for acquiring touch signals of different gestures for multiple times to obtain multiple sample data, wherein the different gestures comprise the upwards-scratching gesture and the upwards-lifting gesture;
and the training module is used for training through a preset training model based on the plurality of sample data to obtain the specified logistic regression model.
7. The apparatus of claim 6, wherein the pre-set training model comprises a loss function model and an initialized logistic regression model; the training module is configured to:
quantizing the plurality of sample data to obtain a sample quantization vector;
inputting the sample quantization vector into the loss function model, and determining an estimation weight by determining a minimum value of the loss function model by a gradient descent method;
and inputting the estimation weight into the initialized logistic regression model to obtain the specified logistic regression model.
8. A terminal, characterized in that the terminal comprises: a processor and a memory, the memory storing one or more computer programs which, when executed by the processor, implement the method of any one of claims 1-4.
9. A computer-readable storage medium having stored therein instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711076834.0A CN107704190B (en) | 2017-11-06 | 2017-11-06 | Gesture recognition method and device, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711076834.0A CN107704190B (en) | 2017-11-06 | 2017-11-06 | Gesture recognition method and device, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107704190A CN107704190A (en) | 2018-02-16 |
| CN107704190B true CN107704190B (en) | 2020-07-10 |
Family
ID=61177907
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711076834.0A Expired - Fee Related CN107704190B (en) | 2017-11-06 | 2017-11-06 | Gesture recognition method and device, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107704190B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109766043A (en) * | 2018-12-29 | 2019-05-17 | 华为技术有限公司 | The operating method and electronic equipment of electronic equipment |
| CN110532755B (en) * | 2019-08-09 | 2020-11-03 | 北京三快在线科技有限公司 | Computer-implemented risk identification method and device |
| CN110688039A (en) * | 2019-09-25 | 2020-01-14 | 大众问问(北京)信息科技有限公司 | Control method, device and equipment for vehicle-mounted application and storage medium |
| CN110703919A (en) * | 2019-10-11 | 2020-01-17 | 大众问问(北京)信息科技有限公司 | Method, device, equipment and storage medium for starting vehicle-mounted application |
| CN114578959B (en) * | 2021-12-30 | 2024-03-29 | 惠州华阳通用智慧车载系统开发有限公司 | Gesture recognition method and system based on touch pad |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577793A (en) * | 2012-07-27 | 2014-02-12 | 中兴通讯股份有限公司 | Gesture recognition method and device |
| CN105988583A (en) * | 2015-11-18 | 2016-10-05 | 乐视致新电子科技（天津）有限公司 | Gesture control method and virtual reality display output device |
| CN106203380A (en) * | 2016-07-20 | 2016-12-07 | 中国科学院计算技术研究所 | Ultrasound wave gesture identification method and system |
| CN103870199B (en) * | 2014-03-31 | 2017-09-29 | 华为技术有限公司 | The recognition methods of user operation mode and handheld device in handheld device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9619035B2 (en) * | 2011-03-04 | 2017-04-11 | Microsoft Technology Licensing, Llc | Gesture detection and recognition |
-
2017
- 2017-11-06 CN CN201711076834.0A patent/CN107704190B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577793A (en) * | 2012-07-27 | 2014-02-12 | 中兴通讯股份有限公司 | Gesture recognition method and device |
| CN103870199B (en) * | 2014-03-31 | 2017-09-29 | 华为技术有限公司 | The recognition methods of user operation mode and handheld device in handheld device |
| CN105988583A (en) * | 2015-11-18 | 2016-10-05 | 乐视致新电子科技（天津）有限公司 | Gesture control method and virtual reality display output device |
| CN106203380A (en) * | 2016-07-20 | 2016-12-07 | 中国科学院计算技术研究所 | Ultrasound wave gesture identification method and system |
Non-Patent Citations (1)
| Title |
|---|
| 特征提取和卷积神经网络在手势识别中的应用研究;王龙;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170215(第02期);正文第4.2节、图4.6 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107704190A (en) | 2018-02-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107704190B (en) | Gesture recognition method and device, terminal and storage medium | |
| US10942580B2 (en) | Input circuitry, terminal, and touch response method and device | |
| US10739854B2 (en) | Terminal and touch response method and device | |
| US20150332439A1 (en) | Methods and devices for hiding privacy information | |
| CN104317520B (en) | Viewing area content processing method and device | |
| US20170123587A1 (en) | Method and device for preventing accidental touch of terminal with touch screen | |
| EP3331226A1 (en) | Method and device for reading messages | |
| CN104486451B (en) | Application program recommends method and device | |
| CN107102772B (en) | Touch control method and device | |
| EP3046042A1 (en) | Apparatus for implementing home button and fingerprint identification on single sensor | |
| CN105159661A (en) | Corner mark display method and apparatus for icon | |
| CN104899610A (en) | Picture classification method and device | |
| EP3208742A1 (en) | Method and apparatus for detecting pressure | |
| CN104484111A (en) | Content display method and device for touch screen | |
| CN113065591B (en) | Target detection method and device, electronic equipment and storage medium | |
| CN107992257A (en) | Split screen method and device | |
| CN107330391A (en) | Product information reminding method and device | |
| CN104484867A (en) | Picture processing method and device | |
| CN108803892B (en) | Method and device for calling third party application program in input method | |
| CN106980409B (en) | Input control method and device | |
| CN105550213A (en) | Picture browsing method and device | |
| CN104573642A (en) | Face recognition method and device | |
| CN104333503A (en) | Dialogue display method in instant communication scene and device thereof | |
| CN111783752A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN106936986A (en) | Application processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Applicant after: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd. Address before: Changan town in Guangdong province Dongguan 523860 usha Beach Road No. 18 Applicant before: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS Corp.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200710 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |