TWI464622B - Method and device for gesture determination - Google Patents

Description

Method and device for gesture judgment

The present invention relates to a method and apparatus for gesture determination, and more particularly to a method and apparatus for gesture determination corresponding to different applications with the same gesture.

A touch sensor or a touch pad or a digitizer can provide detection information of the object above the object. According to the detection information controller, the movement of the object can be recorded, so that the movement track can be judged. Gesture (gesture or heuristic) that provides another command input method outside of the keyboard and mouse.

In the prior art, gestures are mostly used to simulate the operation of a mouse, or to correspond to a specific command in a specific application, such as a double-finger or a gesture in a video display program to provide a reduced or enlarged command. . The same gesture may have different requirements in different applications. When the application and the operating system have the same gestures to execute different commands, there will be conflicts, so the application and the operating system need to avoid using the same gesture. .

It can be seen that the above prior art obviously has inconveniences and defects, and needs to be further improved. In order to solve the above problems, the relevant manufacturers do not bother to find a solution, but the design that has not been applied for a long time has been developed, and the general products and methods have no suitable structure and methods to solve the above problems. Obviously it is an issue that the relevant industry is anxious to solve. Therefore, how to create a new technology is one of the current important research and development topics, and it has become the goal that the industry needs to improve.

An object of the present invention is to provide a method and apparatus for determining a potential, including a sensor and a controller for providing a touch position, and a processor for determining a gesture according to a continuous touch position. The same gesture can be used in a plurality of different applications, but corresponding to different commands of different applications, the processor can trigger the corresponding command according to the current foreground application and the determined gesture.

The object of the present invention and solving the technical problems thereof are achieved by the following technical solutions. A method for gesture determination according to the foregoing invention includes: providing a lookup table, wherein the lookup table records a pattern of at least one gesture, and a trigger corresponding to each gesture, wherein Each triggers a command of a corresponding system or an application; a detection information is obtained by a sensor; and the detection is received according to the detection The information determines the position of one or more objects near or touching the sensor; determining one or plural according to the position of the one or more objects approaching or touching the sensor a moving motion of the object; matching a moving path of the one or more objects with the aspect of the at least one gesture to determine a matched gesture; matching the matching gesture with at least one The corresponding trigger of the application determines the matched trigger; and triggers the corresponding trigger when the matching trigger is judged.

The object of the present invention and solving the technical problems thereof can be further achieved by the following technical measures.

The method for judging the gesture further includes: when the matched gesture does not match the trigger corresponding to the at least one application, matching the matching gesture with a trigger corresponding to the system to determine the trigger of the matching.

The method for judging gestures further includes determining a foreground or focused-on application, and the at least one application includes the foreground application.

The method for judging the gesture further includes selecting a plurality of applications according to the moving path of the one or more objects, and the at least one application includes the selected applications.

The applications are sorted applications, and the sorted applications sequentially match the triggers corresponding to the matched gestures, with the first matched trigger being used as the trigger for the matching.

The method for judging gestures further includes sorting the applications and generating a trigger lookup table according to the sorted applications, wherein the gestures of the corresponding plurality of applications correspond to the prioritized applications in the trigger lookup table. Triggering, and the triggering of the matching is determined by matching the matching lookup table with the matching gesture.

The method for judging gestures further includes displaying an image representing respective gestures in the trigger lookup table, wherein the trigger lookup table is dynamically generated along with a movement path of the one or more objects before the matched gesture is judged. .

When the selected applications include a foreground application and at least one background application, the foreground application is prioritized prior to the background application.

The selected applications are judged based on the starting position, the ending position, or the covered range of the moving path of the one or more objects.

The position of the look-up table and the continuous one or more objects approaching or touching the sensor is stored in a storage unit.

The object of the present invention and solving the technical problems thereof are achieved by the following technical solutions. According to the foregoing invention, a device for determining a gesture includes: a lookup table, recording an aspect of at least one gesture, and a trigger corresponding to each gesture, wherein any one of the commands corresponding to a system or an application is triggered; a sensor for providing a detection information; a controller determining, according to the received detection information, a position at which one or more objects approach or touch the sensor; a processor comprising: according to the one or more Judging a moving path of the one or more objects by the position of the object approaching or touching the sensor; matching a moving path of the one or more objects with the aspect of the at least one gesture to determine a matching gesture; matching the matching The gesture corresponding to at least one application triggers to determine a matching trigger; and triggers the corresponding trigger when the matching trigger is judged.

The object of the present invention and solving the technical problems thereof can be further achieved by the following technical measures.

The processor further includes: when the matched gesture does not match the trigger corresponding to the at least one application, matching the matching gesture with a trigger corresponding to the system to determine the trigger of the matching.

The processor further includes determining a foreground application currently in execution, and the at least one application includes the foreground application.

The processor further includes selecting a plurality of applications according to the moving path of the one or more objects, and the at least one application includes the selected applications.

The foregoing applications are the sorted applications, and the monitor sequentially matches the triggers of the sorted applications corresponding to the matched gestures, and the first matching trigger is used as the trigger of the matching.

The processor further includes sorting the applications and generating a trigger lookup table according to the sorted applications, wherein the gestures of the corresponding plurality of applications correspond to the triggering of the prioritized application in the trigger lookup table. And the matching trigger is judged by the matching gesture matching the trigger lookup table.

The processor further includes displaying an image representing respective gestures in the trigger lookup table, wherein the trigger lookup table is dynamically generated along with a movement path of the one or more objects before the matched gesture is determined.

When the selected applications include a foreground application and at least one background application, the foreground application is prioritized prior to the background application.

The selected applications are judged based on the starting position, ending position or coverage of the moving path of the one or more objects.

The device for determining the gesture further includes a storage unit for storing the look-up table and continuously recording a position where one or more objects approach or touch the sensor.

The invention will be described in detail below with some embodiments. However, the invention may be applied to other embodiments in addition to the disclosed embodiments. The scope of the present invention is not limited by the embodiments, which are subject to the scope of the claims. To provide a clearer description and to enable those skilled in the art to understand the invention, the various parts of the drawings are not drawn according to their relative dimensions, and the ratio of certain dimensions to other related dimensions will be highlighted. The exaggerated and irrelevant details are not completely drawn to illustrate the simplicity of the illustration.

The invention can be applied to an electronic device having a display, including but not limited to a computer, a mobile phone, a portable electronic device (such as a PDA), etc., and one or more application programs can be calculated and displayed on the electronic device, and the operating system of the electronic device The operation with the application can be operated by a processor. In addition, the electronic device further includes a touch device, including a sensor, which can provide detection information of the touch position as an input device of the user, and the touch device can include, but is not limited to, embedded or covered by the display. It can also be independent of the display.

Please refer to the first figure, which is a schematic flowchart of a method for gesture determination provided by the present invention. First, as shown in step 110, a lookup table is provided, which records a pattern of at least one gesture, and a trigger corresponding to each gesture, where each triggers a corresponding A command of a system or a command of an application.

Please refer to the third figure, which is a schematic diagram of one of the examples of the present invention. The lookup table 21 can have a plurality of gestures 211 (eg, gesture 1, gesture 2, gesture 3), and a plurality of triggers 212 (eg, trigger 1, trigger 2, trigger 3, trigger 4, trigger 5). Each gesture may correspond to one or more triggers (such as gesture 1, gesture 2), or there may be no corresponding trigger (such as gesture 3). Each command that triggers the corresponding system or application.

In addition, the system's commands can be to simulate the output or input commands of other input and output devices, as well as to initiate commands for specific programs. Therefore, the same gesture can be different commands for the respective system or different applications. It can be inferred by those skilled in the relevant art that the look-up table can be implemented by using a circuit or a software, and related applications are well known and will not be described herein.

As shown in step 120, a detection information is obtained by a sensor. Those skilled in the art can infer that the sensor can include, but is not limited to, a capacitive, resistive, optical, surface acoustic wave touch sensor, and the detection information includes, but is not limited to, analog signals, digital signals, or numerical representations.

In addition, as shown in step 130, based on the received detection information, it is determined that one or more objects approach or touch the position of the sensor. Including but not limited to capacitive or optical (such as IR based or camera based) sensors, the sensor can provide object detection without touching the sensor. The information is measured, so the detection information and the touch detection information that are not touched can be used as different applications.

As further shown in step 140, the motion of one or more objects is determined based on the position at which the one or more objects approach or touch the sensor. The sensor of the present invention may provide detection information of one or more objects, so that the detection information of one or more objects may be continuously recorded to form a moving path of one or more objects.

In addition, as shown in step 150, the moving path of the one or more objects and the aspect of the at least one gesture are matched to determine a matched gesture. In the aspect of the aforementioned gesture, the aspect of the gesture includes, but is not limited to, a single movement path or a multiple movement path. In an example of the present invention, the aspect of the gesture may be formed by line segments of different angles, and the moving path of the object may be determined by matching the moving path of the object and the order of occurrence of each line segment in each gesture state. Matching gestures.

In addition, as shown in steps 160 and 170, the matching gesture is matched with the trigger corresponding to the at least one application to determine a matched trigger, and the matching gesture is not triggered by the at least one application. When matching, the matching gesture is matched with the trigger corresponding to the system to determine the trigger of the matching. Moreover, as shown in step 180, when the trigger of the match is judged, the corresponding command for triggering the match is triggered.

In an example of the present invention, the foregoing at least one application program is a foreground program currently in execution, in other words, the present invention can determine the current execution foreground program, and find the matching gesture in the lookup table at the current execution. The foreground program in the corresponding trigger. Accordingly, depending on the foreground program, the matching gestures have different triggers, that is, the commands triggered by the matching gestures are different. When the matching gesture does not match the trigger corresponding to the foreground program, the matching gesture is matched with the trigger corresponding to the system to determine the trigger of the foregoing matching.

In another example of the present invention, a plurality of applications are selected according to a moving path of one or more objects, and the at least one application includes the selected applications, that is, the matching trigger is by checking The table matches the matching gesture and the trigger corresponding to the selected applications to determine. When the matching gesture does not match the trigger corresponding to the selected programs, the trigger of the foregoing matching is judged by triggering the corresponding trigger in the table matching system.

The selected program may be determined by selecting a plurality of applications according to the moving path of the one or more objects, such as determining the starting position, ending position or coverage of the moving path of one or more objects. For example, when two objects approach or touch the sensor, the selected program includes an application whose display range includes the position of two objects or at least one object, or the selected program includes a portion whose display range includes two objects. Or all applications that move the path.

Accordingly, the selected application may include a foreground program and at least one application at the beginning, during or at the end of the recording movement path. The invention further comprises sorting the selected application into a sorted application, and the triggering of the matching is to sequentially match the trigger corresponding to the sorted application, and the first matching trigger is used as the trigger of the matching. For example, according to the stacking relationship of the application on the screen, the matching gesture is matched with the trigger corresponding to the foreground application, and when the matching gesture does not match the trigger corresponding to the foreground application, the matching gesture is matched. The corresponding trigger for the next background application, and so on. When the matching gesture does not match the trigger corresponding to the sorted application, the matching gesture is matched with the trigger of the system.

In an example of the present invention, the prompting triggerable gesture is further included, for example, displaying an image representing a triggerable gesture, and may further include an instruction description corresponding to the image. In a preferred embodiment of the present invention, a trigger lookup table is generated according to the sorted application, wherein the gestures of the corresponding plurality of applications correspond to the triggering of the prioritized application in the trigger lookup table, and The trigger of the match is judged by the matching gesture matching the trigger lookup table. For example, the foreground application, the background application, and the system all correspond to the same gesture. For the common corresponding gesture, only the foreground application triggers the corresponding corresponding gesture in the trigger lookup table. Therefore, the trigger lookup table may include a plurality of applications corresponding to the trigger of the system, and corresponding to the corresponding gestures.

In addition, triggering the look-up table may be dynamically generated with the movement path of the one or more objects before the matched gesture is judged. Therefore, as the movement path changes, the display of the image representing the gesture that can be triggered may also change, that is, the generation of the trigger lookup table is prior to the judgment of the matching gesture.

In an example of the invention, the generation of the trigger lookup table is initiated dynamically when a corresponding triggered command of the system is triggered. For example, when an object or a plurality of objects are close to the sensor, the dynamic generation of the trigger table is triggered, that is, when an object or a plurality of objects are close to the sensor, an image representing a triggerable gesture is displayed, and the user is displayed. A gesture can be made based on the prompt of the triggerable gesture, which can be made when the sensor is touched or when the sensor is not touched.

Those skilled in the art can infer that the aforementioned application and image can be displayed on a display, and the present invention will not be described.

Please refer to the second figure, which is a functional block diagram of a gesture judging device according to a preferred mode of the present invention, including a lookup table 21, a sensor 22, a controller 24, a processor 26, and a storage device. Unit 28.

The lookup table 21 may be as described in the foregoing step 110, recording at least one gesture, and corresponding triggering of each gesture, any one of which triggers a command of a corresponding system or an application, and the lookup table 21 may be stored. In the storage unit 28. It can be inferred by those skilled in the relevant art that the look-up table can be implemented by using a circuit or a software, and related applications are well known and will not be described herein. In an example of the present invention, each gesture may correspond to one or more triggers, or there may be no corresponding triggers. In addition, the system's commands can be to simulate the output or input commands of other input and output devices, as well as to initiate commands for specific programs. Therefore, the same gesture can be different commands for the respective system or different applications.

Additionally, the sensor 22 can provide detection information as described in step 120. Those skilled in the art can infer that the sensor can include, but is not limited to, a capacitive, resistive, optical, surface acoustic wave touch sensor, and the detection information includes, but is not limited to, analog signals, digital signals, or numerical representations.

The detection information provided by the sensor 22 is received by the controller 24, and the controller 24 may determine, according to the received detection information, a position 232 at which the one or more objects approach or touch the sensor according to the received detection information. Including but not limited to capacitive or optical or sensory sensors, the sensor can provide detection information of the object before it touches the sensor, so there is no touch detection information and touch. The detection information of the touch can be used as a different application.

The position 232 of the one or more objects approaching or touching the sensor may be received by the processor 26, and the processor 26 stores the one or more objects adjacent to or touching the continuous position 232 of the sensor in the storage unit. 28, and may be as shown in step 140, determining the path of movement of one or more objects in accordance with the continuous position 232 of the one or more objects approaching or touching the sensor. The processor 26 may be configured to execute a movement path determination program 23 for performing a movement path, wherein the movement path determination program 23 may be stored in the storage unit 28.

The processor 26 may also be configured to match the movement path of one or more objects with the at least one gesture as described in step 150 to determine a matching gesture. In the aspect of the aforementioned gesture, the aspect of the gesture includes, but is not limited to, a single movement path or a multiple movement path. In an example of the present invention, the aspect of the gesture may be formed by line segments of different angles, and the moving path of the object may be determined by matching the moving path of the object and the order of occurrence of each line segment in each gesture state. Matching gestures. The processor 26 may perform a gesture matching program 25 to perform gesture matching, wherein the gesture matching program 25 may be stored in the storage unit 28.

In addition, the processor 26 may further be configured to match the matching gesture with the at least one application trigger as described in steps 160 and 170 to determine a matching trigger, and trigger the matching gesture with at least one application. When there is no match, the matching gesture is matched with the trigger of the system to determine the trigger of the match. Moreover, the processor 26 may also be a triggering corresponding command that triggers the matching when the matching trigger is determined as shown in step 180. The processor 26 may be a match that performs a trigger matching program 27 for triggering, wherein the trigger matching program 27 may be stored in the storage unit 28.

Those skilled in the art can infer that in the circuit design, the controller 24 and the processor 26 can be integrated in the same circuit, and the movement path judging program 23, the gesture matching program 25, and the trigger matching program 27 can be integrated into the same program. The design of the present invention for hardware and software includes, but is not limited to, the aforementioned designs. In addition, the aforementioned processor may be a processor including, but not limited to, a computer, a mobile phone, and a portable digital device such as a personal digital assistant (PDA).

As previously described, the processor 26 further includes displaying on the display an image representative of each trigger that triggers the look-up table. For example, in the fourth figure, a computer with a display is covered with a transparent touch sensor. When the user's left hand 46 approaches or touches a knob image 43 of the foreground application 41, a corresponding trigger lookup table is generated, and each triggered image 44 in the trigger lookup table is displayed on the display. , prompting the user to rotate clockwise or counterclockwise. When the user's right hand 45 rotates clockwise, the processor 26 determines the movement path of the left hand 46 and the right hand 45, and accordingly determines the clockwise rotation gesture, when the clockwise rotation gesture and the trigger lookup table When a match is triggered, a corresponding command is triggered, such as rotating the rotated image 43 at an angle that is rotated by the movement path of the right hand 45.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the scope of the claims of the present invention; any equivalent changes or modifications which are made in the spirit of the present invention should be included in the following. The scope of the patent application.

twenty one. . . Look up table

211. . . gesture

212. . . trigger

twenty two. . . sensor

twenty three. . . Moving path judgment program

twenty four. . . Controller

25. . . Gesture matching program

26. . . processor

27. . . Trigger matching program

28. . . Storage unit

41. . . Foreground application

42. . . Background application

43. . . Knob image

44. . . Gesture image

The first figure is a schematic flowchart of a method for judging gestures provided by the present invention; the second figure is a functional block diagram of a device for judging gestures provided by the present invention; The third figure is a schematic diagram of an example of a lookup table of the present invention; and the fourth figure is a schematic diagram of an operation example of the present invention.

21‧‧‧Checklist

22‧‧‧ sensor

23‧‧‧Mobile path judgment program

24‧‧‧ Controller

25‧‧‧ gesture matching program

26‧‧‧ Processor

27‧‧‧Trigger Matching Program

28‧‧‧storage unit

Claims (20)

A method for determining a gesture includes: providing a lookup table that records a plurality of gestures and a plurality of triggers corresponding to at least one gesture, wherein any one of the commands corresponding to a system or an application is triggered; A sensor obtains a detection information; determining, according to the received detection information, a position at which one or more objects approach or touch the sensor; determining, according to the position of the one or more objects approaching or touching the sensor a moving path of one or more objects; matching a moving path of the one or more objects with the aspect of the at least one gesture to determine a matching gesture; matching the matching gesture with a trigger corresponding to the at least one application, To determine the trigger of the match; and trigger the corresponding command to trigger the match when the trigger of the match is judged. The method for determining a gesture according to the first aspect of the patent application, further comprising: when the matching gesture does not match the trigger corresponding to the at least one application, matching the matching gesture with a trigger corresponding to the system to determine The trigger for this match. The method for determining gestures as described in claim 1 further includes determining a foreground application currently being executed, and the at least one application includes the foreground application. The method for determining a gesture according to claim 1, further comprising selecting a plurality of applications according to a movement path of the one or more objects, and the at least one application includes the selected applications. The method for determining a gesture according to claim 4, wherein the applications are sorted applications, and the sorted applications sequentially match the triggers corresponding to the matched gestures to match first. The trigger is triggered as the match. The method for determining gestures as described in claim 4, further comprising: sorting the applications and generating a trigger lookup table according to the sorted applications, wherein the gestures of the corresponding plurality of applications are in the trigger lookup table. The trigger corresponding to the prioritized application, and the trigger of the match is judged by the matching gesture matching the trigger lookup table. The method for judging the gesture described in claim 6 further includes displaying an image representing a corresponding gesture in the trigger lookup table, wherein the trigger lookup table is accompanied by the matching gesture before being judged Or the movement path of a plurality of objects is dynamically generated. The method for determining a gesture according to claim 4, wherein the selected application comprises a foreground application and at least one background application, the foreground application being prior to the background application. The method for determining a gesture according to claim 4, wherein the selected applications are based on a starting position of a moving path of the one or more objects. The position, end position or coverage is judged. The method for determining a gesture according to claim 1, wherein the position of the look-up table and the continuous one or more objects approaching or touching the sensor is stored in a storage unit. A device for determining a gesture includes: a lookup table, recording a plurality of gestures, and a plurality of triggers corresponding to at least one gesture, wherein any one of the commands corresponding to a system or an application is triggered; and a sensor provides a detection information; a controller, determining, according to the received detection information, a position at which one or more objects approach or touch the sensor; a processor comprising: approaching or touching according to the one or more objects The position of the sensor determines a moving path of one or more objects; matches a moving path of the one or more objects with the aspect of the at least one gesture to determine a matching gesture; and matches the matching gesture with at least one The corresponding trigger of the application determines the matching trigger; and triggers the corresponding trigger when the matching trigger is judged. The device for determining a gesture according to claim 11, wherein the processor further includes: the matching gesture corresponding to the at least one application triggering When matching, the matching gesture is matched with the trigger corresponding to the system to determine the trigger of the matching. The apparatus for gesture determination according to claim 11, wherein the processor further comprises determining a foreground application currently being executed, and the at least one application comprises the foreground application. The apparatus for determining a gesture according to claim 11, wherein the processor further comprises: selecting a plurality of applications according to a movement path of the one or more objects, and the at least one application includes the selected applications. Program. The apparatus for determining a gesture according to claim 14, wherein the applications are the sorted applications, and a monitor sequentially matches the triggered applications to trigger the matching gestures. The first matching trigger is used as the trigger for the match. The apparatus for claiming a gesture according to claim 14, wherein the processor further comprises: sorting the applications and generating a trigger lookup table according to the sorted applications, wherein the gestures of the corresponding plurality of applications are A trigger in the lookup table corresponding to the prioritized application is triggered, and the trigger of the match is judged by the matching gesture matching the trigger lookup table. The apparatus for gesture determination according to claim 16, wherein the processor further comprises displaying an image representing each of the corresponding gestures in the trigger lookup table, wherein the trigger lookup table is before the matched gesture is judged. Dynamically generated along with the movement path of the one or more objects. The apparatus for determining a gesture according to claim 14, wherein the selected application comprises a foreground application and at least one background application, the foreground application being prior to the background application. The apparatus for determining a gesture according to claim 14, wherein the selected applications are determined according to a starting position, an ending position, or a coverage of the moving path of the one or more objects. The device for determining the gesture according to claim 11 further includes a storage unit for storing the look-up table and continuously recording a position at which one or more objects approach or touch the sensor.