WO2017088694A1

WO2017088694A1 - Gesture calibration method and apparatus, gesture input processing method and computer storage medium

Info

Publication number: WO2017088694A1
Application number: PCT/CN2016/106167
Authority: WO
Inventors: 李鑫; 朱冰
Original assignee: 努比亚技术有限公司
Priority date: 2015-11-27
Filing date: 2016-11-16
Publication date: 2017-06-01
Also published as: CN105573545A

Abstract

A gesture calibration method and apparatus, a gesture input processing method, a terminal and a computer storage medium. The method comprises: starting an edge gesture calibration mode (S10); repeatedly inputting a certain edge gesture operation according to a pre-set number, collecting characteristic values of the edge gesture operation input each time and recording same (S20); and calculating an average value thereof according to the characteristic values of the edge gesture operation recorded each time to obtain calibration data (S30).

Description

Gesture calibration method, device and gesture input processing method, computer storage medium

Technical field

The present invention relates to the field of mobile terminal technologies, and in particular, to a gesture calibration method and apparatus, a gesture input processing method, a terminal, and a computer storage medium.

Background technique

With the rapid development of communication technology, mobile phones and other mobile terminals have more and more functions. Most of the entertainment functions that can be realized on computers can be realized on mobile terminals. People can watch movies and play games on mobile terminals. Browse web pages, video chats, and more. In order to improve the visual effect of mobile terminals, mobile terminals tend to be more and more large-screen development. However, in view of the portability characteristics of mobile terminals, the size of mobile terminals cannot be increased indefinitely, which requires full use of the external dimensions of mobile terminals to increase The utilization of the screen, so there is a narrow border or even a borderless mobile terminal.

The narrow-frame or borderless mobile terminal makes full use of the external dimensions of the mobile terminal, greatly expands the screen size of the mobile terminal, satisfies the user's demand for a large screen, and at the same time makes the edge input operation diversified. However, due to the individual differences of the user (such as the size of the palm, the strength of the finger pressing, etc.), when the narrow border or the borderless mobile terminal is held, the range of contact with the edge of the screen is greatly different, resulting in the edge gesture being non-standard and the recognition rate being low.

Summary of the invention

The technical problem to be solved by the embodiment of the present invention is to calibrate the edge interaction gesture involved in the mobile terminal, and provide a gesture calibration method, a device, a gesture input processing method, a terminal, and a computer storage medium, and the method includes:

Initiating an edge gesture calibration mode;

Repeatingly inputting an edge gesture operation according to a preset number of times, collecting feature values of the edge gesture operation input each time and recording;

The average value is calculated from the feature values of the edge gesture operation recorded each time, and calibration data is obtained.

In an embodiment of the present invention, the acquiring an edge gesture of the input, collecting the feature value corresponding to the edge gesture according to the preset number of times, and recording, includes:

The driver layer obtains the gesture input event and reports it to the application framework layer; the application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer collects the edge gesture of the user according to the judgment result, and collects the feature value of the edge gesture according to a preset number of times.

In an embodiment of the present invention, the edge gesture operation is a holding operation, and the feature value of the edge gesture operation includes a coordinate value corresponding to the finger, and the feature value is calculated according to the feature value of the edge gesture operation recorded each time. Average, resulting in calibration data, including:

Obtaining coordinate values corresponding to each finger in the edge gesture operation respectively;

An average value is calculated for the coordinate values corresponding to each finger to obtain calibration data of the edge gesture.

In an embodiment of the present invention, the edge gesture operation is a sliding operation, and the feature value of the edge gesture operation includes a start point coordinate value and a stop coordinate value of the slide operation, and the edge gesture operation according to each record The eigenvalues are averaged to obtain calibration data, including:

Obtaining a starting point coordinate value and a ending coordinate value of the finger sliding operation in the edge gesture operation respectively;

The average value is calculated for the starting point coordinate value and the ending coordinate value, respectively, and the calibration data of the edge gesture is obtained.

In an embodiment of the present invention, after calculating the average value according to the feature value corresponding to the edge gesture of each record, and after obtaining the calibration data, the method further includes:

And determining, according to the calibration data of the edge gesture, a hotspot area corresponding to the edge gesture operation, and storing the calibration data in a database to establish a correspondence relationship with the user.

The embodiment of the invention further provides a gesture calibration device, which is applied to the edge interaction of the mobile terminal. The method includes: a startup module, an acquisition module, and a processing module, where

The startup module is configured to enable the mobile terminal to turn on an edge gesture calibration mode;

The acquiring module is configured to repeatedly input an edge gesture operation according to a preset number of times, and collect feature values of the edge gesture operation input each time and record;

And a processing module configured to calculate an average value according to the feature value of the edge gesture operation recorded each time to obtain calibration data.

In an embodiment of the present invention, the acquiring module further includes:

The driving layer is configured to obtain a gesture input event and report to the application framework layer; the application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer is configured to collect an edge gesture of the user according to the determination result, and collect the feature value of the edge gesture according to a preset number of times.

In an embodiment of the present invention, the processing module further includes:

a first processing unit, configured to respectively acquire feature values corresponding to each finger in the edge gesture;

The second processing unit is configured to calculate an average value of the feature values corresponding to each finger to obtain calibration data of the edge gesture.

In an embodiment of the present invention, the apparatus further includes:

And a storage module, configured to determine a hotspot area corresponding to the edge gesture operation according to the calibration data of the edge gesture, and store the calibration data in a database to establish a correspondence relationship with the user.

The startup module, the acquisition module, the processing module, the first processing unit, the second processing unit, and the storage module may use a central processing unit (CPU) when performing processing. , digital signal processor (DSP, Digital Singnal Processor) or programmable logic array (FPGA, Field-Programmable Gate Array) implementation.

The embodiment of the present invention further provides a gesture input processing method, which is applied to an edge interaction of a mobile terminal, and includes: an input device, a driver layer, an application framework layer, and an application layer, where

The driver layer acquires a gesture input event generated by the user through the input device, and reports the event to the application framework layer;

The application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer collects the edge gesture of the user according to the judgment result, and collects the input information of the edge gesture according to a preset number of times.

The embodiment of the invention further provides a terminal, where the terminal includes:

a storage medium configured to store computer executable instructions;

a processor configured to execute computer executable instructions stored on the storage medium, the computer executable instructions comprising:

Initiating an edge gesture calibration mode;

In an embodiment of the present invention, the processor is configured to execute computer executable instructions stored on the storage medium, the computer executable instructions further comprising:

The embodiment of the present invention further provides a computer storage medium, where the computer storage medium stores computer executable instructions, and the computer executable instructions include:

Initiating an edge gesture calibration mode;

In an embodiment of the present invention, the computer executable instructions further include:

A gesture calibration method provided by an embodiment of the present invention has the following beneficial effects: the problem that the edge input gesture is not high due to individual differences or personal operation habits, and the gesture can be accurately recognized in the user gesture operation hot zone. It is convenient for users to input operations at the edge to enhance the user experience.

DRAWINGS

The present invention will be further described below in conjunction with the accompanying drawings and embodiments, in which:

1 is a schematic diagram of screen area division of a mobile terminal according to an embodiment of the present invention;

2 is a flowchart of a gesture calibration method according to an embodiment of the present invention;

3 is a flow chart of a method for calibrating a one-handed gesture in an embodiment of the present invention;

4 is a schematic diagram of a single-hand grip gesture information collection interface according to an embodiment of the present invention;

FIG. 5 is a flowchart of a method for calibrating a right edge sliding gesture according to an embodiment of the present invention; FIG.

6 is a schematic diagram of a right edge sliding gesture information collection interface according to an embodiment of the present invention;

7 is a structural block diagram of a gesture calibration apparatus according to an embodiment of the present invention;

8 is a schematic diagram of a software architecture of a mobile terminal according to an embodiment of the present invention;

9 is a flowchart of an input processing method according to an embodiment of the present invention;

FIG. 10 is a schematic structural diagram of hardware of a user equipment according to an embodiment of the present invention.

detailed description

It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

A mobile terminal embodying various embodiments of the present invention will now be described with reference to the accompanying drawings. In the following description, the use of suffixes such as "module", "component" or "unit" for indicating an element is merely an explanation for facilitating the present invention, and does not have a specific meaning per se. Therefore, "module" and "component" can be used in combination.

The mobile terminal can be implemented in various forms. For example, the terminals described in the present invention may include, for example, mobile phones, smart phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), navigation devices, and the like. Mobile terminals and fixed terminals such as digital TVs, desktop computers, and the like. In the following, it is assumed that the terminal is a mobile terminal. However, those skilled in the art will appreciate that configurations in accordance with embodiments of the present invention can be applied to fixed type terminals in addition to components that are specifically for mobile purposes.

The details are described below by way of specific examples.

FIG. 1 is a schematic diagram of screen area division of a mobile terminal according to an embodiment of the present invention. 1 is a schematic diagram of a narrow-frame mobile terminal. The mobile terminal 100 includes: a C area (gray area) is an edge input area, an area A is a normal touch area, and a B area is a non-display area. It should be noted that FIG. 1 is only an embodiment of a specific application scenario. In an actual application, the embodiment of another specific application scenario is also applicable to a borderless mobile terminal. In the borderless mobile terminal, because there is no border, compared with the above narrow border, the edge input area of the borderless terminal extends the A area outward to the side edge of the terminal edge, and the edge side is the edge of the terminal housing entity. . It can be seen that the specific application scenarios applicable to the embodiments of the present invention are very broad. The edge input area is an area extending from the screen area to the side of the terminal edge. Since the embodiment of the invention is applicable to narrow The border or the borderless mobile terminal fully utilizes the external size of the mobile terminal, greatly expands the screen size of the mobile terminal, satisfies the user's demand for the large screen, and at the same time, the edge input operation is performed through the gesture calibration operation of the edge input area. diversification. In addition, for the B area, the B area may be a non-touch area or a touch area according to different applicable scenarios, and the B area is collectively referred to as a non-display area. When the B area is the touch area, only the operation mode is different from the A area and the C area. In this case, the B area does not display an image, but a virtual function key, such as a home key, can be set. When the B area is a non-touch area, the physical function button can be set in the B area.

In the embodiment of the present invention, the touch operation in the area A is processed according to the existing normal processing manner. For example, clicking an application icon in the area A opens the application. For the touch operation in the C area, it can be defined as the edge touch processing mode. For example, the bilateral sliding in the C area can be defined, that is, the terminal acceleration is performed. The B area is a non-display area. For example, the B area may be provided with a button area, an earpiece, and the like.

In an embodiment of the invention, the C zone may be divided in a fixed manner or a custom partition. Fixed division, that is, a fixed-length, fixed-bandwidth area is set as the C area in the screen area of the mobile terminal. The C area may include a partial area on the left side of the screen of the mobile terminal and a partial area on the right side, and the position is fixedly disposed on both side edges of the mobile terminal, as shown in FIG. Of course, the C zone can also be divided only at one side edge of the mobile terminal.

Customized division, that is, the number, location and size of the area of the C area, which can be customized, for example, can be set by the user, or the mobile terminal can adjust the number, position and size of the area of the C area according to its own needs. . Generally, the basic graphic design of the C area is a rectangle, and the position and size of the C area can be determined by inputting the coordinates of the two vertices of the diagonal of the graphic.

In order to meet the usage habits of different users for different applications, it is also possible to set multiple sets of C-zone setting schemes applied in different application scenarios. For example, under the system desktop, because the icons occupy more places, the width of the C area on both sides is relatively narrower; when the camera icon is clicked to enter the camera application, the number, position, and size of the C area in the scene can be set. The width of the C zone can be set relatively wide without affecting the focus.

The embodiment of the present invention does not limit the division and setting manner of the C area.

FIG. 2 is a flowchart of a gesture calibration method according to an embodiment of the present invention. The gesture calibration method according to an embodiment of the present invention includes:

S10, the edge gesture calibration mode is activated.

In one embodiment, the mobile terminal device has an edge gesture calibration function. Under normal circumstances, the edge gesture calibration function is in an off state. When the edge gesture calibration mode is activated, the mobile terminal can collect the user's edge gesture. Before the user first uses the gesture operation, the user input gesture information needs to be collected according to the user's operation habits. The user's gesture types include, but are not limited to, the left edge is slid, the right edge is slid, the left edge is slid, the right edge is slid, the bilateral is slid, the bilateral is slid, the terminal screen is held at four corners, and the unilateral back and forth slides. Hand hold and so on.

S20: repeatedly inputting an edge gesture operation according to a preset number of times, and collecting feature values of the edge gesture operation input each time and recording.

In one embodiment, the driver layer retrieves input events generated by the user through the input device, such as input operation events through the touch screen. In an embodiment of the invention, the input events include a normal input event (A zone input event) and an edge input event (C zone input event). Normal input events include input operations such as click, double click, and slide in Area A. Edge input events include sliding on the left edge of Zone C, sliding of the left edge, sliding of the right edge, sliding of the right edge, bilateral sliding, bilateral sliding, holding the four corners of the phone, sliding back and forth, holding one Input operation such as grip, one-handed grip. The application framework layer determines whether the gesture input event is an edge gesture. When the gesture input event is an edge gesture, the judgment result is reported to the application layer, and the application layer collects the user's edge gesture according to the determination result, and collects the edge according to the preset number of times. The eigenvalue of the gesture.

It should be understood that when judging the gesture input event, the following manner may also be adopted: the driver layer acquires an input event generated by the user through the input device, and determines whether the gesture input event is an edge input event (C-region input event), and if the edge gesture input event Then, it is reported to the application framework layer, and then reported to the application layer through the application framework layer. The application layer collects the user's edge gesture according to the judgment result, and collects the feature value of the edge gesture according to the preset number of times.

S30, calculating an average value according to the feature value of the edge gesture operation recorded each time, and obtaining Calibration data.

In an embodiment, when the user starts inputting an edge gesture at the edge of the mobile terminal, acquiring feature values corresponding to each finger in the edge gesture respectively; calculating an average value of the feature values corresponding to each finger, and obtaining the Calibration data for edge gestures. The edge gesture feature value includes: a coordinate value of a contact position of the finger with the edge of the screen, and/or a pressure value and a time value of the finger press. And determining, according to the calibration data of the edge gesture, a hotspot area corresponding to the edge gesture operation, and storing the calibration data in a database to establish a correspondence relationship with the user. When the hot spot area receives the gesture operation signal, the detection condition of the area is lowered, for example, the detection threshold of the detection pixel point or the detection threshold of the pressing force is lowered.

FIG. 3 is a flowchart of a method for calibrating a one-handed gesture according to an embodiment of the present invention. The gesture calibration method of the embodiment of the invention includes:

S11, the edge gesture calibration mode is activated.

In one embodiment, the mobile terminal device has an edge gesture calibration function. Under normal circumstances, the edge gesture calibration function is in an off state. When the edge gesture calibration mode is activated, the mobile terminal can collect the user's edge gesture. Before the user first uses the one-hand gesture operation, the input information of the user gesture needs to be collected according to the user's operating habits.

S12: Acquire a one-handed gesture of the user, and collect the feature value corresponding to the one-handed gesture according to the preset number of times and record.

In one embodiment, taking the one-handed gesture as an example, each time the position coordinates C ₁ (downX ₁ , downY ₁ ), C ₂ (downX ₂ ) of the contact point of each finger of the user with the screen edge of the mobile terminal are separately collected and recorded. , downY ₂ ), C ₃ (downX ₃ , downY ₃ ), C ₄ (downX ₄ , downY ₄ ), C ₅ (downX ₅ , downY ₅ ). FIG. 4 is a schematic diagram of a single-hand grip gesture information collection interface according to an embodiment of the present invention. The mobile terminal presents a setting interface to the user, and the user clicks the setting interface to enter the gesture information setting, and then performs gesture feature value collection according to the user's personal touch area or handheld mode. The shaded portion of the C area (gray area) indicates the touch area when the user holds the one hand. The user performs a one-hand grip operation according to the prompt of the mobile terminal and repeats the steps. Each time the mobile terminal receives the one-hand grip gesture signal, the mobile terminal records the position of the user's finger pressing the touch area each time.

Further, the data of the input information of the gesture also includes: touch point area S _C1 , S _C2 , S _C3 ,

S _C4 , S _C5 , and the pressure value and time value of the pressing of each touch point. Similarly, the calibration data of the touch point area S _Ci can be obtained, and the pressure value and the time value of each touch point are obtained according to the average value of the pressure value and the time value of the finger pressing each time the one hand is held. .

S13: Calculate an average value according to the feature value corresponding to the single-handed gesture of each record, and obtain calibration data of the one-handed hand gesture.

The average value of the user's one-handed finger holding position coordinates

(where i=1, 2, 3, 4, 5, which represents the average value of the coordinates of each finger pressing position), that is, calibration data regarded as a one-handed gripping position of the finger, and the calibration data generated by the gesture serves as the calibration data The user holds the gesture calibration data with one hand, determines the hotspot area of the edge gesture operation according to the calibration data of the one-handed gesture, and stores the calibration data in a database to establish a correspondence with the user. When the hot spot area receives the gesture operation signal, the detection condition of the area is lowered, for example, the detection threshold of the detection pixel point or the detection threshold of the pressing force is lowered.

The embodiment of the present invention does not limit the number of finger pressing regions.

Based on the gesture calibration method provided by the embodiment of the present invention, the edge input area of the mobile terminal is customized according to the gesture operation habit of the user. As shown in FIG. 4, the C area is caused by individual differences or personal operation habits. The position or area of the shaded portion of the area and the pressing force are different. The user's gesture feature or user habit is collected to determine the hot spot area of the user's gesture operation, and the gesture recognition rate is improved by lowering the mobile phone detection condition of the hot spot area, thereby facilitating the user's edge input. Operation to enhance the user experience.

FIG. 5 is a flowchart of a method for calibrating a right edge sliding gesture according to an embodiment of the present invention. The gesture calibration method of the embodiment of the invention includes:

S21, starting the edge gesture calibration mode.

In one embodiment, the mobile terminal device has an edge gesture calibration function, under normal conditions The edge gesture calibration function is in the off state. When the edge gesture calibration mode is activated, the mobile terminal can collect the user's edge gesture. Before the user first uses the right edge sliding gesture operation, the user input gesture information needs to be collected according to the user's operating habits.

S22: Acquire a right edge sliding gesture of the user, and collect feature values corresponding to the right edge sliding gesture according to the preset number of times and record.

In one embodiment, the right edge sliding gesture is taken as an example to illustrate the process of collecting gesture input information. As shown in FIG. 6, the shaded portion of the C area (gray area) represents the sliding area of the user, and the user repeats the right edge sliding operation according to the prompt of the mobile terminal, and each time the mobile terminal receives the right edge sliding gesture signal, each time Record the starting position coordinate A (downX, downY), the ending position coordinate B (currentX, currentY), and the sliding time downTime of the user's finger pressing the touch area.

S23: Calculate an average value according to the feature value corresponding to the right edge sliding gesture of each record, and obtain calibration data of the one-hand holding gesture.

In one embodiment, taking the right edge sliding gesture as an example, the feature values of the right edge sliding gesture are collected: the starting position coordinate A (downX, downY) and the ending position coordinate B (currentX, currentY) of the finger contacting the right edge of the screen. The sliding time downTime calculates the sliding distance L and the sliding time downTime according to the starting position and the ending position of the sliding operation, and obtains the speed v=L/downTime when sliding. Among them, there are two ways to calculate the distance:

or

L=|currentY–downY|.

The average of the coordinates of the starting position of each user's sliding operation

That is, it is regarded as the calibration data of the starting position of the sliding operation. Similarly, the calibration data of the end position of the sliding operation can be obtained, and the average value according to the speed of each sliding operation is obtained.

The calibration data of the sliding speed is obtained. The calibration data generated by the gesture is used as calibration data of the right edge sliding gesture, and the hotspot region of the edge gesture operation is determined according to the calibration data of the right edge sliding gesture, and the calibration data is stored in a database to establish a correspondence with the user. . When the hot spot area receives the gesture operation signal, the detection condition of the area is lowered, for example, the detection threshold of the detection pixel point or the detection threshold of the pressing force is lowered.

Based on the gesture calibration method provided by the embodiment of the present invention, the edge input area of the mobile terminal is customized according to the gesture operation habit of the user. As shown in FIG. 6, the shaded portion of the C area (gray area) may be due to individual user differences. Or the personal operation habits are different. By collecting user gesture features or user habits, the user's gestures are calibrated to facilitate user edge input operations and enhance the user experience.

FIG. 7 is a structural diagram of a gesture calibration apparatus according to an embodiment of the present invention. The gesture calibration apparatus of the embodiment of the invention includes:

The startup module 11 is configured to turn on the edge gesture calibration mode of the mobile terminal.

The obtaining module 12 is configured to repeatedly input an edge gesture operation according to a preset number of times, and collect feature values of the edge gesture operation input each time and record.

In one embodiment, the driver layer retrieves input events generated by the user through the input device, such as input operation events through the touch screen. In an embodiment of the invention, the input events include a normal input event (A zone input event) and an edge input event (C zone input event). Normal input events include input operations such as click, double click, and slide in Area A. Edge input events include sliding on the left edge of Zone C, sliding of the left edge, sliding of the right edge, sliding of the right edge, bilateral sliding, bilateral sliding, holding the four corners of the phone, sliding back and forth, holding one Input operation such as grip, one-handed grip. The application framework layer determines whether the gesture input event is an edge gesture, when When the gesture input event is an edge gesture, the judgment result is reported to the application layer, and the application layer collects the edge gesture of the user according to the determination result, and collects the feature value of the edge gesture according to the preset number of times.

The processing module 13 is configured to calculate an average value according to the feature value of the edge gesture operation recorded each time to obtain calibration data. The processing module 13 further includes:

The first processing unit 130 is configured to acquire feature values corresponding to each finger in the edge gesture respectively.

Taking the one-handed gesture as an example, the data of the input information of the one-handed holding gesture includes the finger pressing position coordinates C ₁ (downX ₁ , downY ₁ ), C ₂ (downX ₂ , downY ₂ ), C ₃ (downX ₃ , downY ₃ ), C ₄ (downX ₄ , downY ₄ ), C ₅ (downX ₅ , downY ₅ ).

The second processing unit 131 is configured to calculate an average value of the feature values corresponding to each finger to obtain calibration data of the edge gesture.

In one embodiment, each time the user holds the average of the finger press position coordinates with one hand

That is, it is regarded as calibration data for holding the finger pressing position with one hand, and the calibration data is stored in the database, and is configured to establish a correspondence between the gesture and the usage mode according to the user's operating habits.

Further, the feature value of the gesture may also include: touch point area S _C1 , S _C2 , S _C3 ,

S _C4 , S _C5 , and the pressing force of each touch point. Similarly, the calibration data of the touch point area S _Ci can be obtained, and the calibration data of the pressing force of each touch point is obtained according to the average value of the finger pressing force when each one hand is held.

The storage module 14 is configured to determine a hotspot area corresponding to the edge gesture operation according to the calibration data of the edge gesture, and store the calibration data in a database to establish a correspondence relationship with the user.

Referring to FIG. 8, a software architecture diagram of a mobile terminal according to an embodiment of the present invention is shown. The software architecture of the mobile terminal of the embodiment of the present invention includes: an input device 201, a driver layer 202, an application framework layer 203, and an application layer 204.

The input device 201 receives the input operation of the user, converts the physical input into an electrical signal TP, and transmits the TP to the driving layer 202; the driving layer 202 analyzes the input position to obtain parameters such as specific coordinates and duration of the touched point, and This parameter is uploaded to the application framework layer 203, and communication between the application framework layer 203 and the driver layer 202 can be implemented through a corresponding interface. The application framework layer 203 receives the parameters reported by the driver layer 202, parses, distinguishes the edge input event and the normal input event, and passes the valid input to the specific application of the application layer 204 to meet the application layer 204 according to different Input operations perform different input operation instructions.

Specifically, the driver layer is configured to obtain an input event generated by the user through the input device, and report to the application framework layer.

The application framework layer is configured to determine whether the input event is an edge input event or a normal input event. If it is a normal input event, the normal input event is processed and recognized, and the recognition result is reported to the application layer; if the edge input event is input to the edge input The event is processed and identified, and the recognition result is reported to the application layer.

The application layer is configured to execute a corresponding input instruction based on the reported recognition result.

The mobile terminal in the embodiment of the present invention avoids the operation of distinguishing the A area and the C area in the application framework layer, and establishes the virtual device in the application framework layer, thereby avoiding the dependence of the driver layer on the hardware of the A area and the C area. .

FIG. 9 is a flowchart of an input processing method according to an embodiment of the present invention, including the following steps:

S1. The driver layer acquires an input event generated by the user through the input device, and reports it to the application framework layer.

Specifically, the input device receives an input operation (ie, an input event) of the user, converts the physical input into an electrical signal, and transmits the electrical signal to the driving layer. In an embodiment of the invention, the input events include an A zone input event and a C zone input event. Input events in Zone A include input operations such as click, double click, and slide in Zone A. The input events in Zone C include sliding on the left edge of Zone C, sliding on the left edge, slipping on the right edge, sliding on the right edge, bilaterally sliding, bilateral sliding, sliding on one side, holding a grip, one hand Hold and other input operations.

The driving layer analyzes the input position according to the received electrical signal to obtain related parameters such as specific coordinates and duration of the touched point. The relevant parameters are reported to the application framework layer.

In addition, if the driver layer reports the input event by using the A protocol, the step S1 further includes:

A number (ID) for distinguishing the finger is assigned to each touch point.

Therefore, if the driver layer reports the input event by using the A protocol, the reported data includes the above related parameters and the number of the touched point.

S2. The application framework layer determines whether the input event is an edge input event or a normal input event. If it is a normal input event, step S3 is performed, and if the edge input event is performed, step S4 is performed.

Specifically, the application framework layer can determine whether it is an edge input event or a normal input event according to the coordinates in the relevant parameters of the input event. First, the horizontal axis coordinate of the touched point is acquired, and then the horizontal axis coordinate (ie, the X-axis coordinate) (x) of the touched point is compared with the C-zone width (Wc) and the touchscreen width (W). If Wc<x<(W-Wc), the touch point is in the A area, and the event is a normal input event; otherwise, the event is an edge input event. If the driving layer uses the B protocol to report the input event, the step S2 further includes: assigning a number (ID) for distinguishing the finger to each touch point; and performing all the element information (coordinates, duration, number, etc.) of the touch point. storage.

Therefore, in the embodiment of the present invention, by setting the touch point number, the finger can be distinguished, and the A protocol and the B protocol are compatible; and all the elements of the touch point (coordinates, numbers, and the like of the touch point) are stored, and the edge input can be subsequently determined (for example, , FIT) provides convenience.

In one embodiment, the edge input event is not the same as the channel used for normal input event reporting. Edge input events use dedicated channels.

S3, the application framework layer processes and identifies the normal input event, and reports the recognition result to the application layer.

S4, the application framework layer processes and recognizes the edge input event, and reports the recognition result to the application layer.

Specifically, the process identification includes: performing process identification according to touch point coordinates, duration, number, and the like of the input operation to determine an input operation. For example, based on the coordinates of the touch point, duration The interval and number can identify whether the input operation of the A area is clicked or swiped, or the input operation of the C area unilateral back and forth.

S5. The application layer executes a corresponding input instruction according to the reported recognition result.

Specifically, the application layer includes applications such as a camera, a gallery, and a lock screen. The input operations in the embodiments of the present invention include an application level and a system level, and the system level gesture processing also classifies it as an application layer. Among them, the application level is the manipulation of the application, for example, on, off, volume control, and the like. The system level is the manipulation of the mobile terminal, for example, power on, acceleration, inter-application switching, global return, and the like.

In one embodiment, the mobile terminal sets and stores input commands corresponding to different input operations, including input commands corresponding to edge input operations and input commands corresponding to normal input operations. The application layer receives the recognition result of the reported edge input event, that is, the corresponding input instruction is invoked according to the edge input operation to respond to the edge input operation; the application layer receives the recognition result of the reported normal input event, that is, according to the normal input operation, the corresponding call is performed. The input command responds to the normal input operation.

It should be understood that the input events of the embodiments of the present invention include input operations only in the A zone, input operations only in the C zone, and input operations simultaneously generated in the A zone and the C zone. Thus, the input command also includes input commands corresponding to the three types of input events. The embodiment of the present invention can implement the combination of the input operations of the A zone and the C zone to control the mobile terminal. For example, the input operation is to simultaneously click the corresponding positions of the A zone and the C zone, and the corresponding input instruction is to close an application, therefore, The application can be closed by simultaneously clicking the input operations of the corresponding positions in the A zone and the C zone.

The mobile terminal of the embodiment of the present invention can be implemented in various forms. For example, the terminal described in the present invention may include a mobile terminal such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA, a PAD, a PMP, a navigation device, and the like, and a fixed terminal such as a digital TV, a desktop computer, or the like. .

Correspondingly, the embodiment of the present invention further provides a user equipment, and FIG. 10 is a schematic diagram of a hardware structure thereof. Referring to FIG. 10, the user equipment 1000 includes a touch screen 100, a controller 200, a storage device 310, a GPS chip 320, a communicator 330, a video processor 340, an audio processor 350, and Button 360, microphone 370, camera 380, speaker 390, and motion sensor 400.

The touch screen 100 may be the A zone, the B zone, and the C zone, or the A zone, the B zone, the C zone, and the T zone as described above. The touch screen 100 can be implemented as various types of displays such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, and a plasma display panel (PDP). The touch screen 100 may include a driving circuit that can be implemented, for example, as an a-si TFT, a low temperature polysilicon (LTPS) TFT, and an organic TFT (OTFT), and a backlight unit.

Meanwhile, the touch screen 100 may include a touch sensor for sensing a touch gesture of a user. The touch sensor can be implemented as various types of sensors, such as a capacitor type, a resistance type, or a piezoelectric type. The capacitance type calculates a touch coordinate value by sensing a micro current excited by a user's body when a portion of the user's body (eg, a user's finger) is touched on the surface of the touch screen coated with the conductive material. According to the type of resistance, the touch screen includes two electrode plates, and the touch coordinate value is calculated by sensing a current flowing when the upper and lower plates at the touch point are in contact when the user touches the screen. Further, when the user device 1000 supports the pen input function, the touch screen 100 may sense a user gesture for using an input device such as a pen other than the user's finger. When the input device is a stylus pen including a coil, the user device 1000 may include a magnetic sensor (not shown) for sensing a magnetic field that changes according to the proximity of the coil within the stylus to the magnetic sensor . Thus, in addition to sensing the touch gesture, the user device 1000 can also sense a proximity gesture, ie, the stylus hover over the user device 1000.

The storage device 310 can store various programs and data required for the operation of the user device 1000. For example, the storage device 310 can store programs and data for constructing various screens to be displayed on the respective areas (for example, the A area, the C area).

The controller 200 displays content on each area of the touch screen 100 by using programs and data stored in the storage device 310.

The controller 200 includes a RAM 210, a ROM 220, a CPU 230, a GPU (Graphics Processing Unit) 240, and a bus 250. The RAM 210, the ROM 220, the CPU 230, and the GPU 240 may be connected to each other through a bus 250.

A processor (CPU) 230 accesses the storage device 310 and performs booting using an operating system (OS) stored in the storage device 310. Moreover, the CPU 230 performs various operations by using various programs, contents, and data stored in the storage device 310.

The ROM 220 stores a set of commands for system startup. When an open command is input and power is supplied, the CPU 230 copies the OS stored in the storage device 310 to the RAM 210 according to the command set stored in the ROM 220, and starts the system by running the OS. When the startup is completed, the CPU 230 copies the various programs stored in the storage device 310 to the RAM 210, and performs various operations by running the copy program in the RAM 210. Specifically, the GPU 240 can generate a screen including various objects such as icons, images, and text by using a calculator (not shown) and a renderer (not shown). The calculator calculates feature values such as coordinate values, format, size, and color, wherein the objects are color-coded according to the layout of the screen, respectively.

The GPS chip 320 is a unit that receives GPS signals from a GPS (Global Positioning System) satellite, and calculates the current location of the user equipment 1000. When the navigation program is used or when the current location of the user is requested, the controller 200 can calculate the location of the user by using the GPS chip 320.

The communicator 330 is a unit that performs communication with various types of external devices in accordance with various types of communication methods. The communicator 330 includes a WiFi chip 331, a Bluetooth chip 332, a wireless communication chip 333, and an NFC chip 334. The controller 200 performs communication with various external devices by using the communicator 330.

The WiFi chip 331 and the Bluetooth chip 332 perform communication according to the WiFi method and the Bluetooth method, respectively. When the WiFi chip 331 or the Bluetooth chip 332 is used, various connection information such as a service set identifier (SSID) and a session key may be first transmitted and received, communication may be connected by using connection information, and each of the communication information may be transmitted and received. Kind of information. The wireless communication chip 333 is a chip that performs communication in accordance with various communication standards such as IEEE, Zigbee, 3G (third generation), 3GPP (3rd Generation Partnership Project), and LTE (Long Term Evolution). The NFC chip 334 is a chip that operates according to an NFC (Near Field Communication) method using a bandwidth of 13.56 MHz among various RF-ID bandwidths, and various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860 ～ 960 trillion Hehe and 2.45 GHz.

The video processor 340 is a unit that processes video data included in content received through the communicator 330 or content stored in the storage device 310. Video processor 340 can perform various image processing for video data, such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion.

The audio processor 350 is a unit that processes audio data included in content received through the communicator 330 or content stored in the storage device 310. The audio processor 350 can perform various processing for audio data, such as decoding, amplification, and noise filtering.

The controller 200 can reproduce the corresponding content by driving the video processor 340 and the audio processor 350 when the reproduction program is run for the multimedia content.

The speaker 390 outputs the audio data generated in the audio processor 350.

The button 360 can be various types of buttons, such as mechanical buttons or touch pads or touch wheels formed on some areas such as the front, side or back of the main outer body of the user device 1000.

The microphone 370 is a unit that receives user voice or other sounds and converts them into audio data. The controller 200 can use user voices input through the microphone 370 during the call process, or convert them into audio data and store them in the storage device 310.

The camera 380 is a unit that captures a still image or a video image according to a user's control. Camera 380 can be implemented as a plurality of units, such as a front camera and a rear camera. As described below, camera 380 can be used as a means of obtaining a user image in an exemplary embodiment that tracks the user's gaze.

When the camera 380 and the microphone 370 are provided, the controller 200 can perform a control operation according to a user's voice input through the microphone 370 or a user motion recognized by the camera 380. Therefore, the user equipment 1000 can operate in an action control mode or a voice control mode. When operating in the motion control mode, the controller 200 photographs the user by activating the camera 380, tracks changes in user actions, and performs corresponding operations. When operating in the voice control mode, the controller 200 can operate in the voice recognition mode to analyze the voice input through the microphone 370 and according to the points The user voice is analyzed to perform control operations.

In the user device 1000 supporting the action control mode or the voice control mode, a voice recognition technology or a motion recognition technology is used in the above various exemplary embodiments. For example, when the user performs an action such as selecting an object marked on the home screen or speaking a voice command corresponding to the object, it may be determined that the corresponding object is selected and a control operation matching the object may be performed.

The motion sensor 400 is a unit that senses the movement of the body of the user device 1000. User device 1000 can be rotated or tilted in various directions. The motion sensor 400 can sense moving features such as a rotational direction, an angle, and a slope by using one or more of various sensors such as a geomagnetic sensor, a gyro sensor, and an acceleration sensor.

Moreover, although not shown in FIG. 10, according to an exemplary embodiment, the user device 1000 may further include a USB port connectable to a USB connector, for connecting like a headphone, a mouse, a LAN, and receiving and processing a digital multimedia broadcast. Various input ports of various external components such as DMB chips of (DMB) signals, and various other sensors.

As described above, the storage device 310 can store various programs.

Based on the user equipment shown in FIG. 10, wherein the touch screen is configured to receive an input operation of the user, and convert the physical input into an electrical signal to generate an input event;

The processor includes: a driving module, an application framework module, and an application module;

The driving module is configured to acquire an input event generated by the user through the input device, and report the event to the application framework module.

The application framework module is configured to determine whether the input event is an edge input event or a normal input event. If the input event is a normal input event, the normal input event is processed and recognized, and the recognition result is reported to the application module; The event is processed and identified by the edge input event, and the recognition result is reported to the application module;

The application module is configured to execute a corresponding input instruction according to the reported recognition result.

It should be understood that the principles and details of the processing of the edge input event and the normal input event by the mobile terminal of the above embodiment are also applicable to the user equipment of the embodiment of the present invention.

The mobile terminal, the input processing method, and the user equipment in the embodiment of the present invention avoid the operation of distinguishing between the A area and the C area in the application framework layer, and the virtual device is established in the application framework layer, thereby avoiding distinguishing the A area in the driving layer. And the dependence of the C area on the hardware; by setting the touch point number, the finger can be distinguished, compatible with the A protocol and the B protocol; and can be integrated into the operating system of the mobile terminal, and can be applied to different hardware, different types of mobile terminals, and can be transplanted. Good; all the elements of the touch point (coordinates, numbers, etc. of the touch point) are stored, which can be easily determined by subsequent judgment of the edge input (for example, FIT).

A person skilled in the art will also appreciate that the elements and algorithm steps of the various examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, in order to clearly illustrate the inter In the above description, the composition and steps of the examples have been generally described in terms of functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods for implementing the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present invention.

The steps of a method or method described in connection with the embodiments disclosed herein may be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of In the storage medium.

The embodiments of the present invention have been described above with reference to the drawings, but the present invention is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative and not restrictive, and those skilled in the art In the light of the present invention, many forms may be made without departing from the spirit and scope of the invention as claimed.

Industrial applicability

The gesture calibration scheme of the embodiment of the present invention repeatedly inputs an edge gesture operation according to a preset number of times by starting the edge gesture calibration mode, and collects the feature value of the edge gesture operation input each time and records, according to each record. The eigenvalues of the edge gesture operation are calculated and averaged to obtain calibration data. The problem that the edge input gesture is not high due to individual differences or personal operation habits is solved, and the gesture can be accurately recognized in the user gesture operation hot zone, which facilitates the user's edge input operation and improves the user experience.

Claims

A gesture calibration method is applied to edge interaction of a mobile terminal, and the method includes:

Initiating an edge gesture calibration mode;

Repeatingly inputting an edge gesture operation according to a preset number of times, collecting feature values of the edge gesture operation input each time and recording;

The average value is calculated from the feature values of the edge gesture operation recorded each time, and calibration data is obtained.
The gesture calibration method according to claim 1, wherein the acquiring the input edge gesture, collecting the feature value corresponding to the edge gesture according to a preset number of times and recording, includes:

The driver layer obtains the gesture input event and reports it to the application framework layer; the application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer collects the edge gesture of the user according to the judgment result, and collects the feature value of the edge gesture according to a preset number of times.
The gesture calibration method according to claim 1, wherein the edge gesture operation is a grip operation, and the feature value of the edge gesture operation includes coordinate values corresponding to the finger, the feature of the edge gesture operation according to each recording The values are calculated and the calibration data is obtained, including:

Obtaining coordinate values corresponding to each finger in the edge gesture operation respectively;

An average value is calculated for the coordinate values corresponding to each finger to obtain calibration data of the edge gesture.
The gesture calibration method according to claim 1, wherein the edge gesture operation is a slide operation, and the feature value of the edge gesture operation includes a start point coordinate value and a stop coordinate value of the slide operation, the edge according to each record The eigenvalues of the gesture operation calculate the average value to obtain calibration data, including:

Obtaining a starting point coordinate value and a ending coordinate value of the finger sliding operation in the edge gesture operation respectively;

The average value is calculated for the starting point coordinate value and the ending coordinate value, respectively, and the calibration data of the edge gesture is obtained.
The gesture calibration method according to claim 3 or 4, wherein the calculating the average value according to the feature value corresponding to the edge gesture of each record, and after obtaining the calibration data, further includes:

And determining, according to the calibration data of the edge gesture, a hotspot area corresponding to the edge gesture operation, and storing the calibration data in a database to establish a correspondence relationship with the user.
A gesture calibration device is applied to an edge interaction of a mobile terminal, and includes: a startup module, an acquisition module, and a processing module, where

The startup module is configured to enable the mobile terminal to turn on an edge gesture calibration mode;

The acquiring module is configured to repeatedly input an edge gesture operation according to a preset number of times, and collect feature values of the edge gesture operation input each time and record;

And a processing module configured to calculate an average value according to the feature value of the edge gesture operation recorded each time to obtain calibration data.
The gesture calibrating apparatus according to claim 6, wherein the obtaining module further comprises:

The driving layer is configured to obtain a gesture input event and report to the application framework layer; the application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer is configured to collect an edge gesture of the user according to the determination result, and collect the feature value of the edge gesture according to a preset number of times.
The gesture calibration apparatus according to claim 6, wherein the processing module further comprises:

a first processing unit, configured to respectively acquire feature values corresponding to each finger in the edge gesture;

The second processing unit is configured to calculate an average value of the feature values corresponding to each finger to obtain calibration data of the edge gesture.
The gesture calibration device of claim 6, wherein the device further comprises:

And a storage module, configured to determine a hotspot area corresponding to the edge gesture operation according to the calibration data of the edge gesture, and store the calibration data in a database to establish a correspondence relationship with the user.
A gesture input processing method is applied to an edge interaction of a mobile terminal, including: an input device, a driver layer, an application framework layer, and an application layer, where

The driver layer acquires a gesture input event generated by the user through the input device, and reports the event to the application framework layer;

The application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer collects the edge gesture of the user according to the judgment result, and collects the input information of the edge gesture according to a preset number of times.
A terminal, the terminal comprising:

a storage medium configured to store computer executable instructions;

a processor configured to execute computer executable instructions stored on the storage medium, the computer executable instructions comprising:

Initiating an edge gesture calibration mode;

Repeatingly inputting an edge gesture operation according to a preset number of times, collecting feature values of the edge gesture operation input each time and recording;

The average value is calculated from the feature values of the edge gesture operation recorded each time, and calibration data is obtained.
The terminal of claim 11, wherein the processor is configured to execute computer executable instructions stored on the storage medium, the computer executable instructions further comprising:

The driver layer obtains the gesture input event and reports it to the application framework layer; the application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer collects the edge gesture of the user according to the judgment result, and collects the feature value of the edge gesture according to a preset number of times.
The terminal of claim 11, wherein the processor is configured to execute computer executable instructions stored on the storage medium, the computer executable instructions further comprising:

Obtaining coordinate values corresponding to each finger in the edge gesture operation respectively;

An average value is calculated for the coordinate values corresponding to each finger to obtain calibration data of the edge gesture.
The terminal of claim 11, wherein the processor is configured to execute computer executable instructions stored on the storage medium, the computer executable instructions further comprising:

Obtaining a starting point coordinate value and a ending coordinate value of the finger sliding operation in the edge gesture operation respectively;

The average value is calculated for the starting point coordinate value and the ending coordinate value, respectively, and the calibration data of the edge gesture is obtained.
The terminal of claim 13 or 14, wherein the processor is configured to execute computer executable instructions stored on the storage medium, the computer executable instructions further comprising:

And determining, according to the calibration data of the edge gesture, a hotspot area corresponding to the edge gesture operation, and storing the calibration data in a database to establish a correspondence relationship with the user.
A computer storage medium storing computer executable instructions, the computer executable instructions comprising:

Initiating an edge gesture calibration mode;

Repeatingly inputting an edge gesture operation according to a preset number of times, collecting feature values of the edge gesture operation input each time and recording;

The average value is calculated from the feature values of the edge gesture operation recorded each time, and calibration data is obtained.
The computer storage medium of claim 16 wherein the computer executable instructions further comprise:

The driver layer obtains the gesture input event and reports it to the application framework layer; the application framework layer determines whether the gesture input event is an edge gesture, and when the gesture input event is an edge gesture, the judgment result is reported to the application layer;

The application layer collects the edge gesture of the user according to the judgment result, and collects the feature value of the edge gesture according to a preset number of times.
The computer storage medium of claim 16 wherein the computer executable instructions further comprise:

Obtaining coordinate values corresponding to each finger in the edge gesture operation respectively;

An average value is calculated for the coordinate values corresponding to each finger to obtain calibration data of the edge gesture.
The computer storage medium of claim 16 wherein the computer executable instructions further comprise:

Obtaining a starting point coordinate value and a ending coordinate value of the finger sliding operation in the edge gesture operation respectively;

The average value is calculated for the starting point coordinate value and the ending coordinate value, respectively, and the calibration data of the edge gesture is obtained.
The computer storage medium of claim 18 or 19, wherein the computer executable instructions further comprise:

And determining, according to the calibration data of the edge gesture, a hotspot area corresponding to the edge gesture operation, and storing the calibration data in a database to establish a correspondence relationship with the user.