WO2014146516A1

WO2014146516A1 - Interactive device and method for left and right hands

Info

Publication number: WO2014146516A1
Application number: PCT/CN2014/071408
Authority: WO
Inventors: 贺真; 刘吉林; 李腾跃
Original assignee: 华为技术有限公司
Priority date: 2013-03-20
Filing date: 2014-01-24
Publication date: 2014-09-25
Also published as: CN103164160A

Abstract

Disclosed are an interactive device and method for left and right hands. The device comprises a detection unit, an analysis unit, a matching unit and a feedback unit which are respectively connected with a bus. The detection unit is used for detecting the range of a touched region on a touch screen; the analysis unit is used for receiving the input of the detection unit, further analyzing an inductor array within the range of touch and analyzing model parameters of both hands during touch; the matching unit is used for matching the analysis result of the analysis unit with a feature library and judging models of both hands of a touch source; and the feedback unit is used for feeding back a corresponding trigger behavior to a user interface in accordance with the matching result and the judging result of the matching unit. The present invention realizes a functional operation through the cooperation of left and right hands, cancels a functional button in the prior art, greatly saves the space of the screen, and also minimizes the moving distance of fingers during the interaction of a user so that the interactive process is more convenient and smoother.

Description

Description: A left and right hand interaction device and method

The present invention relates to the field of human-computer interaction technology, and in particular, to a left-right hand interaction device and method. Background technique

When people operate a mobile phone or tablet device, during the touch operation on the screen, many times they need to be forced to click on the function buttons outside the screen, resulting in the entire interaction process is not smooth and inefficient. In addition, when displaying content on the projection screen, it is often necessary to make comments on the projection screen to prompt other participants, and sometimes even need to go back to the front of the computer to perform related operations, which is very troublesome in both the annotation prompt and the content operation. , affecting the fluency of the interaction process.

In the prior art, by setting a soft button on the screen instead of a function button outside the screen, by clicking a soft button around the UI (User Interface), the pop-up related option is expanded, and then the related option is clicked to implement the original mobile phone or tablet. The function of the function buttons on the device. However, in this way, the soft keys take up a certain amount of UI space and affect the line of sight. If the soft button design is too small, it is difficult to click. If the soft button design is too large, in addition to occupying space, it is easy to cause misoperation. At the same time, the soft keys must be fixed in one position and cannot be used as intended.

In addition, by installing a remote control application on a mobile phone or tablet device, and then connecting to the computer using a connection method such as WIFI, the computer on the projection screen can be controlled, and the projection content can be remotely controlled. This method is still a way of positioning by the mouse, and the actual experience is worse than the mouse, and it is not convenient to perform related operations such as commenting.

Summary of the invention

The embodiments of the present invention provide a left-right hand interaction device and method to solve the technical problems of inconvenient operation, poor fluency, and low efficiency of the existing interaction device and method.

In order to solve the above technical problem, the embodiment of the present invention discloses the following technical solutions:

In a first aspect, a right and left hand interaction device is provided, the device comprising a connection to a bus respectively Measuring unit, analyzing unit, matching unit and feedback unit;

The detecting unit is configured to detect a range of the touched area on the touch screen;

The analyzing unit is configured to receive an input of the detecting unit, further analyze an inductor array within a touch range, and analyze a two-hand model parameter when touched;

The matching unit is configured to match the analysis result of the analysis unit with a feature library, and determine a two-hand model of the touch source;

The feedback unit is configured to feed back the corresponding triggering behavior into the user interface according to the matching and the judgment result of the matching unit.

In conjunction with the first aspect, in a first possible implementation of the first aspect, the apparatus further includes a capture unit coupled to the bus for capturing a two-handed model of the user when interacting with the projection screen.

In conjunction with the first possible implementation of the first aspect, in a second possible implementation of the first aspect, the capturing unit comprises an imaging device and a distance sensor.

With reference to the first aspect, or in combination with any of the possible implementations of the foregoing first aspect, in a third possible implementation manner of the first aspect, the feature library includes specific parameters of ten finger touch behaviors of the hands, The analysis result obtained by the analysis unit is matched with the parameters in the feature library to determine the hand model of the touch source.

The second aspect provides a touch screen-based left-right hand interaction method, where the method specifically includes: detecting a contour, a range, and a touch intensity of the touched area on the touch screen;

According to the above detection result, the sensor array in the touch range is further analyzed, and the area of the touch range, the coordinates of the center point, and the direction angle are analyzed;

Matching the parameters obtained by the above analysis with the parameters in the existing feature database to determine the two-hand model of the touch source;

Based on the results of the matching and judgment, the corresponding triggering behavior is called up and fed back into the user interface.

In conjunction with the second aspect, in a first possible implementation of the second aspect, the contour, the range, and the touch intensity of the touched area on the touch screen are detected by a real-time capture function of the array sensor. With reference to the second aspect, in the second possible implementation manner of the second aspect, the method for further analyzing the sensor array in the touch range and analyzing the area, the center point coordinate, and the direction angle of the touch range is specifically:

By using the touch sensor array of the touch screen itself, when the sensor senses the finger contact, a corresponding signal is generated, and the sensing signal of each sensor is statistically analyzed, and the distribution of the sensor can be obtained, thereby further calculating the touch range. Area, center point coordinates, and direction angle.

With reference to the second aspect, in a third possible implementation manner of the second aspect, the feature library includes specific parameters of ten finger touch behaviors of the hands, by performing the analyzed parameters and the parameters in the existing feature database. Match, determine the two-hand model of the source of the touch.

In a third aspect, a left-right hand interaction method based on a projection screen is provided. The method specifically includes: detecting, by an imaging device and a distance sensor, a position and a touch manner of the touched area on the projection screen;

The user's two-hand model of the touch moment is captured by the camera device and the distance sensor;

According to the captured two-hand model, further analysis is performed to obtain specific two-hand model parameters; the parameters obtained by the above analysis are matched with the parameters in the existing feature database to determine the two-hand model of the touch source;

In conjunction with the third aspect, in a first possible implementation manner of the third aspect, the touch manner includes: clicking, sliding, long pressing, dragging, and double clicking.

In combination with the third aspect, in the second possible implementation manner of the third aspect, the further analysis according to the captured two-hand model is specifically:

Through the imaging device and the distance sensor, the depth model of the two hands is captured based on the depth and the color difference information, the orientation and the depth are acquired by the depth model, and then the specific two-hand model parameters are calculated to determine whether the current touch is the left hand or the right hand.

With reference to the second possible implementation manner of the third aspect, in a third possible implementation manner of the third aspect, the specific two-hand model parameters include: azimuth, posture, angle, depth of the hands, And hierarchical relationships.

In the embodiment of the present invention, the function operation is realized by the cooperation of the right and left hands, the function keys in the prior art are eliminated, the screen space is greatly saved, and the distance of the finger movement during the user interaction is also minimized, so that the interaction process is more convenient and smooth. In the meeting scene, people can freely comment and perform routine operations in front of the projection screen, without having to go back to the computer to operate, creating a new concept of interaction in the computer age.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings to be used in the embodiments or the description of the prior art will be briefly described below, and obviously, the attached in the following description The drawings are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative work.

1 is a structural diagram of a right and left hand interaction device according to an embodiment of the present invention;

2 is a flow chart of a touch screen-based right and left hand interaction method provided by the present invention; FIG. 3 is a schematic diagram of finger touch recognition;

Figure 4 is a schematic diagram of the left and right hand touch interaction behavior;

FIG. 5 is a flowchart of a left-right hand interaction method based on a projection screen provided by the present invention; FIG. 6 is a schematic diagram of interaction behavior of left and right hand projection screens.

detailed description

The technical solutions in the embodiments of the present invention are clearly described in conjunction with the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are Some embodiments, rather than all of the embodiments, are invented. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Example 1

The invention provides a right and left hand interaction device, which can be used for right and left hand touch interaction behavior, the device comprises a detection unit, an analysis unit, a matching unit and a feedback unit respectively connected to the bus; the detection unit is configured to detect the touched area on the touch screen range; The analyzing unit is configured to receive an input of the detecting unit, further analyze an inductor array within a touch range, and analyze a two-hand model parameter when touched;

Wherein, the feature library includes specific parameters of ten finger touch behaviors of the hands, and specific parameters of each finger are different, and the hands of the touch source are determined by matching the analysis result obtained by the analysis unit with the parameters in the feature library. model.

The invention realizes the functional operation by the cooperation of the left and right hands, and cancels the function keys for interaction in the prior art, which greatly saves the screen space and also makes the interaction process more convenient and smooth.

Example 2:

1 is a structural diagram of a right and left hand interaction device according to an embodiment of the present invention. The device can be applied to the right and left hand interaction behavior of the touch screen, and can also be applied to the left and right hand interaction behavior of the projection screen.

The apparatus comprises: a detecting unit 11, an analyzing unit 12, a matching unit 13, a feedback unit 14, and a capturing unit 15, which are connected to each other by a bus.

The detecting unit 11 is configured to detect a range of the touched area on the touch screen, including a specific area and an area outline;

The analyzing unit 12 is configured to receive an input of the detecting unit 11, and further analyze a sensor array within a touch range, and analyze a two-hand model parameter when touched;

The matching unit 13 is configured to match the analysis result of the analysis unit 12 with the feature database, and determine the two-hand model of the touch source;

The feedback unit 14 is configured to feed back the corresponding triggering behavior into the user interface according to the matching and judgment result of the matching unit 12.

The capture unit 15 is for capturing a two-hand model of the user when interacting with the projection screen.

The capturing unit 15 includes an imaging device and a distance sensor.

Wherein, the feature library includes specific parameters of ten finger touch behaviors of the hands, each finger The specific parameters are different, and the hand model of the touch source is determined by matching the analysis result obtained by the analysis unit with the parameters in the feature library.

Based on Embodiment 1, the present embodiment adds a capture unit 15 for capturing the user's two-hand model when interacting with the projection screen. In the conference scene, people can freely perform commentary and normal operations on the projection screen without having to go back to the computer to operate, so that the interaction process based on the projection screen is also very convenient and smooth.

Example 3:

2 is a flow chart of a touch screen-based left and right hand interaction method provided by the present invention, and the method includes:

S10 detects a contour, a range, and a touch intensity of the touched area on the touch screen;

5102. further analyzing, according to the foregoing detection result, the sensor array in the touch range, and analyzing an area, a center point coordinate, and a direction angle of the touch range;

5103. Match the parameters obtained by the foregoing analysis with the parameters in the existing feature database to determine the two-hand model of the touch source;

5104. According to the result of the matching and the judgment, the corresponding triggering behavior is called out and fed back to the user interface.

Wherein, the feature library includes specific parameters of ten finger touch behaviors of the hands, and specific parameters of each finger are different, and the parameters obtained by the analysis are matched with the parameters in the existing feature database to determine the source of the touch. The two-hand model, according to the determined two-hand model, the system will call out the corresponding triggering behavior, and feedback to the user interface to complete the relevant operations.

Example 4:

This embodiment provides a flow chart of a left-right hand interaction method based on a touch screen, including the following steps:

The S10 detects the specific area of the touch, including its contour and range, and the intensity of the touch, through real-time capture of the array sensor.

S102. Further analyzing the sensor array in the touch range according to the detection result, and analyzing the area of the touch range, the center point coordinate, and the direction angle.

Figure 3 is a schematic diagram of finger touch recognition. The capacitive touch screen is microscopically and is a sensor. Integrated on the screen to form a square matrix. When the finger touches an area, the sensor covered by the area can sense the change of the capacitance value, thereby obtaining the arrangement of the sensor, and analyzing the shape of the area of the sensor arrangement to identify whether the touch is left-handed or Right hand, and you can determine which finger is by the size and detail of the area.

By using the touch sensor array of the touch screen itself, if each sensor senses a finger contact, a corresponding signal is generated, and the sensing signal of each sensor is statistically analyzed, thereby obtaining the distribution of the sensor, thereby Further calculate the area of the touch range, the coordinates of the center point, and the angle of the direction.

5103. Match the parameters obtained by the foregoing analysis with the parameters in the existing feature database to determine the two-hand model of the touch source.

Specifically, the specific parameters obtained by the analysis are matched with the parameter intervals in the existing feature database, and the parameter interval in the feature database includes a specific parameter interval of ten finger touch behaviors of the hands, and each finger has a specific parameter interval. Difference, if each parameter obtained belongs to a certain interval, it can be judged that the current touch is specifically derived from the finger of the interval, thereby judging the hand model of the touch source.

The system can determine whether the touch finger is left or right, the index finger or the thumb. The interaction here is mainly divided into two categories: left thumb / other fingers.

When it is detected that the current touch is from the left thumb, different actions trigger different behaviors, as follows:

1) When the left thumb is detected to slide to the left, the system will recognize and return to the previous page (Back);

2) When the left thumb is detected to slide up, the system will recognize and return to the home page (Home);

3) When the left thumb is detected to slide down, the system will recognize and open the option list (Menu);

4) When the left thumb is detected to slide to the right, the system will recognize and advance to the next page ( Forward );

5) When the left thumb is detected and the screen is double clicked, the system recognizes and opens the multitasking menu (Mul t i-Tasking). Figure 4 is a schematic diagram of the left and right hand touch interaction behavior. When the left thumb performs the corresponding operation, the menu such as Menu will automatically pop up at the position of the right thumb. If the right thumb is not detected, it will pop up in the right half.

When it is detected that the current touch is from other fingers, normal touch operation behaviors are performed, such as: sliding, clicking, multi-finger touch, and the like.

In this embodiment, the function is controlled by the left hand. This method of canceling the function key greatly saves the screen space, and at the same time minimizes the distance of the finger movement when the user interacts, making the interaction process more convenient and smooth.

Example 5:

FIG. 5 is a flowchart of a left-right hand interaction method based on a projection screen provided by the present invention, where the method specifically includes:

S201, detecting, by the imaging device and the distance sensor, a position and a touch manner of the touched area on the projection screen, where the touch manner includes: clicking, sliding, long pressing, dragging, and double-clicking;

5202. Capture a two-hand model of the user by touching the camera device and the distance sensor;

5203. Perform further analysis according to the captured two-hand model to obtain a specific two-hand model parameter, where the specific two-hand model parameters include: azimuth, posture, angle, depth, and hierarchical relationship of the hands;

5204. Match the parameters obtained by the foregoing analysis with the parameters in the existing feature database to determine the two-hand model of the touch source;

5205. According to the result of the matching and the judgment, the corresponding triggering behavior is called out and fed back to the user interface.

Wherein, the feature library includes specific parameters of the two-hand touch behavior, and by matching the analyzed parameters with the parameters in the existing feature database, determining the two-hand model of the touch source, according to the determined two-hand model, the system will The corresponding triggering behavior is called up and fed back into the user interface to complete the related operations.

Example 6:

The embodiment provides a flow of a left-right hand interaction method based on a projection screen, including the following steps. Step:

The S20 detects the position and the touch manner of the touched area on the projection screen by the imaging device and the distance sensor, and the touch manner includes: clicking, sliding, long pressing, dragging, and double-clicking.

5202. Capture a two-hand model of the user by touching the camera device and the distance sensor.

5203. Perform further analysis according to the captured two-hand model to obtain specific two-hand model parameters, including: azimuth, posture, angle, depth, and hierarchical relationship of the two hands.

Among them, further analysis based on the captured two-hand model is as follows:

5204. Match the parameters obtained by the foregoing analysis with the parameters in the existing feature database to determine the two-hand model of the touch source.

Specifically, the specific parameters obtained by the analysis are matched with the parameter intervals in the existing feature database, and the parameter interval in the feature library includes a specific parameter interval of the two-hand touch behavior, and if each parameter obtained belongs to a certain interval, It is judged that the current touch is specifically derived from the finger of the section, thereby judging the hand model of the touch source.

Figure 6 is a schematic diagram of the interactive behavior of the left and right hand projection screens. All Menus and related operations are displayed next to the touch points, which is easy to interact and operate.

For example, the specific way of the left and right hand touch interaction based on the projection screen can be:

Left-hand interaction behavior ^ under the mouth:

1) Single-finger click: Select / open and other operations;

2) Single-finger long press: pop-up customized Menu, including related operations;

3) Single-finger swipe: Select multiple content at a time, including multiple documents, multiple texts, etc.

4) Long press and drag: Move the selected content to a reasonable position; 5) Sliding to the left above the two fingers: pin, back;

6) Double click: multi-window switching;

7) Two fingers approach, pull away, rotate: zoom out, zoom in, rotate.

Right hand interaction behavior ^ under the mouth:

8) Single-finger swipe: Comment using the default/current tool;

9) Two-finger sliding: Eraser tool;

10) Single-finger long press: Menu, you can change the pen, change tools, save, delete, etc.

In this embodiment, in the conference scene, people can freely perform commentary and normal operations in front of the projection screen, and do not need to go back to the computer to operate, so that the interaction process based on the projection screen is very convenient and smooth.

Those of ordinary skill in the art will appreciate that various aspects of the present invention, or possible implementations of various aspects, can be embodied as a system, method, or computer program product. Thus, aspects of the invention, or possible implementations of various aspects, may be in the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, etc.), or a combination of software and hardware aspects, They are collectively referred to herein as "circuits," "modules," or "systems." Furthermore, aspects of the invention, or possible implementations of various aspects, may take the form of a computer program product, which is a computer readable program code stored in a computer readable medium.

The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing, such as random access memory (RAM), read only memory (ROM), Erase programmable read-only memory (EPROM or flash memory), optical fiber, portable read-only memory (CD-ROM).

The processor in the computer reads the computer readable program code stored in the computer readable medium, such that the processor can perform the functional actions specified in each step or combination of steps in the flowchart; A device that functions as specified in each block, or combination of blocks.

The computer readable program code can execute entirely on the user's computer, partly on the user's computer, as a separate software package, partly on the user's computer, and partly on the remote meter On a computer, or on a remote computer or server. It should also be noted that in some alternative implementations, the functions noted in the various steps of the flowcharts or in the blocks of the block diagrams may not occur in the order noted in the drawings. For example, two steps, or two blocks shown in succession may be executed substantially concurrently, or the blocks may sometimes be performed in the reverse order, depending upon the functionality involved. The spirit and scope of the invention. Thus, it is intended that the present invention cover the modifications and the modifications of the invention

Claims

Rights request

1. A left- and right-hand interactive device, characterized in that the device includes a detection unit, an analysis unit, a matching unit and a feedback unit that are respectively connected to the bus;

The detection unit is used to detect the range of the touched area on the touch screen;

The analysis unit is used to receive input from the detection unit, and further analyze the sensor array within the touch range, and analyze the parameters of the two-hand model during touch;

The matching unit is used to match the analysis results of the analysis unit with the feature library, and determine the two-hand model of the touch source;

The feedback unit is used to feed back the corresponding trigger behavior to the user interface according to the matching and judgment results of the matching unit.

2. The interactive device according to claim 1, wherein the device further includes a capture unit connected to the bus for capturing the user's hand model when interacting with the projection screen.

3. The interactive device according to claim 2, wherein the capturing unit includes a camera device and a distance sensor.

4. The interactive device according to any one of claims 1 to 3, characterized in that the feature library contains specific parameters of the touch behavior of ten fingers of both hands, and the analysis results obtained by the analysis unit are compared with those in the feature library. The parameters are matched to determine the two-hand model of the touch source.

5. A left-hand interaction method based on a touch screen, characterized in that the method specifically includes: detecting the outline, range, and touch intensity of the touched area on the touch screen;

Based on the above detection results, further analyze the sensor array within the touch range, and analyze the area, center point coordinates and direction angle of the touch range;

Match the parameters obtained from the above analysis with the parameters in the existing feature library to determine the two-hand model of the touch source;

According to the results of matching and judgment, the corresponding trigger behavior is called up and fed back to the user interface.

6. The interaction method according to claim 5, characterized in that, through the implementation of the array sensor Time capture function to detect the outline, range, and touch intensity of the touched area on the touch screen.

7. The interaction method according to claim 5, characterized in that the method of further analyzing the sensor array within the touch range and analyzing the area, center point coordinates and direction angle of the touch range is specifically:

Using the touch sensor array of the touch screen itself, when the sensor senses finger contact, it will generate a corresponding signal. By statistically analyzing the sensing signal of each sensor, the distribution of the sensors can be obtained, and the touch range can be further calculated. area, center point coordinates and direction angle.

8. The interaction method according to claim 5, wherein the feature database contains specific parameters of the touch behavior of ten fingers of both hands, and by matching the analyzed parameters with parameters in the existing feature database, A two-hand model that determines the source of a touch.

9. A left-hand and right-hand interaction method based on a projection screen, characterized in that the method specifically includes:

Detect the position and touch mode of the touched area on the projection screen through camera equipment and distance sensors;

Capture the user's hand model at the moment of touch through camera equipment and distance sensors;

Conduct further analysis based on the captured two-hand model to obtain specific two-hand model parameters; match the parameters obtained from the above analysis with the parameters in the existing feature library to determine the two-hand model from which the touch originated;

10. The interaction method according to claim 9, wherein the touch method includes: click, slide, long press, drag and double click.

11. The interaction method according to claim 9, characterized in that further analysis based on the captured hand model is specifically:

Through the camera equipment and distance sensor, the depth model of both hands is captured based on the depth and color difference information. The orientation and depth are obtained through the depth model, and then the specific model parameters of both hands are calculated to determine whether the left hand or the right hand is currently being touched.

12. The interaction method according to claim 11, wherein the specific hands model parameters include: the orientation, posture, angle, depth, and hierarchical relationship of the hands.